How to solve java.net.SocketTimeoutException:60000millis problem in HDFS

Many HDFS users encounter the following error when DFSClient ready file from a certain Data Node.

 "Java.net.SocketTimeoutException: 60000millis timeout while waiting for channel to be ready for read. ch"

The reason about this error is that the Data Node fails to communicate with DFSClient when DFSClient requests some block from that Data Node, which often occurs when the small cluster conntains too many files and the Data Node with such heavy burden cannot schedules a certain block within the limited time (for instance 60 sec). Then DFSClient waits untill this interval fires, than turn to request the other Data Node containing the same block.

If HDFS is used for in-time service, people wil find the 60 sec timeout is too long and they hope to shorten the interval making the DFSClient switch to other Data Node quickly. When I encounter is problem, I seach the Internet for solution. Depressed, No answeris correct.

Then I debug the source code, finding the timeout value is controlled by the config item "ipc.ping.interval". This item has the default value 60000 (millisec). You can add this config item into you"core-site.xml" file and it will take effert when DFSClient starts.

时间: 2024-10-14 22:19:02

How to solve java.net.SocketTimeoutException:60000millis problem in HDFS的相关文章

java.net.SocketTimeoutException: Read timed out

If you get java.net.SocketTimeoutException: Read timed out exception Try setting own timeout value when constructing JedisPool using the following constructor: JedisPool(GenericObjectPool.Config poolConfig, String host, int port, int timeout) Default

jsoup-提示java.net.SocketTimeoutException:Read timed out

使用Jsoup.connect(url).get()连接某网站时偶尔会出现 java.net.SocketTimeoutException:Read timed out异常. 原因是默认的Socket的延时比较短,而有些网站的响应速度比较慢,所以会发生超时的情况. 解决方法: 链接的时候设定超时时间即可. doc = Jsoup.connect(url).timeout(5000).get(); 5000表示延时时间设置为5s. 测试代码如下: 1,不设定timeout时: package js

phoenix连接hbase数据库,创建二级索引报错:Error: org.apache.phoenix.exception.PhoenixIOException: Failed after attempts=36, exceptions: Tue Mar 06 10:32:02 CST 2018, null, java.net.SocketTimeoutException: callTimeou

环境描述: 操作系统版本:CentOS release 6.5 (Final) 内核版本:2.6.32-431.el6.x86_64 phoenix版本:phoenix-4.10.0 hbase版本:hbase-1.2.6 表SYNC_BUSINESS_INFO_BYDAY数据库量:990万+ 问题描述: 通过phoenix客户端连接hbase数据库,创建二级索引时,报下面的错误: 0: jdbc:phoenix:host-10-191-5-226> create index SYNC_BUSI

在Java中进行api调用时,出现“Exception in thread "main" java.net.SocketTimeoutException: timeout”,“Caused by: java.net.SocketTimeoutException: Read timed out”的错误

在本地使用 OkHttpClient 创建请求客户端,调试api接口时,出现了表示中所述的错误,先贴出完整错误代码 Exception in thread "main" java.net.SocketTimeoutException: timeout at okio.Okio$4.newTimeoutException(Okio.java:230) at okio.AsyncTimeout.exit(AsyncTimeout.java:285) at okio.AsyncTimeout

redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect time out

redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: connect timed outat redis.clients.jedis.Connection.connect(Connection.java:154)at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:83)at redis.clients

sparksql读取hive数据报错:java.lang.RuntimeException: serious problem

问题: Caused by: java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException: Index: 0 at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.hadoop.hive

java.net.SocketTimeoutException: Read timed out 异常排查

问题描述:使用RestTemplate调用接口出现该异常,相关调用代码: ResponseEntity<ResultVO> responseEntity; try { responseEntity = restTemplate.getForEntity( url, ResultVO.class ); } catch (Exception e) { log.error("ops获取项目失败" + e.getMessage()); throw new OAuth2Excepti

spark error Caused by: java.io.NotSerializableException: org.apache.hadoop.hdfs.DistributedFileSystem

序列化问题多事rdd遍历过程中使用了没有序列化的对象. 1.将未序列化的变量定义到rdd遍历内部.如定义入数据库连接池. 2.常量定义里包含了未序列化对象 ,提出去吧 如下常量要放到main里,不能放到rdd的遍历中. val HBASE_TABLE = sparkModel.getUserParamsVal("hbbase_table", "default_table_name") 原文地址:https://www.cnblogs.com/shaozhiqi/p/

Java Secure Socket Extension (JSSE) Reference Guide

Skip to Content Oracle Technology Network Software Downloads Documentation Search Java Secure Socket Extension (JSSE) Reference Guide This guide covers the following topics: Skip Navigation Links Introduction Features and Benefits JSSE Standard API S