栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

HDFS报错datanode.DataNode (DataXceiverServer.java:run(168))

HDFS报错datanode.DataNode (DataXceiverServer.java:run(168))

HDFS频繁报如下错误:

2022-02-28 00:15:39,115 WARN  datanode.DataNode (DataXceiverServer.java:run(168)) - hadoop-04:50010:DataXceiverServer:
java.io.IOException: Xceiver count 202 exceeds the limit of concurrent xcievers: 200
        at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:150)
        at java.lang.Thread.run(Thread.java:748)
2022-02-28 00:15:39,217 WARN  datanode.DataNode (DataXceiverServer.java:run(168)) - hadoop-04:50010:DataXceiverServer:
java.io.IOException: Xceiver count 202 exceeds the limit of concurrent xcievers: 200
        at org.apache.hadoop.hdfs.server.datanode.DataXceiverServer.run(DataXceiverServer.java:150)
        at java.lang.Thread.run(Thread.java:748)
2022-02-28 00:15:39,259 INFO  DataNode.clienttrace (DataXceiver.java:requestShortCircuitFds(425)) - src: 127.0.0.1, dest: 127.0.0.1, op: REQUEST_SHORT_CIRCUIT_FDS, blockid: 1109394163, srvID: 2a966b4d-4941-4e55-9961-1a7f46a26b80, success: true
2022-02-28 00:15:39,294 INFO  DataNode.clienttrace (BlockReceiver.java:finalizeBlock(1533)) - src: /172.25.4.65:18826, dest: /172.25.4.65:50010, bytes: 134217728, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_-44093795_1, offset: 0, srvID: 2a966b4d-4941-4e55-9961-1a7f46a26b80, blockid: BP-734843239-172.25.4.62-1610084212828:blk_1109394784_35656232, duration(ns): 333214502
2022-02-28 00:15:39,294 INFO  datanode.DataNode (BlockReceiver.java:run(1506)) - PacketResponder: BP-734843239-172.25.4.62-1610084212828:blk_1109394784_35656232, type=HAS_DOWNSTREAM_IN_PIPELINE, downstreams=2:[172.25.4.64:50010, 172.25.4.63:50010] terminating
2022-02-28 00:15:39,297 INFO  datanode.DataNode (DataXceiver.java:writeBlock(738)) - Receiving BP-734843239-172.25.4.62-1610084212828:blk_1109394792_35656240 src: /172.25.4.65:18832 dest: /172.25.4.65:50010

 

 解决方法:

Hadoop HDFS DataNode 在任何时候都可以提供文件数量的上限。上限由 dfs.datanode.max.transfer.threads属性控制。修改hdfs-site.xml配置为至少4096(HDP集群默认200)。

 

 

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/751899.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号