栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

flink 异常报错 The total number of network buffers is currently set to 22773 of 32768 bytes each

flink 异常报错 The total number of network buffers is currently set to 22773 of 32768 bytes each

flink job正常运行了好久,突然一天发生的异常,查看据异常信息,发现是内存模型出现了问题,根据日志提示修改了几个值,从而解决了问题

报错信息

java.io.IOException: Timeout triggered when requesting exclusive buffers: The total number of network buffers is currently set to 22773 of 32768
bytes each.You can increase this number by setting the configuration keys 'taskmanager.memory.network.fraction' ,
'taskmanager.memory.network.min',and 'taskmanager.memory.network.max', or you may increase the timeout which is 30000ms by setting the key
'taskmanager.network.memory.exclusive-buffers-request-t imeout-ms'


解决办法

StreamExecutionEnvironment environment = StreamExecutionEnvironment.getExecutionEnvironment();
Configuration conf = new Configuration();
conf.setString("taskmanager.memory.network.min", "128MB");
conf.setString("taskmanager.memory.network.max", "2GB");
conf.setString("taskmanager.network.memory.exclusive-buffers-request-timeout-ms", "30000ms");
environment.getConfig().setGlobalJobParameters(conf);

taskmanager.memory.network.min:网络缓存的最小值,默认64MB;
taskmanager.memory.network.max:网络缓存的最大值,默认1GB;
taskmanager.memory.network.fraction:网络缓存占Flink总内存taskmanager.memory.flink.size的比例,默认值0.1。上面两个设置过了,本项会被覆盖,两者设置其一即可。

分别给它提高了一倍
并设置了超时时间为30000ms

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/761035.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号