栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 软件开发 > 后端开发 > Java

线上flink任务重启报错(Hadoop问题)java.lang.NumberFormatException: For input string: “30s“

Java 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

线上flink任务重启报错(Hadoop问题)java.lang.NumberFormatException: For input string: “30s“

线上Flink任务重启时,提交任务失败,具体报错如下:

2021-12-31 18:09:19
java.lang.NumberFormatException: For input string: "30s"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Long.parseLong(Long.java:589)
	at java.lang.Long.parseLong(Long.java:631)
	at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1441)
	at org.apache.hadoop.hdfs.client.impl.DfsClientConf.(DfsClientConf.java:250)
	at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:303)
	at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:287)
	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:159)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3242)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:121)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3291)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3259)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:470)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:356)
	at org.apache.iceberg.hadoop.Util.getFs(Util.java:51)
	at org.apache.iceberg.hadoop.HadoopOutputFile.fromPath(HadoopOutputFile.java:53)
	at org.apache.iceberg.hadoop.HadoopFileIO.newOutputFile(HadoopFileIO.java:64)
	at org.apache.iceberg.io.OutputFileFactory.newOutputFile(OutputFileFactory.java:120)
	at org.apache.iceberg.io.baseTaskWriter$baseRollingWriter.openCurrent(baseTaskWriter.java:264)
	at org.apache.iceberg.io.baseTaskWriter$baseRollingWriter.(baseTaskWriter.java:231)
	at org.apache.iceberg.io.baseTaskWriter$baseRollingWriter.(baseTaskWriter.java:221)
	at org.apache.iceberg.io.baseTaskWriter$RollingFileWriter.(baseTaskWriter.java:303)
	at org.apache.iceberg.io.UnpartitionedWriter.(UnpartitionedWriter.java:33)
	at org.apache.iceberg.flink.sink.RowDataTaskWriterFactory.create(RowDataTaskWriterFactory.java:89)
	at org.apache.iceberg.flink.sink.IcebergStreamWriter.open(IcebergStreamWriter.java:59)
	at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:433)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$2(StreamTask.java:545)
	at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:535)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:575)
	at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573)
	at java.lang.Thread.run(Thread.java:748)

解决方案:
在hdfs-site.xml中新增配置后


    dfs.client.datanode-restart.timeout
    30

问题解决

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/697703.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号