栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

logstash 使用webhdfs 上传文件出现的问题

logstash 使用webhdfs 上传文件出现的问题

记录一次 logstash 6.5.4使用webhdfs插件向hdfs导入数据错误:

[WARN ][logstash.outputs.webhdfs ] Failed to flush outgoing items {:outgoing_count=>1, :exception=>"LogStash::Error", 
:backtrace=>[
"org/logstash/ext/JrubyEventExtLibrary.java:177:in `sprintf'", 
"/export/logstash-6.5.4/vendor/bundle/jruby/2.3.0/gems/logstash-output-webhdfs-3.0.6/lib/logstash/outputs/webhdfs.rb:194:in `block in flush'", 
"org/jruby/RubyArray.java:2481:in `collect'", 
"/export/logstash-6.5.4/vendor/bundle/jruby/2.3.0/gems/logstash-output-webhdfs-3.0.6/lib/logstash/outputs/webhdfs.rb:189:in `flush'", 
"/export/logstash-6.5.4/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/buffer.rb:219:in `block in buffer_flush'",
"org/jruby/RubyHash.java:1343:in `each'", 
"/export/logstash-6.5.4/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/buffer.rb:216:in `buffer_flush'", 
"/export/logstash-6.5.4/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/buffer.rb:159:in `buffer_receive'", 
"/export/logstash-6.5.4/vendor/bundle/jruby/2.3.0/gems/logstash-output-webhdfs-3.0.6/lib/logstash/outputs/webhdfs.rb:182:in `receive'", 
"/export/logstash-6.5.4/logstash-core/lib/logstash/outputs/base.rb:89:in `block in multi_receive'",
"org/jruby/RubyArray.java:1734:in `each'", 
"/export/logstash-6.5.4/logstash-core/lib/logstash/outputs/base.rb:89:in `multi_receive'", 
"org/logstash/config/ir/compiler/OutputStrategyExt.java:114:in `multi_receive'", 
"org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:97:in `multi_receive'", 
"/export/logstash-6.5.4/logstash-core/lib/logstash/pipeline.rb:373:in `block in output_batch'", 
"org/jruby/RubyHash.java:1343:in `each'", "/export/logstash-6.5.4/logstash-core/lib/logstash/pipeline.rb:372:in `output_batch'", 
"/export/logstash-6.5.4/logstash-core/lib/logstash/pipeline.rb:324:in `worker_loop'", 
"/export/logstash-6.5.4/logstash-core/lib/logstash/pipeline.rb:286:in `block in start_workers'"]}

经过分析:
是webhdfs使用不当的原因在官网里的推荐配置是这样

而我没有完全按照该格式

解决方法:

    path最终我替换成了/user/ogstash/logs/output.txt加上dt=
转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/745304.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号