栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

如何查询生产环境logstash报Error parsing json

如何查询生产环境logstash报Error parsing json

1、生产环境有部分报错

信息如下:

2021-10-15T15:49:28,932][WARN ][logstash.filters.json    ][main][7e17a3dc7e2c6df08ed7012ca6bfe17e3277d05d745f2c5bf55d2b01b151e25b] Error parsing json {:source=>"message", :raw=>"查询用户详情接口", :exception=>#}
[2021-10-15T15:49:28,946][ERROR][logstash.outputs.elasticsearch][main][862f1cf74fcefb4312e0b0aa2e9fdf074e2e77c675e5c24e9b7a04d1054f1947] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"%{aName}-log_2021_10", :routing=>nil, :_type=>"_doc"}, #], :response=>{"index"=>{"_index"=>"%{aName}-log_2021_10", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"invalid_index_name_exception", "reason"=>"Invalid index name [%{aName}-log_2021_10], must be lowercase", "index_uuid"=>"_na_", "index"=>"%{aName}-log_2021_10"}}}}
[2021-10-15T15:49:29,446][WARN ][logstash.filters.json    ][main][52e7421840cd9e07ac4081f9b005972562be1954f89c4f4bc5e30c5c19ebae78] Error parsing json {:source=>"message", :raw=>"spAuthAttributes ==========> {"chainCodes":"60d9acb97314488689b899f4495e0857,3d01cb64635b46d4955606e6b356af70,c3283b04c2ad48a09d770b5a96b96a4c,850ff333c6664138b38344a75545ce8e","customLoginStyle":"default","samlEntityID":"portal","securityLevel":"1","spCode":"portal","spLocalLogoutUrl":"","spName":"portal","spOsType":0,"tokenValidPeriod":36000,"url":"https://i.gt.cn","userAttr":""}", :exception=># {"chainCodes":"60d9acb97314488689b899f4495e0857,3d01cb64635b46d4955606e6b356af70,c3283b04c2ad48a09d770b5a96b96a4c,850ff333c6664138b38344a75545ce8e","customLoginStyle":"default","samlEntityID":"portal","securityLevel":"1","spCode":"portal","spLocalLogoutUrl":"","spName":"portal","spOsType":0,"tokenValidPeriod":36000,"url":"https://i.gt.cn","userAttr":""}"; line: 1, column: 18]>}
[2021-10-15T15:49:29,447][WARN ][logstash.filters.json    ][main][52e7421840cd9e07ac4081f9b005972562be1954f89c4f4bc5e30c5c19ebae78] Error parsing json {:source=>"message", :raw=>"redis执行时间:{}1", :exception=>#}

2、处理方法
这是在处理过程中有日志没有按照要进行打印,我们需要新开一个logstash把异常的日志打印出来;
配置如下:

[root@k8s-master1 conf.d]# more aName1.conf 
input{
    kafka{
           bootstrap_servers => "10.152.17.50:9092"
           group_id => "gt-scs-log-new"
           topics => "gt-scs-log"
           consumer_threads => 1
           decorate_events => true
           auto_offset_reset => "latest"
           type => "gt-scs"
    }
}
filter {
    if [type] == "gt-scs" {
      mutate { 
                add_field => { "types" => "%{type}"}
             }
      json {
                source => "message"
          }
      json {
                source => "message"
          }

    }
}
output { 
  if [tags] { 
        stdout {
                codec => "rubydebug"
        }
  }
}
3、结果

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/326844.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号