rn测试
pipeline配置文件:
pipelines:
- name: local
sources:
- type: file
name: demo
lineEnd:
type: carriage_return_line_feed
value: "rn"
charset: utf-8
paths:
- /home/zhanglei/*.log
fields:
topic: "loggie"
interceptors:
- type: iconv
charset: "utf-8"
sink:
type: dev
printEvents: true
codec:
pretty: true
2、测试脚本
3、测试结果输出正常
二、测试编码单行换行符 1、测试场景gbk 编码
rn测试
pipeline配置文件:
pipelines: - name: local sources: - type: file name: demo lineEnd: type: carriage_return_line_feed value: "rn" charset: gbk paths: - /home/zhanglei/*.log fields: topic: "loggie" interceptors: - type: iconv charset: "gbk" sink: type: dev printEvents: true codec: pretty: true2.测试脚本3.测试结果 三、测试非编码 auto 换行符 1、测试场景n测试
pipeline配置文件:
pipelines: - name: local sources: - type: file name: demo lineEnd: type: auto paths: - /home/zhanglei/*.log fields: topic: "loggie" interceptors: - type: iconv sink: type: dev printEvents: true codec: pretty: true2.测试脚本3.测试结果 四、测试编码 auto 换行符 1、测试场景gbk 编码
n测试
pipeline配置文件:
pipelines: - name: local sources: - type: file name: demo lineEnd: type: auto value: "rn" charset: gbk paths: - /home/zhanglei/*.log fields: topic: "loggie" interceptors: - type: iconv charset: "gbk" sink: type: dev printEvents: true codec: pretty: true2.测试脚本3.测试结果 五、测试非编码自定义换行符(特殊字符) 1、测试场景pipeline:
pipelines: - name: local sources: - type: file name: demo lineEnd: type: custome value: "rn" paths: - /home/zhanglei/*.log fields: topic: "loggie" interceptors: - type: iconv sink: type: dev printEvents: true codec: pretty: true2、测试结果 3、结论custom 配置不能解决 特殊ascii码问题
特殊字符使用
lineTerminators = map[string]LineTerminator{ "auto": AutoLineTerminator, "line_feed": LineFeed, "vertical_tab": VerticalTab, "form_feed": FormFeed, "carriage_return": CarriageReturn, "carriage_return_line_feed": CarriageReturnLineFeed, "next_line": NextLine, "line_separator": LineSeparator, "paragraph_separator": ParagraphSeparator, "null_terminator": NullTerminator, }六、测试非编码自定义换行符(非特殊字符) 1、测试场景pipeline:
pipelines: - name: local sources: - type: file name: demo lineEnd: type: custom value: "的" paths: - /home/zhanglei/*.log fields: topic: "loggie" interceptors: - type: iconv sink: type: dev printEvents: true codec: pretty: true2、测试结果 3、结论正常
七、测试编码自定义换行符(特殊字符) 1、测试场景pipeline:
pipelines: - name: local sources: - type: file name: demo lineEnd: type: custom value: "rn" charset: gbk paths: - /home/zhanglei/*.log fields: topic: "loggie" interceptors: - type: iconv charset: gbk sink: type: dev printEvents: true codec: pretty: true2、测试结果 3、结论custom 配置不能解决 特殊ascii码问题
但是可见字符都是正常的,如果出现不可见字符请使用
lineTerminators = map[string]LineTerminator{ "auto": AutoLineTerminator, "line_feed": LineFeed, "vertical_tab": VerticalTab, "form_feed": FormFeed, "carriage_return": CarriageReturn, "carriage_return_line_feed": CarriageReturnLineFeed, "next_line": NextLine, "line_separator": LineSeparator, "paragraph_separator": ParagraphSeparator, "null_terminator": NullTerminator, }解码后出现bug
八、测试编码自定义换行符(非特殊字符) 1、测试场景pipeline:
pipelines: - name: local sources: - type: file name: demo lineEnd: type: custom value: "rn" charset: gbk paths: - /home/zhanglei/*.log fields: topic: "loggie" interceptors: - type: iconv charset: gbk sink: type: dev printEvents: true codec: pretty: true2、测试结果 3、结论非特殊字符解码正常



