Hadoop 压缩文件命令

Hadoop 压缩文件命令

hadoop jar /usr/bin/hadoop/software/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.7.2.4.jar 
        -Dmapred.reduce.tasks=5 
        -Dmapred.output.compress=true 
        -Dmapred.compress.map.output=true 
        -Dmapred.output.compression.codec=org.apache.hadoop.io.compress.BZip2Codec 
        -input /home/wechat_log_2020_02_18.har 
        -output /home/wechat_log_2020_02_18.har.bz2 
        -mapper /bin/cat 
        -reducer /bin/cat 
        -inputformat org.apache.hadoop.mapred.TextInputFormat 
        -outputformat org.apache.hadoop.mapred.TextOutputFormat