## prepare wordcount case
$ cd ~/work/oozie-5.2.1
$ tree oozie
oozie
├── apps
│ └── wordcount
│ ├── job.properties
│ ├── job.properties.orig
│ ├── lib
│ │ └── wordcount.jar
│ ├── workflow.xml
│ └── workflow.xml.orig
└── data
└── wordcount
└── input
└── in.txt
$ cat oozie/apps/wordcount/job.properties
nameNode=hdfs://localhost:9000
resourceManager=localhost:8032
queueName=default
oozieRoot=user/${user.name}/oozie
oozie.wf.application.path=${nameNode}/${oozieRoot}/apps/wordcount/workflow.xml
inputDir=data/wordcount/input
outputDir=data/wordcount/output
$ cat oozie/apps/wordcount/workflow.xml
${resourceManager} ${nameNode} mapred.mapper.new-api true mapred.reducer.new-api true mapreduce.job.queuename ${queueName} mapreduce.job.map.class WordCount$TokenizerMapper mapreduce.job.reduce.class WordCount$IntSumReducer mapreduce.job.output.key.class org.apache.hadoop.io.Text mapreduce.job.output.value.class org.apache.hadoop.io.IntWritable mapreduce.input.fileinputformat.inputdir ${nameNode}/${oozieRoot}/${inputDir} mapreduce.output.fileoutputformat.outputdir ${nameNode}/${oozieRoot}/${outputDir} Map/Reduce failed, error message[${wf:errorMessage(wf:lastErrorNode())}]
Note: all key / value pairs in workflow.xml can be found from
http://localhost:8088/cluster -> Application's History -> Configuration
$ cat oozie/data/wordcount/input/in.txt
hadoop mapreduce hdfs hbase hdfs spark hive hcatalog sqoop flume kafka oozie hue
## upload case
$ hdfs dfs -put oozie /user/sun_xo/oozie
$ hdfs dfs -rm -f -r /user/sun_xo/oozie/data/wordcount/output
## start oozie
$ cd ~/work/oozie-5.2.1
$ bin/oozied.sh start
$ export OOZIE_URL=http://localhost:11000/oozie
$ bin/oozie job -config oozie/apps/wordcount/job.properties -run
job: 0000000-220523112321053-oozie-sun_-W
$ bin/oozie job -info 0000000-220523112321053-oozie-sun_-W
ob ID : 0000000-220602202911687-oozie-sun_-W ------------------------------------------------------------------------------------------------------------------------------------ Workflow Name : wordcount-wf App Path : hdfs://localhost:9000/user/sun_xo/oozie/apps/wordcount/workflow.xml Status : SUCCEEDED Run : 0 User : sun_xo Group : - Created : 2022-06-02 12:30 GMT Started : 2022-06-02 12:30 GMT Last Modified : 2022-06-02 12:30 GMT Ended : 2022-06-02 12:30 GMT CoordAction ID: - Actions ------------------------------------------------------------------------------------------------------------------------------------ ID Status Ext ID Ext Status Err Code ------------------------------------------------------------------------------------------------------------------------------------ 0000000-220602202911687-oozie-sun_-W@:start: OK - OK - ------------------------------------------------------------------------------------------------------------------------------------ 0000000-220602202911687-oozie-sun_-W@wordcount-node OK application_1653709899071_0004SUCCEEDED - ------------------------------------------------------------------------------------------------------------------------------------ 0000000-220602202911687-oozie-sun_-W@end OK - OK - ------------------------------------------------------------------------------------------------------------------------------------
$ hdfs dfs -cat "/user/sun_xo/oozie/data/wordcount/output/*"
flume 1 hadoop 1 hbase 1 hcatalog 1 hdfs 2 hive 1 hue 1 kafka 1 mapreduce 1 oozie 1 spark 1 sqoop 1
reference: 搭建Ozzie环境
reference: 搭建map-reduce开发环境



