栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 软件开发 > 后端开发 > Java

搭建Ozzie环境

Java 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

搭建Ozzie环境

1) build
## download oozie-5.2.1.tar.gz from http://archive.apache.org/dist/
## unpack to ~/work/oozie-src-5.2.1
$ cd ~/work/oozie-src-5.2.1
$ diff pom.xml.orig pom.xml         # modify according to hadoop version
83c83
<         2.6.0
---
>         2.8.0
$ bin/mkdistro.sh -Puber -DskipTests
## the tar ball distro/target/oozie-5.2.1-distro.tar.gz should be create

2) install
$ cd ~/work
$ tar xvf oozie-src-5.2.1/distro/target/oozie-5.2.1-distro.tar.gz
$ cd oozie-5.2.1
## download ExtJS 2.2 library to enable Oozie web console
## http://archive.cloudera.com/gplextras/misc/ext-2.2.zip
$ mkdir libext
$ cp ../downloads/ext-2.2.zip libext

3) configure
## modify oozie-site.xml
$ diff conf/oozie-site.xml.orig conf/oozie-site.xml
65c65,68
>    
>         oozie.service.HadoopAccessorService.hadoop.configurations
>         *=/Users/sun_xo/work/hadoop/etc/hadoop/
>    

## modify core-site.xml and restart hadoop
$ cd ~/work/hadoop
$ mr.sh stop yarn
$ diff -r myconf/core-site.xml etc/hadoop/core-site.xml
27a28,36
>    
>    
>         hadoop.proxyuser.sun_xo.hosts
>         localhost
>    

>    
>         hadoop.proxyuser.sun_xo.groups
>         *
>    

$ mr.sh start yarn

## upload sharelib to hdfs
$ cd ~/work/oozie-5.2.1
$ bin/oozie-setup.sh sharelib create -fs hdfs://localhost:9000 -locallib oozie-sharelib-5.2.1.tar.gz
## check result on http://localhost:50070/explorer.html#/user/sun_xo/share

## create oozie db (in derby as default)
$ cd ~/work/oozie-5.2.1
$ bin/ooziedb.sh create -sqlfile oozie.sql -run

4) start oozie
$ bin/oozied.sh start       # check by jps - check EmbeddedOozieServer
$ bin/oozie admin -oozie http://localhost:11000/oozie -status
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
System mode: NORMAL
## check result on http://localhost:11000/oozie/

5) run case
## prepare examples
$ cd ~/work/oozie-5.2.1
$ tar xvf oozie-examples.tar.gz
$ hdfs dfs -put examples /user/sun_xo/examples  # upload examples
$ cd examples/apps/map-reduce                   # modify job.properties
$ diff job.properties.orig job.properties
19c19
< nameNode=hdfs://localhost:8020
---
> nameNode=hdfs://localhost:9000

## run a map-reduce case in examples
$ cd ~/work/oozie-5.2.1
$ export OOZIE_URL=http://localhost:11000/oozie
$ bin/oozie job -config examples/apps/map-reduce/job.properties -run
job: 0000000-220523112321053-oozie-sun_-W
## check result by job id or check on http://localhost:11000/oozie/
$ bin/oozie job -info 0000000-220523112321053-oozie-sun_-W
$ hdfs dfs -cat "/user/sun_xo/examples/output-data/map-reduce/*"

6) stop oozie
$ bin/oozied.sh stop        # check by jps

reference: https://oozie.apache.org/docs/5.2.1/index.html

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/946639.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号