hadoop、hbase、zookeeper是大数据里面三个重要的产品，关于这三个产品的介绍，各种书籍和网站介绍得很多，在这里不再赘述，仅介绍在ubunt下如何安装和配置这三种产品。
第一步：配置虚拟机为桥接模式
第二步：安装openssh-server，并且登录不需要
| /home/jerry/>apt-get install openssh-server #安装openssh-server /home/jerry/>ssh localhost #登录本机，需要密码 /home/jerry/>exit# 退出刚才的 ssh localhost /home/jerry/>cd ~/.ssh/# 若没有该目录，请先执行一次ssh localhost /home/jerry/>ssh-keygen -t rsa# 会有提示，都按回车就可以 /home/jerry/>cat ./id_rsa.pub >> ./authorized_keys # 加入授权 /home/jerry/>ssh localhost#不需要密码 |
第三步：安装JDK 1.7以上
| /home/jerry/>tar -xzf jdk-8u161-linux-x64.tar.gz -C /usr/local /home/jerry/>cd /usr/local /usr/local/>mv jdk1.8.0_161/ java /usr/local/>gedit ~/.bashrc |
export JAVA_HOME=/usr/local/java export JRE_HOME=$JAVA_HOME/jre export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib export PATH=$PATH:$JAVA_HOME/bin
| /usr/local/>source ~/.bashrc /usr/local/>java -version |
出现版本信息，说明配置正确。
第四步：安装hadoop
| /home/jerry/>tar -xzf hadoop-2.6.4.tar.gz -C /usr/local /home/jerry/>cd /usr/local /usr/local/>mv hadoop-2.6.4/ hadoop /usr/local/>chown -R root ./hadoop /usr/local/>gedit ~/.bashrc |
export HADOOP_HOME=/usr/local/hadoop export PATH=$PATH:$HADOOP_HOME/bin export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib/native" export JAVA_LIBRAY_PATH=$HADOOP_HOME/lib/native export LD_LIBRARY_PATH=$JAVA_LIBRARY_PATH
| /usr/local/>source ~/.bashrc /usr/local/>hadoop version |
#查看hsddop版本有内容，说明配置正确
| /usr/local/>gedit /usr/local/hadoop/etc/hadoop/core-site.xml |
fs.defualtFS hdfs://localhost:9000 hadoop.tmp.dir file:/usr/local/hadoop/tmp Abase for other temporary directories.
| /usr/local/>gedit /usr/local/hadoop/etc/hadoop/hdfs-site.xml |
dfs.replication 1 dfs.namenode.name.dir file:/usr/local/hadoop/tmp/dfs/name dfs.datanode.data.dir file:/usr/local/hadoop/tmp/dfs/data dfs.namenode.http-address 0.0.0.0:50070
dfs.namenode.name.dir：namenode位置dfs.namenode.data.dir：datanode位置dfs.namenode.http-address：hadoop WEB端口
| gedit /usr/local/hadoop/etc/hadoop/hdoop-env.sh |
将语句
export JAVA_HOME=$JAVA_HOME
修改为
export JAVA_HOME=/usr/java/jdk1.8.0_101
hadoop WEB 页面：http://localhost:50070
| /usr/local/>cd hadoop /usr/local/hadoop/>hdfs namenode -format /usr/local/hadoop/>sbin/start-dfs.sh /usr/local/hadoop/>cp etc/hadoop/mapred-site.xml.template etc/hadoop/mapred-site.xml /usr/local/hadoop/>gedit /usr/local/hadoop/etc/hadoop/yarn-site.xml |
yarn.nodemanager.aux-server mapreduce_shuffle
| /usr/local/hadoop/>sbin/start-dfs.sh(./sbin/stop-dfs.sh) /usr/local/hadoop/>sbin/start-yarn.sh(./sbin/stop-yarn.sh) /usr/local/hadoop/> |
hadoop安全模式，比如hadoop非正常退出，会进入安全模式
$hdfs dfsadmin -safemode get|wait|entry|leave|wait
get：获得当前是否进入安全模式wait：等待进入安全模式entry：强制进入安全模式leave：强制离开安全模式
第五步：安装Hbase
| /home/jerry/>tar -xzf hbase-2.2.4-bin.tar.gz -C /usr/local /home/jerry/>cd /usr/local /usr/local/>mv hbase-2.2.4/ hbase /usr/local/>gedit ~/.bashrc |
export Hbase_HOME=/usr/local/hbase export PATH=$PATH:$Hbase_HOME/bin
| /usr/local/>source ~/.bashrc /usr/local/>chown -R root ./habase /usr/local/>gedit hbase/conf/hbase-site.xml |
hbase.rootdir hdfs://localhost:9000/hbase hbase.cluster.distributed true hbase.unsafe.stream.capability.enforce false hbase.zookeeper.property.dataDir /usr/local/zookeeper
hbase.rootdir：hbase在HDFS的位置，端口号一定要与/usr/local/hadoop/etc/hadoop/core-site.xml保持一致。hbase.cluster.distributed：是否为分布式，在这里为伪分布式hbase.unsafe.stream.capability.enforce：在分布式情况下, 一定设置为false（hbase的一个bug）hbase.zookeeper.property.dataDir：zookeeper的位置。
| /usr/local/>hbase version #有结果说明配置正确 /usr/local/>hbase-daemon.sh start master /usr/local/>start-hbase.sh #用于启动Hbase，关闭Hbase用stop-hbase.sh 速度相当慢，./bin/hdfs namenode -format，但是数据全部丢失 /usr/local/>hbase shell#用于打开shell命令行模式 |
Hbase WEB 页面http://localhost:16010
第六步：安装zookeeperz文件
| /home/jerry/>tar -zxvf zookeeper-3.4.11.tar.gz /home/jerry/>mv zookeeper-3.4.11 /usr/local/zookeeper /home/jerry/>cd /usr/local/zookeeper/conf //切换到该目录 /usr/local/zookeeper/conf >cp zoo_sample.cfg zoo.cfg//复制一份取名为zoo.cfg /usr/local/zookeeper/conf >gedit zoo.cfg |
... syncLimit=5 # the directory where the snapshot is stored. # do not use /tmp for storage, /tmp here is just # example sakes. dataDir=/tmp/zookeeper dataLogDir=/tmp/zookeeper/logs # the port at which the clients will connect clientPort=2181 ...
确保/tmp/zookeeper，/tmp/zookeeper/logs存在
| /usr/local/>source ~/.bashrc /usr/local/>chown -R root ./habase /usr/local/>hbase version /usr/local/zookeeper/conf >gedit ~/.bashrc |
export ZOOKEEPER_HOME=/usr/local/zookeeper export PATH=$PATH:$ ZOOKEEPER_HOME/bin
| /usr/local/zookeeper/conf >source ~/.bashrc cd ../bin/ /usr/local/zookeeper/bin>zkServer.sh start |
在实际工作中，建议启动服务以后确定启动没有问题再关闭，最后统一启动，启动顺序为：
haddoophbasezookeep
启动完毕
| jps 114865 NameNode 112467 JobHistoryServer 116114 HRegionServer 111637 ResourceManager 115271 SecondaryNameNode 43386 Jps 115994 HMaster 115932 HQuorumPeer 115023 DataNode 111774 NodeManager |
几种错误现象。
1，Nonode for /hbase/master
hbase(main):005:0> status
ERROR: KeeperErrorCode = Nonode for /hbase/master
For usage try 'help "status"'
Took 8.1764 seconds
原因
HMaster进程没有打开
2，启动hadoopmei没有datanode
cat /usr/local/hadoop/tmp/dfs/name/current/VERSION
记住clusterID
edit /usr/local/hadoop/tmp/dfs/data/current/VERSION
修改clusterID为刚才记录的name里面的clusterID
重新启动hadoop
3，启动hbase后不久HMaster就关闭
如果使用后hbase自带的zookeeper，建议单独安装zookeeper，否则到zookeeper主目录下删除hbase下所有内容
