hadoop高可用集群部署参考: Hadoop3.X分布式高可用集群部署
1.1 版本说明| 软件 | 版本 |
|---|---|
| 操作系统 | CentOS Linux release 7.8.2003 (Core) |
| JAVA | jdk-8u271-linux-x64 |
| Hadoop | hadoop-3.2.2 |
| Hbase | hbase-2.3.6 |
| hostname | IP | 组件 | ||||
|---|---|---|---|---|---|---|
| master | 172.16.20.200 | NameNode | Hbase-HMaster | |||
| secondmaster | 172.16.20.201 | NameNode | Hbase-HMaster | |||
| slave1 | 172.16.20.202 | Zookeeper | DataNode | NodeManage | Hbase-HRegionServer | |
| slave2 | 172.16.20.203 | Zookeeper | DataNode | NodeManage | Hbase-HRegionServer | |
| slave3 | 172.16.20.204 | Zookeeper | DataNode | NodeManage | Hbase-HRegionServer |
下载地址: https://archive.apache.org/dist/hbase/2.3.6/hbase-2.3.6-bin.tar.gz
tar -zxf hbase-2.3.6-bin.tar.gz -C /opt/hadoop/ ln -s /opt/hadoop/hbase-2.3.6 /usr/local/hbase
各节点配置环境变量, /etc/profie下加入
cat >> /etc/profile << 'EOF' #Hbase Hbase_HOME=/usr/local/hbase PATH=$Hbase_HOME/bin:$PATH export PATH Hbase_HOME EOF source /etc/profile2.2 修改配置
cd $Hbase_HOME/confhbase-env.sh
cat > hbase-env.sh << 'EOF' export JAVA_HOME=/usr/java/jdk1.8/jdk1.8.0_271 export Hbase_MANAGES_ZK=false EOF
hbase-site.xmlexport Hbase_MANAGES_ZK=false #指定不使用hbase自带的zookeeper
cat > hbase-site.xml << 'EOF'EOF hbase.rootdir hdfs://mycluster/hbase hbase.cluster.distributed true hbase.zookeeper.quorum slave1:2181,slave2:2181,slave3:2181
regionservers指定访问端口
hbase.master.info.port 16010
cat > regionservers << EOF master secondmaster slave1 slave2 slave3 EOFbackup-masters
cat > backup-masters << EOF secondmaster EOF拷贝hadoop核心配置
cp $HADOOP_HOME/etc/hadoop/hdfs-site.xml $Hbase_HOME/conf/cp $HADOOP_HOME/etc/hadoop/core-site.xml $Hbase_HOME/conf/2.3 同步配置
rsync -av /opt/hadoop/hbase-2.3.6 root@sm:/opt/hadoop/rsync -av /opt/hadoop/hbase-2.3.6 root@s1:/opt/hadoop/rsync -av /opt/hadoop/hbase-2.3.6 root@s2:/opt/hadoop/rsync -av /opt/hadoop/hbase-2.3.6 root@s3:/opt/hadoop/
并在节点上创建软连接
ln -s /opt/hadoop/hbase-2.3.6 /usr/local/hbase2.4 启动
master节点集群方式启/停hbase集群
$Hbase_HOME/bin/start-hbase.sh$Hbase_HOME/bin/stop-hbase.sh
四、验证启动状态 5.1 命令查看单节点手动启动方式
hbase-daemon.sh start masterhbase-daemon.sh start regionserver
查看zk数据
zkCli.shls /hbase[backup-masters, draining, hbaseid, master, master-maintenance, rs, splitWAL, table]
JPS查看
master节点
// JPS命令查看15928 HMaster
slave节点
// JPS命令查看14425 HRegionServer5.2 web页面查看
访问master和secondmaster的16010端口, 查看hbase主页
五、高可用验证停止master节点Master进程, 访问secondmaster的spark页面,查看状态是否切换为ALIVE



