YARN-HA 配置

• YARN-HA 工作机制
• 配置 YARN-HA 集群
• HADOOP HA 的最终规划

1）官方文档：
http://hadoop.apache.org/docs/r3.1.3/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html
2）YARN-HA 工作机制

1）环境准备
（1）修改 IP
（2）修改主机名及主机名和 IP 地址的映射
（3）关闭防火墙
（4）ssh 免密登录
（5）安装 JDK，配置环境变量等
（6）配置 Zookeeper 集群

2）规划集群

3）核心问题
a .如果当前 active rm 挂了，其他 rm 怎么将其他 standby rm 上位核心原理跟 hdfs 一样，利用了 zk 的临时节点
b. 当前 rm 上有很多的计算程序在等待运行,其他的 rm 怎么将这些程序接手过来接着跑 rm 会将当前的所有计算程序的状态存储在 zk 中,其他 rm 上位后会去读取，然后接着跑

4）具体配置
（1）yarn-site.xml

 
   
    yarn.nodemanager.aux-services  
    mapreduce_shuffle 
    
    
   
    yarn.resourcemanager.ha.enabled  
    true 
    
    
   
    yarn.resourcemanager.cluster-id  
    cluster-yarn1 
    
    
   
    yarn.resourcemanager.ha.rm-ids  
    rm1,rm2,rm3 
    
    
    
   
    yarn.resourcemanager.hostname.rm1  
    hadoop102 
    
    
   
    yarn.resourcemanager.webapp.address.rm1  
    hadoop102:8088 
    
    
   
    yarn.resourcemanager.address.rm1  
    hadoop102:8032 
    
    
   
    yarn.resourcemanager.scheduler.address.rm1  
    hadoop102:8030 
    
    
   
    yarn.resourcemanager.resource-tracker.address.rm1  
    hadoop102:8031 
    
    
    
   
    yarn.resourcemanager.hostname.rm2  
    hadoop103 
    
   
    yarn.resourcemanager.webapp.address.rm2  
    hadoop103:8088 
    
   
    yarn.resourcemanager.address.rm2  
    hadoop103:8032 
    
   
    yarn.resourcemanager.scheduler.address.rm2  
    hadoop103:8030 
    
   
    yarn.resourcemanager.resource-tracker.address.rm2  
    hadoop103:8031 
    
    
    
   
    yarn.resourcemanager.hostname.rm3  
    hadoop104 
    
    
   
    yarn.resourcemanager.webapp.address.rm3  
    hadoop104:8088 
    
    
   
    yarn.resourcemanager.address.rm3  
    hadoop104:8032 
    
    
   
    yarn.resourcemanager.scheduler.address.rm3  
    hadoop104:8030 
    
    
   
    yarn.resourcemanager.resource-tracker.address.rm3  
    hadoop104:8031 
    
    
   
    yarn.resourcemanager.zk-address  
    hadoop102:2181,hadoop103:2181,hadoop104:2181 
    
    
   
    yarn.resourcemanager.recovery.enabled  
    true 
    
    
   
    yarn.resourcemanager.store.class  
    org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore 
    
    
   
    yarn.nodemanager.env-whitelist  
    JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLAS SPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME 
   

4）启动 YARN
（1）在 hadoop102 或者 hadoop103 中执行：

[root@hadoop102 ~]$ start-yarn.sh

（2）查看服务状态

[root@hadoop102 ~]$ yarn rmadmin -getServiceState rm1

（3）可以去 zkCli.sh 客户端查看 ResourceManager 选举锁节点内容：

[root@hadoop102 ~]$ zkCli.sh
[zk: localhost:2181(CONNECTED) 16] get -s 
/yarn-leader-election/cluster-yarn1/ActiveStandbyElectorLock
cluster-yarn1rm1
cZxid = 0x100000022
ctime = Tue Jul 14 17:06:44 CST 2020
mZxid = 0x100000022
mtime = Tue Jul 14 17:06:44 CST 2020
pZxid = 0x100000022
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x30000da33080005
dataLength = 20
numChildren = 0

（4）web 端查看 hadoop102:8088 和 hadoop103:8088 的 YARN 的状态

将整个 ha 搭建完成后,集群将形成以下模样

上图：