Hive搭建 ha高可用集群
- Hive搭建 ha高可用集群
- 第一步 修改hdfs-site.xml
- 第二步 修改core-site.xml
- 第三步 修改mapred-site.xml
- 第四步 修改yarn-site.xml
Hive搭建 ha高可用集群
第一步 修改hdfs-site.xml
dfs.nameservices
rootcloud
dfs.ha.namenodes .rootcloud
nn1,nn2
dfs.namenode.rpc-address.cluster1.nn1
hadoop01:8020
dfs.namenode.servicerpc-address.rootcloud.nn1
hadoop01:8021
dfs.namenode.http-address.cluster1.nn1
hadoop01:50070
dfs.namenode.rpc-address.cluster1.nn2
hadoop02:8020
dfs.namenode.servicerpc-address.rootcloud.nn1
hadoop02:8021
dfs.namenode.http-address.cluster1.nn2
hadoop02:50070
dfs.namenode.shared.edits.dir
qjournal:// hadoop01:8485; hadoop02:8485; hadoop03:8485 /rootcloud-journal-new
dfs.journalnode.edits.dir
/data/rootcloud/hadoop-3.2.2/journal
dfs.ha.automatic-failover.enabled
shell(/bin/true)
dfs.client.failover.proxy.provider.rootcloud
org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider
dfs.ha.fencing.methods
shell(/bin/true)
dfs.ha.fencing.ssh.private-key-files
/root/.ssh/id_rsa
dfs.ha.fencing.ssh.connect-timeout
50000
第二步 修改core-site.xml
fs.defaultFS
hdfs:// rootcloud
hadoop.tmp.dir
/data/app/hadoop-3.2.2/etc/hadoop/tmp
ha.zookeeper.quorum
hadoop01:2181,hadoop02:2181,hadoop03:2181
第三步 修改mapred-site.xml
mapreduce.framework.name
yarn
第四步 修改yarn-site.xml
yarn.resourcemanager.ha.enabled
true
yarn.resourcemanager.cluster-id
yarn-ha
yarn.resourcemanager.ha.rm-ids
rm1,rm2
yarn.resourcemanager.hostname.rm1
hadoop01
yarn.resourcemanager.hostname.rm2
hadoop02
yarn.resourcemanager.zk-address
hadoop01:2181,hadoop02:2181,hadoop03:2181
yarn.nodemanager.aux-services
mapreduce_shuffle