Hadoop全分布

一：Hadoop前期准备工作 1.进入网卡位置更改静态ip地址使ip地址生效

vim /etc/sysconfig/network-scripts/ifcfg-eth0

systemctl restart network

2.配置映射

ip   主机名1
ip   主机名2
ip   主机名3

3.关闭防火墙

systemctl status firewalld.service  //查询防火墙状态
systemctl stop firewalld.service    //关闭防火墙
systemctl status firewalld.service    
systemctl disable firewalld.service //永久防火墙

4.关闭liunx 服务

setenforce 0
gentenforce  // 查询状态

5.生成密钥，三台机器互通

ssh-keygen -t rsa  // 生成公钥和私钥
ssh-copy-id master   // hadoop102
ssh-copy-id slave2    // hadoop103
ssh-copy-id slave3   //  hadoop104

6.导入Hadoop 和 jdk 安装包

hadoop-2.7.7 
jdk.....

7.解压安装包

tar-zxvf hadoop-2.7.7
tar-zxvf jdk...

8.配置环境变量

vim /etc/profile
#HADOOP_HOME
export HADOOP_HOME=/opt/module/hadoop-2.7.7
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin

#JAVA_HOME
export JAVA_HOME=/opt/module/jdk1.8.0_162
export PATH=$PATH:$JAVA_HOME/bin
//生效环境变量
source /etc/profile

二:配置HDFS

vim core-site.xml  //配置文件名
        
                fs.defaultFS
                hdfs://hadoop102:9000
        
        
                hadoop.tmp.dir
                /opt/module/hadoop-2.7.7/data/tmp
        

vim hdfs-site.xml //配置文件名

         
	dfs.replication
	3
        
        
                dfs.namenode.secondary.http-address
                hadoop104:50090
        
        
                dfs.namenode.name.dir
                /opt/module/hadoop-2.7.7/name
        
        
                dfs.datanode.data.dir
                /opt/module/hadoop-2.7.7/data

三：配置YARN

vim yarn-site.xml 


        
                yarn.nodemanager.aux-services
                mapreduce_shuffle
        
        
                yarn.resourcemanager.hostname
                hadoop103
        
        
                yarn.log-aggregation-enable
                true
        
        
                yarn.log-aggregation.retain-seconds
                604800

// 改配置名
mv mapred-site.xml.template mapred-site.xml

//
vim mapred-site.xml
        
                mapreduce.framework.name
                yarn
        
        ec
                mapreduce.jobhistory.address
                hadoop102:10020
        
        
                mapreduce.jobhistory.webapp.address
                hadoop102:19888
        

// 配置jdk路径
vim hadoop-env.sh 
vim yarn-env.sh 
vim mapred-env.sh

四：群起配置

vim   /opt/module/hadoop-2.7.7/etc/hadoop/slaves

hadoop102
hadoop103
hadoop104

五：分发文件和环境变量

{	scp -r /opt/module root@hadoop103:/opt       
	scp -r /opt/module root@hadoop104:/opt       }

scp /etc/profile root@hadoop103:/etc/profile
scp /etc/profile root@hadoop104:/etc/profile

source /etc/profile

六：格式化 namenode

bin/hdfs namenode -format

七：群起配置

hadoop102  hdfs   // start-dfs.sh   // stop-dfs.sh
hadoop103  yarn  //start-yarn.sh   // stop-yarn.sh

Hadoop全分布

Java相关栏目本月热门文章