https://blog.csdn.net/m0_59186440/article/details/122170209https://blog.csdn.net/m0_59186440/article/details/122170209https://blog.csdn.net/m0_59186440/article/details/122170209前文链接

前面一节我们完成了虚拟机的安装，下面我们进行配置

vi /etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0
HWADDR=00:0C:29:16:AB:34
TYPE=Ethernet
#UUID=07c5088c-f83e-4e84-b2cb-32ae75117085
onBOOT=yes
NM_ConTROLLED=yes
BOOTPROTO=static

IPADDR=192.168.106.131
NETMASK=255.255.255.0
GATEWAY=192.168.106.2
DNS1=192.168.106.2

 重启网络

service network restart

连接xshell

 

 二、安装yum软件

1. 进入yum配置文件夹

cd /etc/yum.repos.d/

修改之前的文件名

mv CentOS-base.repo CentOS-base.repo.bak

2．下载最新网络源

（注：如果命令运行失败，请自行搜索另外找一个运行）

curl -o /etc/yum.repos.d/CentOS-base.repo  http://file.kangle.odata.cc/repo/Centos-6.repo

3．挂载镜像

mount /dev/dvd /media/

4．安装上传软件

yum install lrzsz –y

5．安装vim软件

yum install vim -y

 

 三、安装JDK  1.上传jdk-8u121-linux-x64.tar.gz文件到/opt目录 jdk下载地址

https://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.htmlhttps://www.oracle.com/technetwork/java/javase/downloads/java-archive-javase8-2177648.html

 

 2.解压jdk文件

tar -zxvf jdk-8u121-linux-x64.tar.gz -C /usr/local

 

3.配置环境变量

vim /etc/profile

export JAVA_HOME=/usr/local/jdk1.8.0_121

export PATH=$JAVA_HOME/bin:$PATH

export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

 

 4.环境变量生效

source /etc/profile

 5.检查jdk是否安装成功

java -version

四、搭建hadoop

1. 上传hadoop-2.7.4.tar.gz文件到/opt目录 下载地址版本2.7.4

https://archive.apache.org/dist/hadoop/common/https://archive.apache.org/dist/hadoop/common/https://archive.apache.org/dist/hadoop/common/

2. 解压缩hadoop-2.7.4.tar.gz 文件

tar -zxvf hadoop-2.7.4.tar.gz -C /usr/local

解压后即可，看到/usr/local/hadoop-2.7.4文件夹

3.配置环境变量

vim /etc/profile

export HADOOP_HOME=/export/servers/hadoop-2.7.4
export PATH=$PATH:$HADOOP_HOME/bin:
$HADOOP_HOME/sbin

 

初始化环境变量

source /etc/profile

验证Hadoop是否成功安装配置，命令如下

hadoop version

 

hadoop-env.sh	配置Hadoop运行所需的环境变量
yarn-env.sh	配置Yarn运行所需的环境变量
core-site.xml	集群全局参数，用于定义系统级别的参数，如HDFS URL、Hadoop的临时目录等
hdfs-site.xml	HDFS参数，如名称节点和数据节点的存放位置、文件副本的个数、文件读取的权限等
mapred-site.xml	MapReduce参数，包括JobHistory Server和应用程序参数两部分，如reduce任务的默认个数、任务所能够使用内存的默认上下限等
yarn-site.xml	集群资源管理系统参数，配置ResourceManager，NodeManager的通信端口，web监控端口等

 

 

  （在底部Configuration标签内添加以下Configuration标签内的内容，其他文件也是） 1.core-site.xml配置·

    
    	fs.defaultFS  
        hdfs://master:8020  
        
    
      hadoop.tmp.dir
      /var/log/hadoop/tmp
    

 

2.hadoop-env.sh配置

export JAVA_HOME=/usr/local/jdk1.8.0_121

 

 

3.yarn-env.sh配置

export JAVA_HOME=/usr/local/jdk1.8.0_121

 

 

4.mapred-site.xml配置 复制cp mapred-site.xml.template mapred-site.xml

    mapreduce.framework.name
    yarn



    mapreduce.jobhistory.address
    master:10020


     mapreduce.jobhistory.webapp.address
     master:19888

5.yarn-site.xml配置

  
    yarn.resourcemanager.hostname
    master
      
  
    yarn.resourcemanager.address
    ${yarn.resourcemanager.hostname}:8032
  
  
    yarn.resourcemanager.scheduler.address
    ${yarn.resourcemanager.hostname}:8030
  
  
    yarn.resourcemanager.webapp.address
    ${yarn.resourcemanager.hostname}:8088
  
  
    yarn.resourcemanager.webapp.https.address
    ${yarn.resourcemanager.hostname}:8090
  
  
    yarn.resourcemanager.resource-tracker.address
    ${yarn.resourcemanager.hostname}:8031
  
  
    yarn.resourcemanager.admin.address
    ${yarn.resourcemanager.hostname}:8033
  
  
    yarn.nodemanager.local-dirs
    /data/hadoop/yarn/local
  
  
    yarn.log-aggregation-enable
    true
  
  
    yarn.nodemanager.remote-app-log-dir
    /data/tmp/logs
  
 
 yarn.log.server.url 
 http://master:19888/jobhistory/logs/
 URL for job history server


   yarn.nodemanager.vmem-check-enabled
    false
  
 
    yarn.nodemanager.aux-services
    mapreduce_shuffle
  
  
    yarn.nodemanager.aux-services.mapreduce.shuffle.class
      org.apache.hadoop.mapred.ShuffleHandler
      
  
        yarn.nodemanager.resource.memory-mb  
        2048  
   
   
        yarn.scheduler.minimum-allocation-mb  
        512  
    
   
        yarn.scheduler.maximum-allocation-mb  
        4096  
  
  
    mapreduce.map.memory.mb 
    2048 
  
  
    mapreduce.reduce.memory.mb 
    2048 
  
  
    yarn.nodemanager.resource.cpu-vcores 
    1 
 

 

6.hdfs-site.xml配置

    dfs.namenode.name.dir
    file:///data/hadoop/hdfs/name


    dfs.datanode.data.dir
    file:///data/hadoop/hdfs/data


     dfs.namenode.secondary.http-address
     master:50090


     dfs.replication
     3

 

---------------- 以上master节点配置完成  -------------