软件版本:CentOS Linux release 7.5.1804 (Core)
安装准备:
1、安装jdk
2、主机名不能包含下划线_
3、配置hosts
4、配置master主机免密登录各节点
5、关闭防火墙和selinux
安装过程摘要:
1、解压安装包
2、配置环境变量
3、修改配置文件
4、目录复制到所有节点
5、namenode格式化
6、启动hadoop
7、验证
1、安装jdk用yum安装,先查看有哪些可用的版本:
yum list java*
安装
sudo yum install java-1.8.0-openjdk.x86_64
验证安装
java -version
输出:
openjdk version "1.8.0_312"
OpenJDK Runtime Environment (build 1.8.0_312-b07)
OpenJDK 64-Bit Server VM (build 25.312-b07, mixed mode)
安装完成
虚拟机安装hadoop,需要所有主机都先配置成静态地址。
查看有哪些网络接口:
ip addr
输出:
1: lo:
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3:
link/ether 08:00:27:12:57:fe brd ff:ff:ff:ff:ff:ff
inet 10.15.53.138/24 brd 10.15.53.255 scope global noprefixroute dynamic enp0s3
valid_lft 86377sec preferred_lft 86377sec
inet6 fe80::7a51:3d6f:39e8:6a7d/64 scope link noprefixroute
valid_lft forever preferred_lft forever
查到主机实际的网络接口名称是enp0s3,到网卡配置目录:/etc/sysconfig/network-scripts/下面找到主机的网卡文件,编辑:
sudo vi /etc/sysconfig/network-scripts/ifcfg-enp0s3
修改或添加以下内容:
onBOOT=yes BOOTPROTO=static IPADDR= NETMASK= GATEWAY= DNS1=
配置后重启网络服务
service network restart
配置完成。
3、配置hostssudo vi /etc/hosts
添加以下内容:
10.15.53.240 app 10.15.53.241 hadoop1 10.15.53.242 hadoop2 10.15.53.243 hadoop3
测试,使用配置好的名称访问主机,能访问即配置成功
ping hadoop3 PING hadoop3 (127.0.0.1) 56(84) bytes of data. 64 bytes from hadoop3 (127.0.0.1): icmp_seq=1 ttl=64 time=0.034 ms 64 bytes from hadoop3 (127.0.0.1): icmp_seq=2 ttl=64 time=0.043 ms4 、配置ssh免密登录
基本思路如下:
A免密访问B
1、A生成秘钥
2、B创建.ssh/authorized_keys文件
3、将A的id_rsa.pub文件内容复制给B的authorized_keys文件
4、设置B的权限,.ssh为700,authorized_keys的权限为600
具体步骤
进入hadoop1,生成id_rsa.pub
ssh-keygen
进入hadoop2,创建.ssh/authorized_keys
mkdir ~/.ssh vi ~/.ssh/authorized_keys
编辑authorized_keys文件,将hadoop1的id_rsa.pub文件内容追加到hadoop2的authorized_keys文件中
设置一个文件和一个目录的权限:
chmod 600 authorized_keys cd .. chmod 700 .ssh
测试,在hadoop1上登录hadoop2:
ssh hadoop2
输出:
The authenticity of host 'hadoop2 (10.15.53.242)' can't be established.
ECDSA key fingerprint is SHA256:gQherZTQAY5P7B8wP9Y/R5ZIn33r84/M84R87G9HXlo.
ECDSA key fingerprint is MD5:5e:b5:d4:f7:60:07:34:48:d8:6a:c9:de:0d:c5:db:0c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hadoop2,10.15.53.242' (ECDSA) to the list of known hosts.
Last login: Fri Nov 19 14:55:50 2021 from 10.15.53.64
[lht@hadoop2 ~]$
使用主机名可以免密登录,则设置成功。
5、关闭防火墙和selinux查看防火墙状态
systemctl status firewalld
输出:
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
Active: active (running) since 五 2021-11-19 14:28:57 CST; 1h 23min ago
Docs: man:firewalld(1)
Main PID: 654 (firewalld)
CGroup: /system.slice/firewalld.service
└─654 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid
11月 19 14:28:56 hadoop1 systemd[1]: Starting firewalld - dynamic firewall daemon...
11月 19 14:28:57 hadoop1 systemd[1]: Started firewalld - dynamic firewall daemon.
关闭防火墙
systemctl stop firewalld #关闭服务 systemctl disable firewalld #关闭自启动
查看selinux状态
getenforce
Enforcing 开启
Permissive 关闭
关闭selinux
setenforce 0
停止自启动
vi /etc/selinux/config
将配置文件中的 SELINUX=disabled 改为 SELINUX=enforcing 即可
6、安装hadoop 6.1、下载下载地址:
https://downloads.apache.org/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gzhttps://downloads.apache.org/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz
curl -o hadoop3.tar.gz https://downloads.apache.org/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz
tar -xvf hadoop3.tar.gz mv hadoop-3.3.1 ../app/hadoop36.2、配置环境变量
安装hadoop主目录为:/home/lht/app/hadoop3
创建软连接
ln -s hadoop3 hadoop
配置环境变量
sudo /etc/profile
将下面两行加入环境变量
export HADOOP_HOME=/home/lht/app/hadoop export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$PATH
环境变量生效
source /etc/profile
验证配置
hadoop version
报错了:
[lht@hadoop1 app]$ hadoop version
ERROR: JAVA_HOME is not set and could not be found.
这是没有设置java的环境变量
配置java环境变量:
未完成,持续更新中…………
3、修改配置文件
4、目录复制到所有节点
5、namenode格式化
6、启动hadoop
7、验证



