Docker部署hadoop

一、环境准备二、安装并启动Docker三、 配置hadoop四、启动hadoop集群五、关闭hadoop集群六、卸载Docker

操作系统：Centos7.9 二、安装并启动Docker

安装

方法一：从阿里云镜像获取安装包

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

方法二：从docker官方获取安装包

 sudo yum install -y yum-utils

 sudo yum-config-manager 
    --add-repo 
    https://download.docker.com/linux/centos/docker-ce.repo

建议使用方法一安装过程较快

获取到安装包以后开始安装

yum install -y docker-ce docker-ce-cli containerd.io

安装完成以后可输入docker version验证docker是否安装成功
如果出现
Client: Docker Engine - Community
Version: 20.10.12
API version: 1.41
Go version: go1.16.12
Git commit: e91ed57
Built: Mon Dec 13 11:45:41 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.12
API version: 1.41 (minimum version 1.12)
Go version: go1.16.12
Git commit: 459d0df
Built: Mon Dec 13 11:44:05 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.12
GitCommit: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
runc:
Version: 1.0.2
GitCommit: v1.0.2-0-g52b36a2
docker-init:
Version: 0.19.0
GitCommit: de40ad0
表示安装成功

启动

sudo systemctl start docker

下载doker下的hadoop

docker pull kiwenlau/hadoop1.0

在docker中搭建网络

docker network create –d=bridge hadoop

创建hadoop-master容器

docker run -itd 
--net=hadoop 
--name hadoop-cluster1 
--hostname haddoop-master 
-p 50070:50070 
-p 9000:9000 
-p 8088:8088 
kiwenlau/hadoop1.0

创建hadoop-slave1容器

docker run -itd 
--net=hadoop 
--name hadoop-cluster2 
--hostname haddoop-slave1 
kiwenlau/hadoop1.0

创建hadoop-slave2容器

docker run -itd 
--net=hadoop 
--name hadoop-cluster3 
--hostname haddoop-slave2 
kiwenlau/hadoop1.0

进入hadoop-master容器

docker exec -it hadoop-master bash

启动hadoop集群

./start-hadoop.sh

使用jps命令查看启动的服务

root@hadoop-master:~# jps
1549 SecondaryNameNode
1356 NameNode
8098 Jps
1705 ResourceManager

使用exit可以退出当前容器的命令窗口

 exit

此时hadoop容器仍会在后台继续运行
root@hadoop-master:~# exit
exit
[root@qh ~]#

退出容器命令窗口后可使用docker stop hadoop-master命令停止容器，停止后容器不再运行

[root@qh ~]# docker stop hadoop-master

可使用docker ps命令查看所有容器的运行情况

[root@qh ~]# docker ps
ConTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5927d483b848 a59a34125272 “sh -c 'service ssh …” 2 days ago Up 2 days hadoop-slave1
290b68fef58d a59a34125272 “sh -c 'service ssh …” 2 days ago Up 2 days hadoop-slave2
785a90585d6b a59a34125272 “sh -c 'service ssh …” 2 days ago Stop 5 seconds ago 0.0.0.0:8088->8088/tcp, :::8088->8088/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 0.0.0.0:50070->50070/tcp, :::50070->50070/tcp hadoop-master
可见hadoop-master已经停止

卸载 Docker软件包

 sudo yum remove docker-ce docker-ce-cli containerd.io

卸载docker后，主机上的容器等一系列文件不会自动删除，删除所有docker有关文件可用：

 sudo rm -rf /var/lib/docker
 sudo rm -rf /var/lib/containerd