栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

Hadoop部署

Hadoop部署

官网:hadoop.apache.org

下载地址:

Apache Hadoophttps://hadoop.apache.org/releases.html

1.下载安装包

 2.上传到Linux中/tmp目录中

[root@localhost ~]# cd software
[root@localhost software]# rz

 3.添加一个维护的用户和文件夹

[root@localhost software]# useradd ssn
[root@localhost software]# id ssn
uid=1001(ssn) gid=1001(ssn) groups=1001(ssn)
[root@localhost software]# pwd
/root/software
[root@localhost software]# su - ssn
[ssn@localhost ~]$ mkdir source software app log data lib tmp
[ssn@localhost ~]$ ls
app  data  lib  log  software  source  tmp

创建tmp目录的原因:系统自带的tmp目录,定期不访问,根据当前版本特性,自动清除,默认是30天

4.移动安装包,并赋予权限

[ssn@localhost ~]$ exit
[root@localhost software]# mv ~/software/hadoop-3.2.2.tar.gz  /home/ssn/software/
[root@localhost software]# chown ssn:ssn /home/ssn/software
[ssn@localhost hadoop]$ 
 

e.NameNode daemon and DataNode daemon启动:(名称节点和数据节点启动)

[ssn@localhost hadoop]$ sbin/start-dfs.sh

[ssn@localhost hadoop]$ sbin/start-dfs.sh
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [localhost.localdomain]
localhost.localdomain: Warning: Permanently added 'localhost.localdomain' (ECDSA) to the list of known hosts.

[ssn@localhost hadoop]$

jps查看服务有没有成功启动,但是不保险,建议还是使用ps -ef | grep hadoop   

    

 f.访问http://localhost:9870/

如果使用的云主机,需要配置安全组才能访问,id地址:9870,访问成功如图所示

 g.

注意:

先用root用户执行

rm -rf /tmp/hadoop

然后使用ssn用户执行

[ssn@localhost hadoop]$bin/hdfs dfs -mkdir /user
[ssn@localhost hadoop]$bin/hdfs dfs -mkdir /user/ssn
[ssn@localhost hadoop]$bin/hdfs dfs -mkdir input
[ssn@localhost hadoop]$bin/hdfs dfs -put etc/hadoop/*.xml input
[ssn@localhost hadoop]$bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.2.2.jar grep input output 'dfs[a-z.]+'
[ssn@localhost hadoop]$bin/hdfs dfs -get output output
[ssn@localhost hadoop]$bin/hdfs dfs -cat output/*
[ssn@localhost hadoop]$$ sbin/stop-dfs.sh

 

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/600676.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号