- 确认已经与java开发环境(java -version)下载hadoop.tar.zip安装包链接解压到指定位置配置环境变量(.bashrc)修改配置文件（直接搜hadoop安装）：core-site.xml,hdfs-site.xml,mapred-site.xml,hadoop-env.sh,yarn-site.xml相关命令
# 启动hadoop所有节点 start-all.sh # 关闭 stop-all.sh # 查看java进程，正常应该有Jps,Namenode,Datanode,ResourceManager,NodeManager jps # 关闭安全模式，不关闭 Hbase会出错 hdfs dfsadmin -safemode leave
遇到的问题参考
1
2
- 使用Hadoop命令时有警告
2022-03-18 13:42:49,610 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
解决方法：忽略警告，修改hadoop/etc/hadoop/log4j.properties文件,新增
log4j.logger.org.apache.hadoop.util.NativeCodeLoader=ERROR
- start-all.sh报错：只能被root执行
[hadoop@main ~]$ start-all.sh WARNING: Attempting to start all Apache Hadoop daemons as hadoop in 10 seconds. WARNING: This is not a recommended production deployment configuration. WARNING: Use CTRL-C to abort. Starting namenodes on [0.0.0.0] ERROR: namenode can only be executed by root. Starting datanodes ERROR: datanode can only be executed by root. Starting secondary namenodes [main] ERROR: secondarynamenode can only be executed by root. Starting resourcemanager ERROR: resourcemanager can only be executed by root. Starting nodemanagers ERROR: nodemanager can only be executed by root.
解决方法：修改hadoop/etc/hadoop/hadoop-env.sh，将最后的’root’替换为执行’命令的用户名(我的是hadoop)’
export HDFS_NAMENODE_USER=hadoop export HDFS_DATANODE_USER=hadoop export HDFS_SECONDARYNAMENODE_USER=hadoop export YARN_RESOURCEMANAGER_USER=hadoop export YARN_NODEMANAGER_USER=hadoop
- start-all.sh报错：无法写logs
解决办法：给权限，执行sudo chmod -R 777 logs
[hadoop@main ~]$ start-all.sh WARNING: Attempting to start all Apache Hadoop daemons as hadoop in 10 seconds. WARNING: This is not a recommended production deployment configuration. WARNING: Use CTRL-C to abort. Starting namenodes on [0.0.0.0] 0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts. 0.0.0.0: ERROR: Unable to write in /usr/local/hadoop/logs. Aborting. Starting datanodes localhost: ERROR: Unable to write in /usr/local/hadoop/logs. Aborting. Starting secondary namenodes [main] main: Warning: Permanently added 'main,172.17.43.2' (ECDSA) to the list of known hosts. main: ERROR: Unable to write in /usr/local/hadoop/logs. Aborting. Starting resourcemanager ERROR: Unable to write in /usr/local/hadoop/logs. Aborting. Starting nodemanagers localhost: ERROR: Unable to write in /usr/local/hadoop/logs. Aborting.
- start-all.sh执行后，datanode未被启动
原因：多次执行初始化命令hadoop namenode –format
解决方法：
不保留文件：删除hadoop/hdfs/data后，重新启动hdaoop保留文件：
参考
1
2
网页无法访问
解决方法：
- hadoop2.x端口为50070，hadoop3.x端口为9870对应端口没有放通：服务器的防火墙，云服务器的安全策略
网页内无法上传文件
原因：域名使用的是hadoop机器名
解决方法：
- 使用其他工具上传文件修改C:WindowsSystem32driversetchosts，域名映射
在win10的IDEA上使用Big Data Tools时，各种错误
- 端口不对：hadoop3.x使用8720端口没找到HADOOP_HOME环境变量：下载残血版hadoop，只保留bin目录，并修改环境变量HADOOP_HOME和path缺少文件(winutils.exe hadoop.dll)无法运行：下载并添加进hadoop/bin
2&3 参考
2&3 一次解决，链接参考里的
使用IDEA开发HDFS程序上传文件，文件名有，内容为空
原因：请求的namenode，需要把数据传给datanode；(同6，但6的解决办法不适用)但datanode使用的是hadoop机器名，不是正式的域名
解决方法：
// Configuration conf = new Configuration();
// 加上下面这一句，直接使用真实域名
conf.set("dfs.client.use.datanode.hostname", "true");
