栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

Hadoop filesystem连接失败问题解决

Hadoop filesystem连接失败问题解决

每次重启wsl的时候,hadoop的fs都连不上了!!!

$ hadoop fs -ls /
-ls: java.net.UnknownHostException: master
Usage: hadoop fs [generic options]
        [-appendToFile  ... ]
        [-cat [-ignoreCrc]  ...]
        [-checksum  ...]
        [-chgrp [-R] GROUP PATH...]
        [-chmod [-R]  PATH...]
        [-chown [-R] [OWNER][:[GROUP]] PATH...]
        [-copyFromLocal [-f] [-p] [-l] [-d] [-t ]  ... ]
        [-copyToLocal [-f] [-p] [-ignoreCrc] [-crc]  ... ]
        [-count [-q] [-h] [-v] [-t []] [-u] [-x] [-e]  ...]
        [-cp [-f] [-p | -p[topax]] [-d]  ... ]
        [-createSnapshot  []]
        [-deleteSnapshot  ]
        [-df [-h] [ ...]]
        [-du [-s] [-h] [-v] [-x]  ...]
        [-expunge]
        [-find  ...  ...]
        [-get [-f] [-p] [-ignoreCrc] [-crc]  ... ]
        [-getfacl [-R] ]
        [-getfattr [-R] {-n name | -d} [-e en] ]
        [-getmerge [-nl] [-skip-empty-file]  ]
        [-help [cmd ...]]
        [-ls [-C] [-d] [-h] [-q] [-R] [-t] [-S] [-r] [-u] [-e] [ ...]]
        [-mkdir [-p]  ...]
        [-moveFromLocal  ... ]
        [-moveToLocal  ]
        [-mv  ... ]
        [-put [-f] [-p] [-l] [-d]  ... ]
        [-renameSnapshot   ]
        [-rm [-f] [-r|-R] [-skipTrash] [-safely]  ...]
        [-rmdir [--ignore-fail-on-non-empty]  ...]
        [-setfacl [-R] [{-b|-k} {-m|-x } ]|[--set  ]]
        [-setfattr {-n name [-v value] | -x name} ]
        [-setrep [-R] [-w]   ...]
        [-stat [format]  ...]
        [-tail [-f] ]
        [-test -[defsz] ]
        [-text [-ignoreCrc]  ...]
        [-touchz  ...]
        [-truncate [-w]   ...]
        [-usage [cmd ...]]

Generic options supported are:
-conf         specify an application configuration file
-D                define a value for a given property
-fs  specify default filesystem URL to use, overrides 'fs.defaultFS' property from configurations.
-jt   specify a ResourceManager
-files                 specify a comma-separated list of files to be copied to the map reduce cluster
-libjars                specify a comma-separated list of jar files to be included in the classpath
-archives ,...>          specify a comma-separated list of archives to be unarchived on the compute machines

The general command line syntax is:
command [genericOptions] [commandOptions]

暂时的解决方式如下:
看看ssh启动了没:

ps -e | grep ssh

没有的话重启一下

sudo service ssh restart

如果是这样就启动成功了:

$ ps -e | grep ssh
 7999 ?        00:00:00 sshd

尝试重启一些filesystem

$ start-dfs.sh
Starting namenodes on [master]
master: ssh: Could not resolve hostname master: Name or service not known
Starting datanodes
Starting secondary namenodes [master]
master: ssh: Could not resolve hostname master: Name or service not known

报这个错是因为core-site.xml写的和/etc/hosts中的不一致,改一下core-site.xml,路径是${HADOOP_HOME}/etc/hadoop/:

我把master改成localhost了。
再次重启:

$ start-dfs.sh
Starting namenodes on [localhost]
Starting datanodes
localhost: datanode is running as process 8278.  Stop it first.
Starting secondary namenodes [master]

如果报这个错就先stop一下:

$ stop-all.sh
WARNING: Stopping all Apache Hadoop daemons as yc in 10 seconds.
WARNING: Use CTRL-C to abort.
Stopping namenodes on [localhost]
Stopping datanodes
Stopping secondary namenodes [master]
master: ssh: Could not resolve hostname master: Name or service not known
Stopping nodemanagers
Stopping resourcemanager

然后再次重启:

$ start-dfs.sh
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [master]

这样就可以了,最后看看hadoop fs可以启动没有:

hadoop fs -ls /
Found 2 items
drwxr-xr-x   - yc supergroup          0 2021-12-09 09:39 /index
drwxr-xr-x   - yc supergroup          0 2021-12-08 20:24 /wordcount

ok!

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/650972.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号