栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

Hadoop或Spark运行缺少winutils.exe文件

Hadoop或Spark运行缺少winutils.exe文件

问题现象:

21/11/28 20:29:55 WARN Shell: Did not find winutils.exe: {}
java.io.FileNotFoundException: Could not locate Hadoop executable: D:Programhadoop-3.3.1binwinutils.exe -see https://wiki.apache.org/hadoop/WindowsProblems
        at org.apache.hadoop.util.Shell.getQualifiedBinInner(Shell.java:619)
        at org.apache.hadoop.util.Shell.getQualifiedBin(Shell.java:592)
        at org.apache.hadoop.util.Shell.(Shell.java:689)
        at org.apache.hadoop.util.StringUtils.(StringUtils.java:79)
        at org.apache.hadoop.conf.Configuration.getTimeDurationHelper(Configuration.java:1886)
        at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1846)
        at org.apache.hadoop.conf.Configuration.getTimeDuration(Configuration.java:1819)
        at org.apache.hadoop.util.ShutdownHookManager.getShutdownTimeout(ShutdownHookManager.java:183)
        at org.apache.hadoop.util.ShutdownHookManager$HookEntry.(ShutdownHookManager.java:207)
        at org.apache.hadoop.util.ShutdownHookManager.addShutdownHook(ShutdownHookManager.java:304)
        at org.apache.spark.util.SparkShutdownHookManager.install(ShutdownHookManager.scala:181)
        at org.apache.spark.util.ShutdownHookManager$.shutdownHooks$lzycompute(ShutdownHookManager.scala:50)

 

 

解决:

这个问题是大部分初学者都会犯的错误,解决办法就是将官网下的源码在Windows上编译一下,但是这个相当麻烦。

简单方法就是直接将编译好的winutils.exe文件放在Hadoop安装文件夹得bin包下,重启IDE

https://url68.ctfile.com/f/1445568-522738759-a23515
(访问密码:5719)

 

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/618663.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号