栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 软件开发 > 后端开发 > Java

Spark 启动报错

Java 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

Spark 启动报错

Spark 启动报错
    • 报错
    • 报错原因
    • 修改方法

报错
starting org.apache.spark.deploy.master.Master, logging to /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-centos01.out
failed to launch: nice -n 0 /opt/modules/spark-2.4.5/bin/spark-class org.apache.spark.deploy.master.Master --host centos01 --port 7077 --webui-port 8080
  	at java.lang.Class.getMethod0(Class.java:3018)
  	at java.lang.Class.getMethod(Class.java:1784)
  	at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
  	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
  Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger
  	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
  	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
  	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
  	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
  	... 7 more
full log in /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-centos01.out
centos02: starting org.apache.spark.deploy.worker.Worker, logging to /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-centos02.out
centos03: starting org.apache.spark.deploy.worker.Worker, logging to /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-centos03.out
centos02: failed to launch: nice -n 0 /opt/modules/spark-2.4.5/bin/spark-class org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://centos01:7077
centos03: failed to launch: nice -n 0 /opt/modules/spark-2.4.5/bin/spark-class org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://centos01:7077
centos02:   	at java.lang.Class.getMethod0(Class.java:3018)
centos02:   	at java.lang.Class.getMethod(Class.java:1784)
centos02:   	at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
centos02:   	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
centos02:   Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger
centos02:   	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
centos02:   	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
centos02:   	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
centos02:   	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
centos02:   	... 7 more
centos02: full log in /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-centos02.out
centos03:   	at java.lang.Class.getMethod0(Class.java:3018)
centos03:   	at java.lang.Class.getMethod(Class.java:1784)
centos03:   	at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
centos03:   	at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
centos03:   Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger
centos03:   	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
centos03:   	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
centos03:   	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
centos03:   	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
centos03:   	... 7 more
centos03: full log in /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-centos03.out
报错原因

缺少Hadoop依赖包

修改方法

在spark-env.sh加入以下代码

export SPARK_DIST_CLASSPATH=$(hadoop classpath)
export SPARK_DIST_CLASSPATH=$(/opt/modules/hadoop-2.10.0/bin/hadoop classpath)
转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/969836.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号