- 报错
- 报错原因
- 修改方法
starting org.apache.spark.deploy.master.Master, logging to /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-centos01.out failed to launch: nice -n 0 /opt/modules/spark-2.4.5/bin/spark-class org.apache.spark.deploy.master.Master --host centos01 --port 7077 --webui-port 8080 at java.lang.Class.getMethod0(Class.java:3018) at java.lang.Class.getMethod(Class.java:1784) at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger at java.net.URLClassLoader.findClass(URLClassLoader.java:382) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ... 7 more full log in /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.master.Master-1-centos01.out centos02: starting org.apache.spark.deploy.worker.Worker, logging to /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-centos02.out centos03: starting org.apache.spark.deploy.worker.Worker, logging to /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-centos03.out centos02: failed to launch: nice -n 0 /opt/modules/spark-2.4.5/bin/spark-class org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://centos01:7077 centos03: failed to launch: nice -n 0 /opt/modules/spark-2.4.5/bin/spark-class org.apache.spark.deploy.worker.Worker --webui-port 8081 spark://centos01:7077 centos02: at java.lang.Class.getMethod0(Class.java:3018) centos02: at java.lang.Class.getMethod(Class.java:1784) centos02: at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) centos02: at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) centos02: Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger centos02: at java.net.URLClassLoader.findClass(URLClassLoader.java:382) centos02: at java.lang.ClassLoader.loadClass(ClassLoader.java:418) centos02: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) centos02: at java.lang.ClassLoader.loadClass(ClassLoader.java:351) centos02: ... 7 more centos02: full log in /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-centos02.out centos03: at java.lang.Class.getMethod0(Class.java:3018) centos03: at java.lang.Class.getMethod(Class.java:1784) centos03: at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544) centos03: at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526) centos03: Caused by: java.lang.ClassNotFoundException: org.slf4j.Logger centos03: at java.net.URLClassLoader.findClass(URLClassLoader.java:382) centos03: at java.lang.ClassLoader.loadClass(ClassLoader.java:418) centos03: at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355) centos03: at java.lang.ClassLoader.loadClass(ClassLoader.java:351) centos03: ... 7 more centos03: full log in /opt/modules/spark-2.4.5/logs/spark-hadoop-org.apache.spark.deploy.worker.Worker-1-centos03.out报错原因
缺少Hadoop依赖包
修改方法在spark-env.sh加入以下代码
export SPARK_DIST_CLASSPATH=$(hadoop classpath) export SPARK_DIST_CLASSPATH=$(/opt/modules/hadoop-2.10.0/bin/hadoop classpath)



