Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.classForName(Ljava/lang/String;)Ljava/lang/Class;
at org.apache.spark.streaming.scheduler.JobGenerator.liftedTree1$1(JobGenerator.scala:52)
at org.apache.spark.streaming.scheduler.JobGenerator.
at org.apache.spark.streaming.scheduler.JobScheduler.
at org.apache.spark.streaming.StreamingContext.
at org.apache.spark.streaming.StreamingContext.
at streaming.SparkStream01_WordCount$.main(SparkStream01_WordCount.scala:20)
at streaming.SparkStream01_WordCount.main(SparkStream01_WordCount.scala)
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.classForName(Ljava/lang/String;)Ljava/lang/Class; at org.apache.spark.streaming.scheduler.JobGenerator.liftedTree1$1(JobGenerator.scala:52) at org.apache.spark.streaming.scheduler.JobGenerator.(JobGenerator.scala:51) at org.apache.spark.streaming.scheduler.JobScheduler. (JobScheduler.scala:55) at org.apache.spark.streaming.StreamingContext. (StreamingContext.scala:184) at org.apache.spark.streaming.StreamingContext. (StreamingContext.scala:85) at streaming.SparkStream01_WordCount$.main(SparkStream01_WordCount.scala:20) at streaming.SparkStream01_WordCount.main(SparkStream01_WordCount.scala)
原因 maven 导入的依赖版本问题
1.因为IDEA-maven的依赖包是直接在pom文件中添加代码,在初次使用时,很容易出现版本不一致的情况;
2.我的程序是spark-scala程序,还需要spark依赖的版本与scala匹配,比如scala2.12只适用于spark2.4以上;
我的scala SDK 是2.12
spark-streaming与scala版本对应
org.apache.spark
spark-streaming_2.12
3.0.0



