栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

Exception in thread “main“ java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.classForName(Lj

Exception in thread “main“ java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.classForName(Lj

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.classForName(Ljava/lang/String;)Ljava/lang/Class;
    at org.apache.spark.streaming.scheduler.JobGenerator.liftedTree1$1(JobGenerator.scala:52)
    at org.apache.spark.streaming.scheduler.JobGenerator.(JobGenerator.scala:51)
    at org.apache.spark.streaming.scheduler.JobScheduler.(JobScheduler.scala:55)
    at org.apache.spark.streaming.StreamingContext.(StreamingContext.scala:184)
    at org.apache.spark.streaming.StreamingContext.(StreamingContext.scala:85)
    at streaming.SparkStream01_WordCount$.main(SparkStream01_WordCount.scala:20)
    at streaming.SparkStream01_WordCount.main(SparkStream01_WordCount.scala)
 

Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.util.Utils$.classForName(Ljava/lang/String;)Ljava/lang/Class;
	at org.apache.spark.streaming.scheduler.JobGenerator.liftedTree1$1(JobGenerator.scala:52)
	at org.apache.spark.streaming.scheduler.JobGenerator.(JobGenerator.scala:51)
	at org.apache.spark.streaming.scheduler.JobScheduler.(JobScheduler.scala:55)
	at org.apache.spark.streaming.StreamingContext.(StreamingContext.scala:184)
	at org.apache.spark.streaming.StreamingContext.(StreamingContext.scala:85)
	at streaming.SparkStream01_WordCount$.main(SparkStream01_WordCount.scala:20)
	at streaming.SparkStream01_WordCount.main(SparkStream01_WordCount.scala)

原因  maven 导入的依赖版本问题

1.因为IDEA-maven的依赖包是直接在pom文件中添加代码,在初次使用时,很容易出现版本不一致的情况;
2.我的程序是spark-scala程序,还需要spark依赖的版本与scala匹配,比如scala2.12只适用于spark2.4以上;

我的scala  SDK 是2.12

spark-streaming与scala版本对应

 

















        
            org.apache.spark
            spark-streaming_2.12
            3.0.0

        

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/779489.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号