栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

A master URL must be set in your configuration

A master URL must be set in your configuration

org.apache.spark.SparkException: A master URL must be set in your configuration
	at org.apache.spark.SparkContext.(SparkContext.scala:376)
	at com.ck.data.batch.customer.CustomerDateCountJob$.main(CustomerDateCountJob.scala:26)
	at com.ck.data.batch.customer.CustomerDateCountJob.main(CustomerDateCountJob.scala)
2021-12-30 16:09:01,873 ERROR Utils - Uncaught exception in thread main
java.lang.NullPointerException
	at org.apache.spark.SparkContext.org$apache$spark$SparkContext$$postApplicationEnd(SparkContext.scala:2591)
	at org.apache.spark.SparkContext$$anonfun$stop$1.apply$mcV$sp(SparkContext.scala:2099)
	at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1531)
	at org.apache.spark.SparkContext.stop(SparkContext.scala:2098)
	at org.apache.spark.SparkContext.(SparkContext.scala:612)
	at com.ck.data.batch.customer.CustomerDateCountJob$.main(CustomerDateCountJob.scala:26)
	at com.ck.data.batch.customer.CustomerDateCountJob.main(CustomerDateCountJob.scala)
2021-12-30 16:09:01,881 INFO  SparkContext - Successfully stopped SparkContext
Exception in thread "main" org.apache.spark.SparkException: A master URL must be set in your configuration
	at org.apache.spark.SparkContext.(SparkContext.scala:376)
	at com.ck.data.batch.customer.CustomerDateCountJob$.main(CustomerDateCountJob.scala:26)
	at com.ck.data.batch.customer.CustomerDateCountJob.main(CustomerDateCountJob.scala)

主要是master没有设置

.setMaster("local")

def main(args: Array[String]): Unit = {
    val sparkConf = new SparkConf().setAppName("name").setMaster("local")
    val sc = new SparkContext(sparkConf)
    process(sc)
    sc.stop()
  }

 由于又要提交上线每次又必须删除,所以决定在idea的opt配置一个全局的,不用每次添加删除

配置完之后删除已有的Application,如下截图中序号2上面的那个CustomerDateCountJob(每个人的都不同)

 

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/687904.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号