问题1:spark版本问题
Caused by: java.lang.UnsatisfiedlinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.createDirectoryWithMode0(Ljava/lang/String;I)V
一般碰到结尾为V,很可能是版本问题
解决办法:
问题2:明明hue是可以直接创建的,但是spark代码一直报库不存在
org.apache.spark.sql.catalyst.analysis.NoSuchDatabaseException: Database 'user_profile0224' not found;
解决办法:
a.检查hadooponwindows-master环境变量
b.增加配置
val sparkConf: SparkConf = new SparkConf().setAppName("task_sql_app")
.setMaster("local[*]")
// .set("hive.metastore.uris","thrift://cdh01:9083")
val sparkSession: SparkSession = SparkSession.builder()
.config(sparkConf)
.config("hive.metastore.uris","thrift://cdh01:9083")
.config("spark.sql.warehouse.dir","hdfs://cdh01:8020/user/hive/warehouse")
.enableHiveSupport()
.getOrCreate()



