IDEA中Spark整合hive案例

需要IDEA配置好Scala

1、新建一个maven工程

2、新建一个Scala类

3、导入依赖

        
            junit
            junit
            4.11
            test
        
        
            org.apache.spark
            spark-core_2.11
            2.1.1
        
        
            org.apache.spark
            spark-sql_2.11
            2.1.1
        
        
            org.apache.spark
            spark-hive_2.11
            2.1.1
        
        
            org.apache.spark
            spark-graphx_2.11
            2.1.1
        
        
            mysql
            mysql-connector-java
            5.1.25
        
    

4、hive中创建表，并插入一条数据create table test(id int,name string);
insert into test values(1,"zs");

5、操作代码如下：

def main(args: Array[String]): Unit = {
    val spark: SparkSession = SparkSession.builder().appName("SparkToHive")
      .master("local[*]").config("hive.metastore.uris", "thrift://192.168.91.135:9083")
      .enableHiveSupport()
      .getOrCreate()
      
    val ordersDF: Dataframe = spark.sql("select * from test")
    ordersDF.printSchema()
    ordersDF.show()
  }

6、操作结果