Apache Hive安装部署

​

• CentOS7，JDK8，启动Apache Hadoop，启动Mysql5.7（MySql配置远程连接权限）

• 下载Apache Hive 2.3.6版本

  Apache Hive各个版本下载地址 http://archive.apache.org/dist/hive/

• 创建 /hive文件路径，并上传apache-hive-2.3.6-bin.tar.gz到此路径下

• 解压hive安装包

  #  tar -zxvf apache-hive-2.3.6-bin.tar.gz
  

• 添加系统环境变量HIVE_HOME以及修改PATH变量

  #  vim /etc/profile
  

  export JAVA_HOME=/usr/local/src/jdk8
  export JRE_HOME=${JAVA_HOME}/jre
  export CLASS_PATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
  export SPARK_HOME=/spark/spark2.4.8
  export HADOOP_HOME=/hadoop/hadoop-2.7.7
  export HIVE_HOME=/hive/apache-hive-2.3.6-bin
  export PATH=$PATH:$JAVA_HOME/bin:$SPARK_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin
  

  # 保存并退出  使其生效
  source /etc/profile
  

  4、配置文件

• 在安装路径的conf路径下，创建配置文件 hive-site.xml，并写入对应的mysql信息

touch hive-site.xml

	
		javax.jdo.option.ConnectionURL
		jdbc:mysql://192.168.230.132:3306/hive?createDatabaseIfNotExist=true&verifyServerCertificate=false&useSSL=false
	
	
		javax.jdo.option.ConnectionDriverName
		com.mysql.jdbc.Driver
	
	
		javax.jdo.option.ConnectionUserName
		root
	
	
		javax.jdo.option.ConnectionPassword
		123456
	
	
		hive.metastore.warehouse.dir
		/user/hive/warehouse
	
	
		dfs.webhdfs.enabled
		true
	

• 添加MySQL驱动包 mysql-connector-java-5.1.40-bin.jar 该jar包放置在hive的安装根路径下的
  lib目录中，hive要读写MySQL

• 将对应的Hadoop集群中的 core-site.xml 和 hdfs-site.xml 两个配置文件都放置在Hive安装目
  录下conf目录中。

复制hive到其他节点
scp -r apache-hive-2.3.6-bin root@hadoop2:/hive
scp -r apache-hive-2.3.6-bin root@hadoop3:/hive

• 查看Hive的版本 hive --service version

  [root@localhost bin]# hive --service version
  Hive 2.3.6
  Git git://HW13934/Users/gates/tmp/hive-branch-2.3/hive -r 2c2fdd524e8783f6e1f3ef15281cc2d5ed08728f
  Compiled by gates on Tue Aug 13 11:56:35 PDT 2019
  From source with checksum c44b0b7eace3e81ba3cf64e7c4ea3b19
  

• 2.x版本需要对元数据库mysql进行初始化，在bin路径下执行 schematool -dbType mysql -initSchema

  [root@localhost bin]# schematool -dbType mysql -initSchema
  SLF4J: Class path contains multiple SLF4J bindings.
  SLF4J: Found binding in [jar:file:/hive/apache-hive-2.3.6-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  SLF4J: Found binding in [jar:file:/hadoop/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
  SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
  metastore connection URL:	 jdbc:mysql://192.168.230.132:3306/hive?createDatabaseIfNotExist=true&verifyServerCertificate=false&useSSL=false
  metastore Connection Driver :	 com.mysql.jdbc.Driver
  metastore connection User:	 root
  Starting metastore schema initialization to 2.3.0
  Initialization script hive-schema-2.3.0.mysql.sql
  Initialization script completed
  schemaTool completed
  [root@localhost bin]# 
  

• 进入bin路径下 执行hive指令

  [root@localhost bin]# hive
  which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/local/src/jdk8/bin:/spark/spark2.4.8/bin:/hadoop/hadoop-2.7.7/bin:/hadoop/hadoop-2.7.7/sbin:/hive/apache-hive-2.3.6-bin/bin:/root/bin)
  SLF4J: Class path contains multiple SLF4J bindings.
  SLF4J: Found binding in [jar:file:/hive/apache-hive-2.3.6-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  SLF4J: Found binding in [jar:file:/hadoop/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
  SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
  SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
  
  Logging initialized using configuration in jar:file:/hive/apache-hive-2.3.6-bin/lib/hive-common-2.3.6.jar!/hive-log4j2.properties Async: true
  Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
  hive> show databases;
  OK
  default
  Time taken: 3.055 seconds, Fetched: 1 row(s)
  hive> 
  

• exit；

  hive> exit;
  

• 创建hive配置文件中所需要的hdfs路径并赋予一定的权限

  [root@localhost bin]# hdfs dfs -mkdir -p /usr/hive/warehouse
  [root@localhost bin]# hdfs dfs -mkdir -p /usr/hive/tmp
  [root@localhost bin]# hdfs dfs -mkdir -p /usr/hive/log 
  [root@localhost bin]# hdfs dfs -chmod g+w /usr/hive/warehouse
  [root@localhost bin]# hdfs dfs -chmod g+w /usr/hive/tmp
  [root@localhost bin]# hdfs dfs -chmod g+w /usr/hive/log
  

[root@localhost apache-hive-2.3.6-bin]# mkdir log

修改的Hadoop集群中的 core-site.xml 和 hdfs-site.xml，所有节点都的修改。重启Hadoop集群

• 修改hadoop集群的hdfs-site.xml配置文件：加入一条配置信息，表示启用webhdfs

  
      dfs.webhdfs.enabled
      true
  
  

• 修改hadoop集群的core-site.xml配置文件：加入两条配置信息：表示设置hadoop集群的代理用户

  
      hadoop.proxyuser.root.hosts
      *
  
  
      hadoop.proxyuser.root.groups
      *
  
  

• 重启Hadoop服务

• 启动Hiveserver2服务

  [root@localhost bin]# nohup hiveserver2 1>/hive/apache-hive-2.3.6-bin/log/hiveserver.log 2>/hive/apache-hive-2.3.6-bin/log/hiveserver.err &
  

• 启动beeline客户端，HUE,HDP的hive界面

  [root@localhost bin]# beeline -u jdbc:hive2://192.168.230.132:10000 -n root
  

• 扩展：Hiveserver2 Web UI

  • Hive从2.0 版本开始，为 HiveServer2 提供了一个简单的 WEB UI 界面，界面中可以直观的看到当前链
    接的会话、历史日志、配置参数以及度量信息。

  • 修改hive-site.xml 配置文件

    
        hive.server2.webui.host
        192.168.230.132
    
    
        hive.server2.webui.port
        15010
    
    

  • 重启 Hiveserver2，访问 Web UI： http://192.168.230.132:15010

    192.168.230.132
    

    hive.server2.webui.port 15010 ~~~

  • 重启 Hiveserver2，访问 Web UI： http://192.168.230.132:15010