hive 1.2
es cluster: 6.8.2 [secured ES, SSL]
建表语句
异常1 Cannot initialize SSLCREATE EXTERAL TABLE test(
....
) TBLPROPERTIES (
'COLUMN_STATS_ACCURATE'='false',
'es.nodes'='secured-nodes',
'es.port'='9200',
'es.nodes.client.only'='true',
'es.input.use.sliced.partitions'='false',
'es.net.http.auth.user'='username',
'es.net.http.auth.pass'='password',
'es.net.ssl' = 'true',
'es.net.ssl.keystore.location' = 'hdfs:///nameserver1/hive/es_certificates/prod-es-secured-key-store.p12',
'es.net.ssl.keystore.pass' = '***',
'es.net.ssl.keystore.type' = 'pkcs12',
'es.net.ssl.truststore.location' = 'hdfs:///nameservice1/hive/es_certificates/prod-es-secured-trusted-store.p12',
'es.net.ssl.truststore.pass' = '***',
'transient_lastDdlTime'='1533294367')
.EsHadoopIllegalStateException: Cannot initialize SSL - Expected to find keystore file at [hdfs:///nameservice1/application/lib/es_certificates/fds-prod-es-secured-key-store.p12] but was unable to. Make sure that it is available on the classpath, or if not, that you have specified a valid URI.', sqlState=None, infoMessages=['*org.apache.hive.service.cli.HiveSQLException:java.io.IOException: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot initialize SSL - Expected to find keystore file at [hdfs:///nameservice1/application/lib/es_certificates/fds-prod-es-secured-key-store.p12] but was unable to.
我把证书文件上传到HDFS上，然后使用hdfs路径，可惜不认识。把证书下载到集群的每个节点，路径改成file:///path/to/*.p12 上述问题解决。
CREATE EXTERAL TABLE test(
....
) TBLPROPERTIES (
'COLUMN_STATS_ACCURATE'='false',
'es.nodes'='secured-nodes',
'es.port'='9200',
'es.nodes.client.only'='true',
'es.input.use.sliced.partitions'='false',
'es.net.http.auth.user'='username',
'es.net.http.auth.pass'='password',
'es.net.ssl' = 'true',
'es.net.ssl.keystore.location' = 'file:///opt/hive/es_certificates/prod-es-secured-key-store.p12',
'es.net.ssl.keystore.pass' = '***',
'es.net.ssl.keystore.type' = 'pkcs12',
'es.net.ssl.truststore.location' = 'file:///opt/hive/es_certificates/prod-es-secured-trusted-store.p12',
'es.net.ssl.truststore.pass' = '***',
'transient_lastDdlTime'='1533294367')
启用调试功能：
hive --hiveconf hive.root.logger=DEBUG, console -e "your create table hql"
可以看到加载证书的日志：
异常2 Cannot initialize SSL - parseAlgParameters failed: ObjectIdentifier() -- data isn't an object ID (tag = 48)DEBUG commonshttp.SSLSocketFactory: Loading keystore located at [file:///opt/hive/es_certificates/prod-es-secured-key-store.p12]
DEBUG commonshttp.SSLSocketFactory: Loading keystore located at [file:///opt/hive/es_certificates/prod-es-secured-trusted-store.p12]
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot initialize SSL - parseAlgParameters failed: ObjectIdentifier() -- data isn't an object ID (tag = 48) at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSSLContext(SSLSocketFactory.java:173) at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.getSSLContext(SSLSocketFactory.java:158) at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSocket(SSLSocketFactory.java:127) at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:482) at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:112) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:380) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:344) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:348) at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:158) at org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion(RestClient.java:574)
在安装了jdk的系统上，执行下面命令
keytool -importkeystore -srckeystore test.p12 [3] -destkeystore test.jks [4] -deststoretype jks
使用jks模式证书
异常3 Cannot initialize SSL - Invalid keystore format'es.net.ssl.keystore.location' = 'file:///opt/hive/es_certificates/prod-es-secured-key-store.jks',
'es.net.ssl.keystore.pass' = '***',
'es.net.ssl.keystore.type' = 'JKS',
'es.net.ssl.truststore.location' = 'file:///opt/hive/es_certificates/prod-es-secured-trusted-store.p12',
'es.net.ssl.truststore.pass' = '***',
Caused by: org.elasticsearch.hadoop.EsHadoopIllegalStateException: Cannot initialize SSL - Invalid keystore format at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSSLContext(SSLSocketFactory.java:173) at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.getSSLContext(SSLSocketFactory.java:158) at org.elasticsearch.hadoop.rest.commonshttp.SSLSocketFactory.createSocket(SSLSocketFactory.java:127) at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707) at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387) at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397) at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323) at org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.execute(CommonsHttpTransport.java:482) at org.elasticsearch.hadoop.rest.NetworkClient.execute(NetworkClient.java:112) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:380) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:344) at org.elasticsearch.hadoop.rest.RestClient.execute(RestClient.java:348) at org.elasticsearch.hadoop.rest.RestClient.get(RestClient.java:158) at org.elasticsearch.hadoop.rest.RestClient.remoteEsVersion(RestClient.java:574) at org.elasticsearch.hadoop.rest.InitializationUtils.discoverEsVersion(InitializationUtils.java:320)
出现这个问题，是由于truststore的证书没有改成jks，全部改成jks。
'es.net.ssl.keystore.location' = 'file:///opt/hive/es_certificates/prod-es-secured-key-store.jks',
'es.net.ssl.keystore.pass' = '***',
'es.net.ssl.keystore.type' = 'JKS',
'es.net.ssl.truststore.location' = 'file:///opt/hive/es_certificates/prod-es-secured-trusted-store.jks',
'es.net.ssl.truststore.pass' = '***',
参考文章：
spark 读写elasticsearch v7.x之1 建立连接 - 知乎
Spark ElasticSearch EsHadoopIllegalArgumentException unable to find keystore with valid URI - Stack Overflow
