ElasticSearch相关以及升级到7.4.2等

1、ES在Linux中搭建

1、下载elasticsearch包

[root@gz-vcard-dev-app16-120-108 etc]# wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.4.2.tar.gz

2、解压

[root@gz-vcard-dev-app16-120-108 etc]# tar -zxvf elasticsearch-7.4.2.tar.gz -C /usr/local/

3、启动报错

错误一：can not run elasticsearch as root

从5.0开始 elasticsearch 安全级别提高了不允许采用root帐号启动所以我们要添加一个用户用来启动 elasticsearch

错误二：max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

修改/etc/security/limits.conf文件在文件末尾添加如下

错误三：max number of threads [3818] for user [es] is too low, increase to at least [4096]

/etc/sysctl.conf文件末尾添加 vm.max_map_count = 2621441

错误四：with lock id [0]; maybe these locations are not writable or multiple nodes were started without increasing [node.max_local_storage_nodes] (was [1])?

           vim config/elasticsearch.yml 文件随意地方添加变量

           node.max_local_storage_nodes: 256

4、创建用户

[root@gz-vcard-dev-app16-120-108 etc]# useradd es //创建es用户

[root@gz-vcard-dev-app16-120-108 etc]# chown -R es:es /usr/local/elasticsearch-7.4.2///把目录权限赋予给es用户

[root@gz-vcard-dev-app16-120-108 etc]# su es //切换至es用户

5、更改配置文件

vim elasticsearch.yml

network.host:本机ip

http.port:端口

6、启动

普通启动：[es@gz-vcard-dev-app16-120-108 bin]$ sh /usr/local/elasticsearch-7.4.2/bin/elasticsearch

后台启动：[es@gz-vcard-dev-app16-120-108 bin]$ ./elasticsearch -d

访问地址：http://10.250.120.108:9200

7、kibana和elasticsearch进行整合

1.1、下载kibana包

Kibana官方下载地址：Download Kibana Free | Get Started Now | Elastic

根据ElasticSearch版本及安装环境下载相应的Kibana安装包尽量低版本要么版本一致

https://artifacts.elastic.co/downloads/kibana/kibana-5.3.0-linux-x86_64.tar.gz

解压：[root@gz-vcard-dev-app16-120-108 local]# tar -xvf kibana-7.5.1-linux-x86_64.tar.gz

1.2、配置kibana

[root@gz-vcard-dev-app16-120-108 config]# pwd
/usr/local/kibana-7.5.1-linux-x86_64/config
[root@gz-vcard-dev-app16-120-108 config]# vim kibana.yml

1.3、kibana启动

切换目录至kibana中bin

[root@gz-vcard-dev-app16-120-108 bin]# pwd
/usr/local/kibana-7.5.1-linux-x86_64/bin

[root@gz-vcard-dev-app16-120-108 bin]# ./kibana &

2.ES升级到7.4.2

1.设置默认值：

POST vcomloan_merch_info/_update_by_query
{
"script": {
"lang": "painless",
"source": "ctx._source.refund_status= '0'"
}
}

2.查询示例

GET /vcomloan_merch_info/_search
{
"from" : 0,
"size" : 10,
"query" : {
"bool" : {
"must" : [
{
"term" : {
"apply_no": {
"value" : "1653767874181795848",
"boost" : 1.0
}
}
}
],
"adjust_pure_negative" : true,
"boost" : 1.0
}
}
}

3. Java 中es相关操作

5.x版本与7.x版本差异对比

elasticsearch 5.x --TransportClient

elasticsearch 7.x --RestHighLevelClient

先判断当前数据是否存在

public boolean existData(RestHighLevelClient client,String indexName,  String orderNo) {

    BoolQueryBuilder builder = QueryBuilders.boolQuery();
    builder.must(QueryBuilders.termQuery(EsMerchIndexKey.ORDER_NO, orderNo ));
    SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    sourceBuilder.query(builder);

    SearchRequest searchRequest = new SearchRequest(indexName);
    searchRequest.source(sourceBuilder);
    SearchResponse searchResponse = null;
    try{
        logger.debug("elasticSearch检查数据是否存在existData入参:{}", sourceBuilder);
        searchResponse= client.search(searchRequest,RequestOptions.DEFAULT);
    }catch(Exception e){
        logger.error("queryListData查询异常：{}", e.getMessage());
    }
    if(searchResponse.getHits().getTotalHits().value>0){
        return true;
    }else{
        return false;

    }

}

增加：

增加 ：

   IndexResponse response = client.prepareIndex()
                    .setIndex(indexName)
                    .setType(typeName)
                    .setSource(asString, XContentType.JSON)
                    .setId(id).get();
 return "created".equals(response.getResult().getLowercase()) ? 1 : 0;

elasticsearch 7.x

IndexRequest request = new IndexRequest(indexName).source(asString,XContentType.JSON);
request.id(orderNo);
IndexResponse response = client.index(request, RequestOptions.DEFAULT);
return "created".equals(response.getResult().getLowercase()) ? 1 : 0;

修改

elasticsearch 5.x

            UpdateResponse updateResponse = client.prepareUpdate(indexName, typeName, id)
                    .setDoc(asString, XContentType.JSON).get();

 return "updated".equals(updateResponse.getResult().getLowercase()) ? 1 : 0;

elasticsearch 7.x

String docId = EsHelper.getByDocId(client,indexName,orderNo);
String asString = objectMapper.writevalueAsString(object);
UpdateRequest updateRequest = new UpdateRequest(indexName,docId).doc(asString,XContentType.JSON);
UpdateResponse updateResponse = client.update(updateRequest, RequestOptions.DEFAULT);
return "updated".equals(updateResponse.getResult().getLowercase()) ? 1 : 0;

查询：

elasticsearch 5.x

SearchRequestBuilder searchRequestBuilder = client.prepareSearch(indexName)
        .setTypes(typeName)
        .setQuery(queryBuilder)
        .setFrom(0)
        .setSize(1).setTimeout(timevalue);

logger.debug("elasticSearch根据多条件查询详情入参:{}", searchRequestBuilder);

SearchResponse searchResponse = searchRequestBuilder.get();
SearchHits searchHits = searchResponse.getHits();

HashMap objectObjectHashMap = new HashMap<>();
for(SearchHit hit : searchHits.getHits()){
    objectObjectHashMap.putAll(hit.getSource());
}

return objectObjectHashMap;

elasticsearch 7.x

SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(queryBuilder);
SearchRequest searchRequest = new SearchRequest(indexName);
searchRequest.source(sourceBuilder);
SearchResponse searchResponse = null;
try{
    log.debug("elasticSearch根据多条件查询详情queryDetailMsg入参:{}", sourceBuilder);
    searchResponse= client.search(searchRequest,RequestOptions.DEFAULT);
}catch(Exception e){
    log.error("queryDetailMsg查询异常：{}", e.getMessage());
    throw new MsException(ErrorCode.ERR_SYSTEM, "系统异常");
}
SearchHits searchHits = searchResponse.getHits();
SearchHit[] hits=searchHits.getHits();
Map objectObjectHashMap =null;
if(hits!=null&&hits.length>0){
    objectObjectHashMap= hits[0].getSourceAsMap();
}
return objectObjectHashMap;

统计：

elasticsearch 5.x
SearchRequestBuilder searchRequestBuilder = client.prepareSearch(indexName)
        .setTypes(typeName)
        .setQuery(queryBuilder)
        .setFrom(pageNo)
        .setSize(pageSize)
        .addAggregation(termsBuilder);

logger.debug("elasticSearch搜索统计接口请求入参：[{}]", searchRequestBuilder);

return searchRequestBuilder.get();

elasticsearch 7.x

SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(queryBuilder);
sourceBuilder.aggregation(termsBuilder);
sourceBuilder.from(pageNo);
sourceBuilder.size(pageSize);
SearchRequest searchRequest = new SearchRequest(indexName);
searchRequest.source(sourceBuilder);
SearchResponse searchResponse = null;
try{
    log.debug("elasticSearch搜索统计接口statisticsData请求入参：[{}]", sourceBuilder);
    searchResponse= client.search(searchRequest,RequestOptions.DEFAULT);
}catch(Exception e){
    log.error("statisticsData查询异常：{}", e.getMessage());
    throw new MsException(ErrorCode.ERR_SYSTEM, "系统异常");
}

elasticsearch 5.x

getByIdData

GetRequestBuilder getRequestBuilder = client.prepareGet(indexName, typeName, id);

GetResponse getResponse = getRequestBuilder.get();

return getResponse.getSource();

elasticsearch 7.x

BoolQueryBuilder builder = QueryBuilders.boolQuery();
builder.must(QueryBuilders.termQuery(EsMerchIndexKey.ORDER_NO, id ));
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(builder);

SearchRequest searchRequest = new SearchRequest(indexName);
searchRequest.source(sourceBuilder);
SearchResponse searchResponse = null;
try{
    searchResponse= client.search(searchRequest,RequestOptions.DEFAULT);
}catch(Exception e){
    log.error(QUERY_LIST_DATA_EXCEPTION, e.getMessage());
    throw new MsException(ErrorCode.ERR_SYSTEM, "系统异常");
}
SearchHits hits =searchResponse.getHits();
SearchHit[] searchHits = hits.getHits();
Map sourceAsMap=null;
if(searchHits!=null&&searchHits.length>0){
    sourceAsMap= searchHits[0].getSourceAsMap();
}
return sourceAsMap;

ElasticSearch相关以及升级到7.4.2等

大数据系统相关栏目本月热门文章