栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

springboot整合elasticsearch实践,两种方式操作索引文档,解析附件中的内容,搜索附件中的中文

springboot整合elasticsearch实践,两种方式操作索引文档,解析附件中的内容,搜索附件中的中文

目录

一、依赖引入

二、修改配置文件

2.1、配置elasticsearch的服务地址和端口号

2.2、编写配置类连接elasticsearch

 2.3、实体类加入注解

2.4、dao继承ElasticsearchRepository

2.5、创建service接口和实现类,模拟调用。

三、开始编码测试

        3.1 、创建文档

3.2 、批量创建

3.3 、查询方法

3.4、删除方法

3.5、分页查询,条件查询

四、上传附件,可搜索附件中的中文

4.1、需要先安装附件解析插件ingest-attachment

4.2、定义管道

五、多附件上传解析

5.1、定义多附件的管道pipeline

5.2、上传附件代码实现如下


一、依赖引入

            org.springframework.boot
            spring-boot-starter-data-elasticsearch
        
        
        
            commons-io
            commons-io
            2.5
        
        
            org.elasticsearch
            elasticsearch
            7.9.0
        
        
        
            org.elasticsearch.client
            elasticsearch-rest-high-level-client
            7.9.0
        
        
        
            org.apache.logging.log4j
            log4j-api
            2.8.2
        
        
            org.apache.logging.log4j
            log4j-core
            2.8.2
        
        
            com.fasterxml.jackson.core
            jackson-databind
            2.9.9
        
        
        
            junit
            junit
            4.12
        

        
            org.springframework.boot
            spring-boot-devtools
            runtime
            true
        
        
            org.springframework.boot
            spring-boot-configuration-processor
            true
        
        
            org.projectlombok
            lombok
            true
        
        
            org.springframework.boot
            spring-boot-starter-test
            test
        
        
            junit
            junit
            4.13.1
            test
        
        
            org.springframework
            spring-web
            5.3.3
        
        
            com.alibaba
            fastjson
            1.2.61
        

二、修改配置文件

2.1、配置elasticsearch的服务地址和端口号

2.2、编写配置类连接elasticsearch
package com.whx.springbootelasticsearch.config;

import lombok.Data;
import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Data
@Configuration
@ConfigurationProperties(prefix = "elasticsearch")
public class ElasticsearchConfig {

    private String host;

    private Integer port;

    
    @Bean
    public RestHighLevelClient restHighLevelClient(){
        RestClientBuilder builder = RestClient.builder(new HttpHost(host, port, "http"));
        RestHighLevelClient restHighLevelClient = new RestHighLevelClient(builder);
        return restHighLevelClient;
    }

}

 2.3、实体类加入注解

2.4、dao继承ElasticsearchRepository
package com.whx.springbootelasticsearch.dao;

import com.whx.springbootelasticsearch.entity.SendReport;
import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface SendReportDao extends ElasticsearchRepository {


}

2.5、创建service接口和实现类,模拟调用。
package com.whx.springbootelasticsearch.service;

import com.whx.springbootelasticsearch.entity.SendReport;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;

import java.util.List;

public interface SendReportService {

    SendReport saveSendReport(SendReport sendReport);

    SendReport getById(String id);

    void deleteById(String id);

    Iterable saveAll(List sendReports);

    Page findAll(Pageable page);

}

实现类

package com.whx.springbootelasticsearch.service;

import com.whx.springbootelasticsearch.dao.SendReportDao;
import com.whx.springbootelasticsearch.entity.SendReport;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.stereotype.Service;

import java.util.List;
import java.util.Optional;

@Service
public class SendReportServiceImpl implements SendReportService {

    private SendReportDao sendReportDao;

    @Autowired
    public void setSendReportDao(SendReportDao sendReportDao) {
        this.sendReportDao = sendReportDao;
    }

    @Override
    public SendReport saveSendReport(SendReport sendReport) {

        SendReport save = sendReportDao.save(sendReport);
        return save;
    }

    @Override
    public SendReport getById(String id) {
        Optional sendReportDaoById = sendReportDao.findById(id);
        return sendReportDaoById.get();
    }

    @Override
    public void deleteById(String id) {
        sendReportDao.deleteById(id);
    }

    @Override
    public Iterable saveAll(List sendReports) {
        Iterable reportIterable = sendReportDao.saveAll(sendReports);
        return reportIterable;
    }

    @Override
    public Page findAll(Pageable page) {
        Page all = sendReportDao.findAll(page);
        return all;
    }


}

三、开始编码测试

        3.1 、创建文档
package com.whx.springbootelasticsearch;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.whx.springbootelasticsearch.entity.SendReport;
import com.whx.springbootelasticsearch.service.SendReportService;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.indices.CreateIndexResponse;
import org.elasticsearch.common.xcontent.XContentType;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

@SpringBootTest
public class ESTest_create {

    @Autowired
    private RestHighLevelClient restHighLevelClient;

    @Autowired
    private SendReportService sendReportService;

    
    @Test
    public void testCreate(){
        SendReport sendReport = new SendReport();
        sendReport.setId("1001");
        sendReport.setBizTitle("这个是标题1");
        sendReport.setItemCode("123");
        sendReport.setReportType("1");

        SendReport saveSendReport = sendReportService.saveSendReport(sendReport);

        System.out.println(saveSendReport);
    }


    
    @Test
    public void testCreate2() throws Exception {
        SendReport sendReport = new SendReport();
        sendReport.setId("1002");
        sendReport.setBizTitle("标题22");
        sendReport.setItemCode("1234");
        sendReport.setReportType("1");

        IndexRequest indexRequest = new IndexRequest("send_report");
        indexRequest.id(sendReport.getId()).source(new ObjectMapper().writevalueAsString(sendReport), XContentType.JSON);
        CreateIndexRequest createIndexRequest = new CreateIndexRequest("send_report");
        createIndexRequest.source(new ObjectMapper().writevalueAsString(sendReport), XContentType.JSON);
        //相同的id就会是更新,否则是创建
        IndexResponse index = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
        System.out.println(index.getResult());
    }

}

3.2 、批量创建
package com.whx.springbootelasticsearch;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.whx.springbootelasticsearch.entity.SendReport;
import com.whx.springbootelasticsearch.service.SendReportService;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.bulk.BulkResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.common.time.DateUtils;
import org.elasticsearch.common.xcontent.XContentType;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;

@SpringBootTest
public class ESTest_create_batch {

    @Autowired
    private RestHighLevelClient restHighLevelClient;

    @Autowired
    private SendReportService sendReportService;

    @Test
    public void testCreateBatch(){
        List sendReports = new ArrayList<>();
        sendReports.add(new SendReport().setId("1001").setBizTitle("这是标题1").setItemCode("11").setReportType("1").setCreatedBy(11l).setCreatedDate(new Date()));
        sendReports.add(new SendReport().setId("1002").setBizTitle("这是标题2").setItemCode("22").setReportType("1").setCreatedBy(11l).setCreatedDate(new Date()));
        sendReports.add(new SendReport().setId("1003").setBizTitle("这是标题3").setItemCode("33").setReportType("2").setCreatedBy(11l).setCreatedDate(new Date()));
        sendReports.add(new SendReport().setId("1004").setBizTitle("这是标题4").setItemCode("44").setReportType("2").setCreatedBy(11l).setCreatedDate(new Date()));

        Iterable iterable = sendReportService.saveAll(sendReports);
        for (SendReport sendReport : iterable) {
            System.out.println(sendReport);
        }
    }


    @Test
    public void testCreateBatch2() throws Exception {
        BulkRequest bulkRequest = new BulkRequest();
        String json5 = new ObjectMapper().writevalueAsString(new SendReport().setId("2005").setBizTitle("前端开发").setItemCode("55").setReportType("3").setCreatedBy(22l));
        String json6 = new ObjectMapper().writevalueAsString(new SendReport().setId("2006").setBizTitle("后端开发").setItemCode("66").setReportType("3").setCreatedBy(22l).setCreatedDate(new Date()));
        String json7 = new ObjectMapper().writevalueAsString(new SendReport().setId("2007").setBizTitle("这是随便起的标题").setItemCode("77").setReportType("4").setCreatedBy(22l).setCreatedDate(new Date()));
        String json8 = new ObjectMapper().writevalueAsString(new SendReport().setId("2008").setBizTitle("不知道标题叫什么了").setItemCode("88").setReportType("4").setCreatedBy(22l));

        bulkRequest.add(new IndexRequest("send_report").id("2005").source(json5, XContentType.JSON));
        bulkRequest.add(new IndexRequest("send_report").id("2006").source(json6, XContentType.JSON));
        bulkRequest.add(new IndexRequest("send_report").id("2007").source(json7, XContentType.JSON));
        bulkRequest.add(new IndexRequest("send_report").id("2008").source(json8, XContentType.JSON));

        BulkResponse bulk = restHighLevelClient.bulk(bulkRequest, RequestOptions.DEFAULT);
        boolean b = !bulk.hasFailures();
        System.out.println("批量创建成功:" + b);
        System.out.println("failureMessage:" + bulk.buildFailureMessage());
    }

}

3.3 、查询方法
package com.whx.springbootelasticsearch;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.whx.springbootelasticsearch.entity.SendReport;
import com.whx.springbootelasticsearch.service.SendReportService;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

@SpringBootTest
public class ESTest_get {

    @Autowired
    private RestHighLevelClient restHighLevelClient;

    @Autowired
    private SendReportService sendReportService;

    
    @Test
    public void testGet(){
        SendReport sendReport = sendReportService.getById("1001");
        System.out.println(sendReport);
    }


    
    @Test
    public void testGet2() throws Exception {
        GetRequest getRequest = new GetRequest("send_report").id("1001");

        GetResponse response = restHighLevelClient.get(getRequest, RequestOptions.DEFAULT);
        System.out.println(response.getSourceAsString());
    }

    
    @Test
    public void testGet3() throws Exception {
        SearchRequest searchRequest = new SearchRequest();
        searchRequest.indices("send_report");

        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        SearchHits hits = searchResponse.getHits();
        for (SearchHit hit : hits) {
            System.out.println(hit.getSourceAsString());
        }

    }

}

3.4、删除方法
package com.whx.springbootelasticsearch;

import com.whx.springbootelasticsearch.service.SendReportService;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;

@SpringBootTest
public class ESTest_delete {

    @Autowired
    private RestHighLevelClient restHighLevelClient;

    @Autowired
    private SendReportService sendReportService;

    
    @Test
    public void testDelete(){
        sendReportService.deleteById("1001");
        System.out.println("删除成功");
    }


    
    @Test
    public void testDelete2() throws Exception {
        DeleteRequest deleteRequest = new DeleteRequest("send_report").id("1002");

        DeleteResponse deleteResponse = restHighLevelClient.delete(deleteRequest, RequestOptions.DEFAULT);
        System.out.println(deleteResponse.toString());
    }


}

3.5、分页查询,条件查询
package com.whx.springbootelasticsearch;

import com.whx.springbootelasticsearch.entity.SendReport;
import com.whx.springbootelasticsearch.service.SendReportService;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.unit.Fuzziness;
import org.elasticsearch.common.unit.Timevalue;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;

@SpringBootTest
public class ESTest_search {

    @Autowired
    private RestHighLevelClient restHighLevelClient;

    @Autowired
    private SendReportService sendReportService;

    @Test
    public void testSearch(){
        //构造分页,第一页,每页3条
        Pageable pageable = Pageable.ofSize(3).withPage(0);
        Page page = sendReportService.findAll(pageable);
        System.out.println("totalPage:" + page.getTotalPages());
        System.out.println("totalElement:" + page.getTotalElements());
        for (SendReport sendReport : page.getContent()) {
            System.out.println(sendReport);
        }
    }

    @Test
    public void testSearch2() throws Exception {
        SearchRequest searchRequest = new SearchRequest("send_report");
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.matchAllQuery());
        searchSourceBuilder.from(0); //从第一页开始
        searchSourceBuilder.size(3); //每页3条记录

        searchRequest.source(searchSourceBuilder);
        SearchResponse response = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        float maxScore = response.getHits().getMaxScore();
        System.out.println("maxScore=" + maxScore);
        for (SearchHit hit : response.getHits().getHits()) {
            System.out.println(hit.getSourceAsString());
        }
    }

    @Test
    public void testSearch3() throws Exception {
        // 创建BoolQueryBuilder
        BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();

        // 子查询“且”关系
        BoolQueryBuilder childBoolQueryBuilder = new BoolQueryBuilder()
                .must(QueryBuilders.matchPhraseQuery("bizTitle", "标题7")); //不分词查询matchPhraseQuery
        BoolQueryBuilder childBoolQueryBuilder2 = new BoolQueryBuilder()
                .must(QueryBuilders.matchQuery("bizTitle", "标题7")); //分词查询matchQuery
//        BoolQueryBuilder childBoolQueryBuilder3 = new BoolQueryBuilder()
//                .must(QueryBuilders.nestedQuery("send_report",
//                        QueryBuilders.boolQuery()
//                                .must(QueryBuilders.rangeQuery("annualRevenue.val").gt(30).lte(90)), ScoreMode.None)
//                );
//        boolQueryBuilder.must(childBoolQueryBuilder);
        boolQueryBuilder.must(childBoolQueryBuilder2);
//        boolQueryBuilder.must(childBoolQueryBuilder3);
        // 创建SearchSourceBuilder
        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        // 查询条件生成DSL语句
        searchSourceBuilder.query(boolQueryBuilder);
        // 从多少
        searchSourceBuilder.from(0);
        // 查多少条数据,如果设置“0”返回count数量
        searchSourceBuilder.size(3);
        // 排序规则 字符串排序会报错
//        searchSourceBuilder.sort("id", SortOrder.ASC);
        // 设置超时
        Timevalue t=new Timevalue(3000);
        searchSourceBuilder.timeout(t);

        SearchRequest searchRequest = new SearchRequest("send_report");
        searchRequest.source(searchSourceBuilder);
        SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        System.out.println("search total:" + searchResponse.getHits().getTotalHits().value);
        for (SearchHit hit : searchResponse.getHits().getHits()) {
            System.out.println(hit.getSourceAsString());
        }

    }

四、上传附件,可搜索附件中的中文

elasticesearch可检索附件中的文字,首先需要安装附件解析插件,定义附件管道pipeline,上传附件将附件中的文字解析出来,后续即可查询。

上传附件有两种方式:一种是将附件转成base64编码上传,第二种是指定cbor方式直接上传附件。

4.1、需要先安装附件解析插件ingest-attachment

ingest-attachment 下载

https://artifacts.elastic.co/downloads/elasticsearch-plugins/ingest-attachment/ingest-attachment-7.9.0.zip

版本号不一样,将7.9.0换成自己的版本下载即可。

下载后解压到es的plugins目录下,启动es

4.2、定义管道

4.2.1、官网的介绍如下

 

 4.2.2、代码实现如下

    
    @Test
    public void putAttachmentPipeline() throws Exception {
        String source = "{n" +
                "  "description" : "Extract attachment information",n" +
                "  "processors" : [n" +
                "    {n" +
                "      "attachment" : {n" +
                "        "field" : "data",n" +
                "        "properties": [ "content", "title" ],n" +
                "        "indexed_chars": -1,n" +
                "        "ignore_missing": truen" +
                "      },n" +
                "      "remove": {n" +
                "        "field": "data"n" +
                "      }n" +
                "    }n" +
                "  ]n" +
                "}";

        PutPipelineRequest pipelineRequest = new PutPipelineRequest("attachment",new BytesArray(source),XContentType.JSON);

        AcknowledgedResponse acknowledgedResponse = restHighLevelClient.ingest().putPipeline(pipelineRequest, RequestOptions.DEFAULT);
        System.out.println("创建管道成功:" + acknowledgedResponse.isAcknowledged());
    }

"remove":{

        "field" : "data"

}

指将附件上传解析到附件中的内容之后将源文件移除,因为附件很占空间,而且我们只需要用来检索,并不需要源文件。

    @Test
    public void putCBORAttachmentPipeline() throws Exception {
        String source = "{n" +
                "  "description" : "Extract attachment information",n" +
                "  "processors" : [n" +
                "    {n" +
                "      "attachment" : {n" +
                "        "field" : "data",n" +
                "        "indexed_chars": -1,n" +
                "        "ignore_missing": truen" +
                "      },n" +
                "      "remove": {n" +
                "        "field": "data"n" +
                "      }n" +
                "    }n" +
                "  ]n" +
                "}";

        PutPipelineRequest pipelineRequest = new PutPipelineRequest("cbor-attachment",new BytesArray(source),XContentType.JSON);

        AcknowledgedResponse acknowledgedResponse = restHighLevelClient.ingest().putPipeline(pipelineRequest, RequestOptions.DEFAULT);
        System.out.println("创建管道成功:" + acknowledgedResponse.isAcknowledged());
    }
    @Test
    public void testPutFile3() throws Exception {
//        String path = "D:\test\testText3.txt";
        String path = "D:\test\测试pdf.pdf";
        File file = new File(path);
        FileInputStream inputFile = new FileInputStream(file);
        byte[] buffer = new byte[(int)file.length()];
        inputFile.read(buffer);
        inputFile.close();
        //将文件转成base64编码
        String fileString = base64Utils.encodeToString(buffer);

        XContentBuilder builder = XContentFactory.jsonBuilder();
        builder.startObject();
        {
            builder.field("data", fileString);
            builder.field("title", file.getName());
        }
        builder.endObject();
//        Map map = new HashMap<>();
//        map.put("data", buffer);
//        map.put("title", file.getName());
//        map.put("name", "测试文件上传base64转码的文件pdf格式的");

        IndexRequest indexRequest = new IndexRequest("test_file").id("222base64");
        indexRequest.source(builder);
//        indexRequest.source(JSON.toJSonString(map), XContentType.JSON);

        //设置文件管道attachment
        indexRequest.setPipeline("attachment");

        IndexResponse index = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
        System.out.println(index.getResult());
    }
    @Test
    public void testPut() throws Exception {
//        String path = "D:\test\testText.txt";
        String path = "D:\test\测试pdf.pdf";
        File file = new File(path);
        FileInputStream inputFile = new FileInputStream(file);
        byte[] buffer = new byte[(int)file.length()];
        inputFile.read(buffer);
        inputFile.close();
        //构建cbor
        XContentBuilder xContentBuilder = XContentFactory.cborBuilder();
        xContentBuilder.startObject();
        xContentBuilder.field("data", buffer);
        xContentBuilder.endObject();

        IndexRequest indexRequest = new IndexRequest("test_file").id("222");
        //将文件data放入
        indexRequest.source(xContentBuilder);

        //设置文件管道cbor-attachment
        indexRequest.setPipeline("cbor-attachment");

        IndexResponse index = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
        System.out.println(index.getResult());
    }

查询文档内容:

 4.2.3、查询文档附件中的内容

    @Test
    public void getDoc() throws Exception {
        SearchRequest searchRequest = new SearchRequest();
        searchRequest.indices("test_file");

        SearchResponse search = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        System.out.println("total hits: " + search.getHits().getTotalHits().value);
        for (SearchHit hit : search.getHits().getHits()) {
            System.out.println(hit.getSourceAsString());
        }
    }

    @Test
    public void testSearch() throws Exception {
        String keyword = "测试";

        SearchRequest searchRequest = new SearchRequest("test_file");

        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
        sourceBuilder.query(QueryBuilders.multiMatchQuery(keyword,"attachment.content","attachment.title"));
        sourceBuilder.from(0); //从第几页开始
        sourceBuilder.size(10);
        searchRequest.source(sourceBuilder);

        SearchResponse response = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);
        System.out.println("total hits: " + response.getHits().getTotalHits().value);
        for (SearchHit hit : response.getHits().getHits()) {
            System.out.println(hit.getSourceAsString());
        }

    }

五、多附件上传解析

5.1、定义多附件的管道pipeline
{
  "attachments" : {
    "description" : "Extract attachment information from arrays",
    "processors" : [
      {
        "foreach" : {
          "field" : "attachments",
          "processor" : {
            "attachment" : {
              "target_field" : "_ingest._value.attachment",
              "field" : "_ingest._value.data"
            }
          }
        }
      },
      {
        "foreach" : {
          "field" : "attachments",
          "processor" : {
            "remove" : {
              "field" : "_ingest._value.data"
            }
          }
        }
      }
    ]
  }
}

5.2、上传附件代码实现如下
@Test
    public void testPutFile2() throws Exception {
        String path = "D:\test\testText3.txt";
        String path2 = "D:\test\测试pdf.pdf";
        File file = new File(path);
        FileInputStream inputFile = new FileInputStream(file);
        byte[] buffer = new byte[(int)file.length()];
        inputFile.read(buffer);
        inputFile.close();
        //将文件转成base64编码
        String fileString = base64Utils.encodeToString(buffer);

        //附件2
        File file2 = new File(path2);
        FileInputStream inputFile2 = new FileInputStream(file2);
        byte[] buffer2 = new byte[(int)file2.length()];
        inputFile2.read(buffer2);
        inputFile2.close();
        //将文件转成base64编码
        String fileString2 = base64Utils.encodeToString(buffer2);


        Map map = new HashMap<>();

        Map data1 = new HashMap<>();
        data1.put("data", fileString);
        data1.put("filename", file.getName());

        Map data2 = new HashMap<>();
        data2.put("data", fileString2);
        data2.put("filename", file2.getName());

        List> mapList = Arrays.asList(data1, data2);
        map.put("attachments", mapList);
        map.put("name", "测试多文件上传");

        IndexRequest indexRequest = new IndexRequest("test_file").id("12345");
        indexRequest.source(map);

        //设置文件管道attachments
        indexRequest.setPipeline("attachments");

        IndexResponse index = restHighLevelClient.index(indexRequest, RequestOptions.DEFAULT);
        System.out.println(index.getResult());
    }

结果如下

elasticsearch官方文档:7.9.0版本文档

elastisearch高级客户端:Java REST Client [7.9]

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/758482.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号