Spring Boot入门+深入(十二)-检索ElasticSearch

一、Spring Boot与检索

开源的 ElasticSearch 是目前全文搜索引擎的首选。它可以快速的存储、搜索和分析海量数据。

Elasticsearch是一个分布式搜索服务，提供Restful API，底层基于Lucene，采用多shard（分片）的方式保证数据安全，并且提供自动resharding的功能，github 等大型的站点也是采用了ElasticSearch作为其搜索服务。

以 员工文档 的形式存储为例：一个文档代表一个员工数据。存储数据到 ElasticSearch 的行为叫做索引 ，但在索引一个文档之前，需要确定将文档存储在哪里。

一个 ElasticSearch 集群可以包含多个索引 ，相应的每个索引可以包含多个类型 。 这些不同的类型存储着多个文档 ，每个文档又有多个属性 。

ElasticSearch中的概念和数据库有点类似。

        – 索引-----------数据库

        – 类型-----------表

        – 文档-----------表中的记录  文档还可理解为json数据

        – 属性-----------列

Elasticsearch 是面向文档 的，意味着它存储整个对象或文档。Elasticsearch 不仅存储文档，而且 索引 每个文档的内容，使之可以被检索。在 Elasticsearch 中，我们对文档进行索引、检索、排序和过滤—​而不是对行列数据。这是一种完全不同的思考数据的方式，也是 Elasticsearch 能支持复杂全文检索的原因。

Elasticsearch 使用 Javascript Object Notation（或者 JSON）作为文档的序列化格式。JSON 序列化为大多数编程语言所支持，并且已经成为 NoSQL 领域的标准格式。 它简单、简洁、易于阅读。

如下图理解：

 二、ElasticSearch入门 ElasticSearch软件安装

ElasticSearch环境docker安装：参照Spring Boot入门+深入(六)-Docker

ElasticSearch官网资料学习：Elasticsearch: 权威指南 | Elastic

Postman给ElasticSearch发请求测试:

以官网资料为例：

PUT /megacorp/employee/1
{
    "first_name" : "John",
    "last_name" :  "Smith",
    "age" :        25,
    "about" :      "I love to go rock climbing",
    "interests": [ "sports", "music" ]
}
注意，路径 /megacorp/employee/1 包含了三部分的信息：
    megacorp：索引名称
    employee：类型名称
    1：特定雇员的ID

PUT /megacorp/employee/2
{
    "first_name" :  "Jane",
    "last_name" :   "Smith",
    "age" :         32,
    "about" :       "I like to collect rock albums",
    "interests":  [ "music" ]
}

PUT /megacorp/employee/3
{
    "first_name" :  "Douglas",
    "last_name" :   "Fir",
    "age" :         35,
    "about":        "I like to build cabinets",
    "interests":  [ "forestry" ]
}

Postman给ElasticSearch发PUT请求

 点击发送后响应结果

 依次把第二条和第三条数据添加进ElasticSearch中

检索GET请求

GET /megacorp/employee/1

将 HTTP 命令由 PUT 改为 GET 可以用来检索文档，同样的，可以使用 DELETe 命令来删除文档，以及使用 HEAD 指令来检查文档是否存在。如果想更新已存在的文档，只需再次 PUT 。

轻量搜索

GET /megacorp/employee/_search

全查：_search

使用查询表达式查询

GET /megacorp/employee/_search   
{
    "query" : {
        "match" : {
            "last_name" : "Smith"
        }
    }
}
Postman中发送POST请求，json中放入查询条件进行查询

更多的功能参照官方文档学习

三、Spring Boot与ElasticSearch整合

创建Spring Boot工程进行整合ElasticSearch测试

SpringBoot默认支持两种技术来和ES交互

spring-boot-autoconfigure-x.x.x.RELEASE.jar下面有ElasticSearch

1.Jest（默认不生效）

需要导入jest的工具包（io.searchbox.client.JestClient）

@Configuration
@ConditionalOnClass(JestClient.class)
@EnableConfigurationProperties(JestProperties.class)
@AutoConfigureAfter(GsonAutoConfiguration.class)
public class JestAutoConfiguration {

JestAutoConfiguration.java生效条件是有JestClient.java

Spring Boot 与Jest整合

pom.xml

		
		
			io.searchbox
			jest
			5.3.3
		

application.properties

spring.elasticsearch.jest.uris=http://192.168.0.113:9200

Bean

import io.searchbox.annotations.JestId;

public class Article {

    @JestId  //标识主键
    private Integer id;
    private String author;
    private String title;
    private String content;

    public Integer getId() {
        return id;
    }

    public void setId(Integer id) {
        this.id = id;
    }

    public String getAuthor() {
        return author;
    }

    public void setAuthor(String author) {
        this.author = author;
    }

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

    public String getContent() {
        return content;
    }

    public void setContent(String content) {
        this.content = content;
    }
}

测试类：

import io.searchbox.client.JestClient;
import io.searchbox.core.Index;
import io.searchbox.core.Search;
import io.searchbox.core.SearchResult;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import java.io.IOException;

@RunWith(SpringRunner.class)
@SpringBootTest
public class SpringbootElasticApplicationTest {

	@Autowired
	JestClient jestClient;

	@Test
	public void test01() {
		//1、给Es中索引（保存）一个文档；
		Article article = new Article();
		article.setId(1);
		article.setTitle("一个好消息");
		article.setAuthor("张无忌");
		article.setContent("Hello World");

		//构建一个索引功能
		Index index = new Index.Builder(article).index("lwz").type("news").build();

		try {
			jestClient.execute(index);//执行
		} catch (IOException e) {
			e.printStackTrace();
		}
	}
	
	@Test //测试搜索
	public void search(){
		//查询表达式
		String json ="{n" +
				"    "query" : {n" +
				"        "match" : {n" +
				"            "content" : "hello"n" +
				"        }n" +
				"    }n" +
				"}";

		//构建搜索功能
		Search search = new Search.Builder(json).addIndex("lwz").addType("news").build();

		try {
			SearchResult result = jestClient.execute(search);//执行
			System.out.println(result.getJsonString());
		} catch (IOException e) {
			e.printStackTrace();
		}
	}

}

test01()执行后，查询http://192.168.0.113:9200/lwz/news/1

{
    "_index": "lwz",
    "_type": "news",
    "_id": "1",
    "_version": 1,
    "_seq_no": 0,
    "_primary_term": 1,
    "found": true,
    "_source": {
        "id": 1,
        "author": "张无忌",
        "title": "一个好消息",
        "content": "Hello World"
    }
}

search()执行检索，控制台打印log结果

{"took":169,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":1,"relation":"eq"},"max_score":0.2876821,"hits":[{"_index":"lwz","_type":"news","_id":"1","_score":0.2876821,"_source":{"id":1,"author":"张无忌","title":"一个好消息","content":"Hello World"}}]}}

2.SpringData ElasticSearch

ElasticsearchAutoConfiguration.java配置了Client 

1.Client 需要配置节点信息clusterNodes；clusterName等

@Configuration
@ConditionalOnClass({ Client.class, TransportClientFactoryBean.class,
		NodeClientFactoryBean.class })
@EnableConfigurationProperties(ElasticsearchProperties.class)
public class ElasticsearchAutoConfiguration implements DisposableBean {

...
	@Bean
	@ConditionalOnMissingBean
	public Client elasticsearchClient() {
		try {
			return createClient();
		}
		catch (Exception ex) {
			throw new IllegalStateException(ex);
		}
	}
...

2.ElasticsearchDataAutoConfiguration.java为我们配置了ElasticsearchTemplate来操作Elasticsearch

@Configuration
@ConditionalOnClass({ Client.class, ElasticsearchTemplate.class })
@AutoConfigureAfter(ElasticsearchAutoConfiguration.class)
public class ElasticsearchDataAutoConfiguration {

	@Bean
	@ConditionalOnMissingBean
	@ConditionalOnBean(Client.class)
	public ElasticsearchTemplate elasticsearchTemplate(Client client,
			ElasticsearchConverter converter) {
		try {
			return new ElasticsearchTemplate(client, converter);
		}
		catch (Exception ex) {
			throw new IllegalStateException(ex);
		}
	}
....

3.ElasticsearchRepository 类似JPA  自己写接口继承ElasticsearchRepository，来操作Elasticsearch

@Configuration
@ConditionalOnClass({ Client.class, ElasticsearchRepository.class })
@ConditionalOnProperty(prefix = "spring.data.elasticsearch.repositories", name = "enabled", havingValue = "true", matchIfMissing = true)
@ConditionalOnMissingBean(ElasticsearchRepositoryFactoryBean.class)
@import(ElasticsearchRepositoriesRegistrar.class)
public class ElasticsearchRepositoriesAutoConfiguration {

}

Spring Boot 与SpringData ElasticSearch整合

pom.xml

	org.springframework.boot
	spring-boot-starter-data-elasticsearch
 

application.properties

spring.data.elasticsearch.cluster-name=docker-cluster
spring.data.elasticsearch.cluster-nodes=192.168.0.113:9300

启动Spring Boot项目，验证连接ElasticSearch是否有问题

控制台报错信息：org.elasticsearch.transport.NodeDisconnectedException: [][192.168.0.113:9300][cluster:monitor/nodes/liveness] disconnected

Spring连接Elasticsearch报错:NodeDisconnectedException[[][IP:9300]

什么原因呢？

查看你的Elasticsearch安装目录的logs下的日志.

#查看日志
docker logs 容器ID

日志信息

{"type": "server", "timestamp": "2021-11-28T01:20:25,155Z", "level": "WARN", "component": "o.e.t.TcpTransport", "cluster.name": "docker-cluster", "node.name": "23da5ddf57e3", "message": "exception caught on transport layer [Netty4TcpChannel{localAddress=/172.17.0.3:9300, remoteAddress=/192.168.0.171:52120}], closing connection", "cluster.uuid": "kidKPuQZRLKe2QlYuuOLoQ", "node.id": "v9gf8_LbQjWwCEgEV48buQ" , 
"stacktrace": ["java.lang.IllegalStateException: Received message from unsupported version: [2.0.0] minimal compatible version is: [6.8.0]",
"at org.elasticsearch.transport.InboundDecoder.ensureVersionCompatibility(InboundDecoder.java:210) ~[elasticsearch-7.9.2.jar:7.9.2]",
"at org.elasticsearch.transport.InboundDecoder.readHeader(InboundDecoder.java:177) ~[elasticsearch-7.9.2.jar:7.9.2]",
"at org.elasticsearch.transport.InboundDecoder.internalDecode(InboundDecoder.java:75) ~[elasticsearch-7.9.2.jar:7.9.2]",
"at org.elasticsearch.transport.InboundDecoder.decode(InboundDecoder.java:53) ~[elasticsearch-7.9.2.jar:7.9.2]",
"at org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:101) ~[elasticsearch-7.9.2.jar:7.9.2]",
"at org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:82) ~[elasticsearch-7.9.2.jar:7.9.2]",
"at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:76) ~[transport-netty4-client-7.9.2.jar:7.9.2]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.handler.logging.LoggingHandler.channelRead(LoggingHandler.java:271) [netty-handler-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:615) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:578) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-common-4.1.49.Final.jar:4.1.49.Final]",
"at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.49.Final.jar:4.1.49.Final]",
"at java.lang.Thread.run(Thread.java:832) [?:?]"] }

其中：Received message from unsupported version: [2.0.0] minimal compatible version is: [6.8.0]",
"at org.elasticsearch.transport.InboundDecoder.ensureVersionCompatibility(InboundDecoder.java:210) ~[elasticsearch-7.9.2.jar:7.9.2]"

从不受支持的版本[2.0.0]接收到的消息最小兼容版本为：[6.8.0]，现在我们使用的是elasticsearch-7.9.2。

结论：低版本ja包的不支持高版本的Elasticsearch

【ES版本有可能不合适】涉及到版本匹配的问题？

在Spring官网，找到Spring Data模块，找到Spring Data ElasticSearch，点击Reference Doc，

找Project metadata Version Control - https://github.com/spring-projects/spring-data-elasticsearch

点击连接往下翻，找到The compatibility between Spring Data Elasticsearch, Elasticsearch client drivers and Spring Boot versions can be found in the reference documentation.

点击reference documentation，连接到 版本匹配关系表。

 解决方式：调整自己项目版本。

 如果版本不适配：
        1）、升级SpringBoot版本
        2）、安装对应版本的ES

调整完版本匹配，继续进行整合测试。

启动SpringBoot，后台打印日志adding transport node : 192.168.0.113:9300

说明连接上ElasticSearch。

控制台显示错误：

[Shroud] node {#transport#-1}{192.168.0.113}{192.168.0.113:9300} not part of the cluster Cluster [docker-cluster], ignoring...

说明：application.properties中的spring.data.elasticsearch.cluster-name=docker-cluster

cluster-name配置错误。

Bean

import org.springframework.data.elasticsearch.annotations.document;

@document(indexName = "lwz",type = "book")
public class Book {
    private Integer id;
    private String bookName;
    private String author;

    public Integer getId() {
        return id;
    }

    public void setId(Integer id) {
        this.id = id;
    }

    public String getBookName() {
        return bookName;
    }

    public void setBookName(String bookName) {
        this.bookName = bookName;
    }

    public String getAuthor() {
        return author;
    }

    public void setAuthor(String author) {
        this.author = author;
    }

    @Override
    public String toString() {
        return "Book{" +
                "id=" + id +
                ", bookName='" + bookName + ''' +
                ", author='" + author + ''' +
                '}';
    }
}

Repository

import org.springframework.data.elasticsearch.repository.ElasticsearchRepository;

import java.util.List;

public interface BookRepository extends ElasticsearchRepository {
   public List findByBookNameLike(String bookName);//自定义查询方法
}

测试类：

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
public class SpringbootElasticApplicationTest1 {

    @Autowired
    BookRepository bookRepository;

    @Test
    public void test01(){
		Book book = new Book();
		book.setId(1);
		book.setBookName("西游记");
		book.setAuthor("吴承恩");
		bookRepository.index(book);
    }

    @Test
    public void test02(){
        for (Book book : bookRepository.findByBookNameLike("游")) {
            System.out.println(book);
        };
    }
}

test01()插入ES数据---->类似于JPA插入数据库数据

test02()查询ES数据---->类似于JPA查询数据库数据

test02()查询结果：Book{id=1, bookName='西游记', author='吴承恩'}

Spring Boot入门+深入(一)：Spring Boot入门+深入(一)

Spring Boot入门+深入(十三)：Spring Boot入门+深入(十三)-任务