小伙伴们，你们好呀，我是老寇，跟我一起学习es 7.6.2
注：请点击我，获取源码
注：借鉴黑马程序员学习es
链接：https://pan.baidu.com/s/1swrV9ffJnmz4S0mfkuBbIw
提取码：1111
视频地址
老寇-es通过关键字高亮显示
目录
一、搜索入门
二、DSL入门
三、Query DSL语法
四、filter
五、定制排序规则
六、代码实现
一、搜索入门
一、搜索入门
二、DSL入门
三、Query DSL语法
四、filter
五、定制排序规则
六、代码实现
1.无条件搜索命令
GET /index/_search
GET /message/_search
2.传参搜索命令
GET /index/_search?q=filed:value
GET /message/_search?q=id:1424966164936024065
问题扩展： + 和 - 区别（见如下举例说明）
GET /message/_search?q=+id:1424966164936024065 #查询id=1424966164936024065的数据 GET /message/_search?q=-id:1424966164936024065 #查询id!=1424966164936024065的数据
3.分页搜索命令
GET /index/_search?size=x&from=x
GET /message/_search?size=10&from=0 注：类似sql > select * from message 0,10
问题扩展：分页过深，对性能有哪些影响？
1.消耗网络带宽，搜的过深，各分片(shard)要把数据传递给协调节点(coordinating node),这个过程中有大量数据传输，消耗网络 2.消耗内存,各节点(shard)要把数据传给协调节点(coordinating node),这个传递回来的数据,被协调节点(coordinating node)保存在内存中,这样会大量消耗内存 3.消耗cpu,协调节点(coordinating node)要把传回来的数据进行排序,这个排序过程很消耗cpu 因此，出于对深度分页(deep paging)的性能考虑，能少用就尽量少用
二、DSL入门
es所独有的搜索语言（ps：有点类似sql语言），可以在请求体携带搜索条件，功能强大
1.查询全部
举个栗子
GET /message/_search
{
"query": {
"match_all": {}
}
}
注：看到这里，小伙伴们就要问了，es的get请求为什么可以携带请求体？这是因为es对此做出处理，至于是怎么处理的，感兴趣的小伙伴，可以去查阅资料
2.数据排序
举个栗子
GET /message/_search
{
"query": {
"match": {
"desc": "群聊"
}
},
"sort": [
{
"createDate": {
"order": "desc"
}
}
]
}
3.分页查询
举个栗子
GET /message/_search
{
"query": {
"match_all": {}
},
"from": 0,
"size": 10
}
4.返回指定字段
举个栗子
GET /message/_search
{
"query": {
"match_all": {}
},
"_source": ["username","data"]
}
三、Query DSL语法
1.DSL 命令
{
query_name: {
argument:value
......
}
}
或者
{
query_name:{
field_name:{
argument:value
......
}
}
}
举个栗子
GET /message/_search
{
"query": {
"match": {
"desc": "群聊"
}
}
}
2.多条件组合搜索
举个栗子
GET /message/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"username": "admin"
}
}
],
"should": [
{
"match": {
"desc": "群聊"
}
}
],
"must_not": [
{
"match": {
"desc": "私聊"
}
}
]
}
}
}
GET /message/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"sendId": "1363109342432645122"
}
}
],
"should": [
{
"match": {
"username": "admin"
}
},
{
"bool": {
"must": [
{
"match": {
"data": "无名，天地之始，有名，万物之母。"
}
}
]
}
}
]
}
}
}
3.dsl语法
match_all - 举个栗子
GET /message/_search
{
"query": {
"match_all": {}
}
}
match - 举个栗子
GET /message/_search
{
"query": {
"match": {
"username": "admin"
}
}
}
multi_match - 举个栗子
GET /message/_search
{
"query": {
"multi_match": {
"query": "生日快乐",
"fields": ["data","data.pinyin"]
}
}
}
range query - 举个栗子
GET /message/_search
{
"query": {
"range": {
"id": {
"gte": 1359036315055083522,
"lte": 1359036315055083522
}
}
}
}
term query - 举个栗子
GET /message/_search
{
"query": {
"term": {
"username": {
"value": "admin"
}
}
}
}
terms query - 举个栗子
GET /message/_search
{
"query": {
"terms": {
"data": [
"年年",
"岁岁"
]
}
}
}
exists query（查询有默写字段值的文档） - 举个栗子
GET /message/_search
{
"query": {
"exists": {
"field": "remark"
}
}
}
full query（返回包含与搜索词类似词的文档） - 举个栗子
生日1 > 生日
GET /message/_search
{
"query": {
"fuzzy": {
"data": {
"value": "生日1"
}
}
}
}
ids query - 举个栗子
GET /message/_search
{
"query": {
"ids": {
"values": ["1426744462376591362","1426752233562071042"]
}
}
}
prefix（前缀查询） - 举个栗子
GET /message/_search
{
"query": {
"prefix": {
"data": {
"value": "生日快乐"
}
}
}
}
regexp query（正则查询） - 举个栗子
GET /message/_search
{
"query": {
"regexp": {
"data": "生日*"
}
}
}
分词搜索 - 举个栗子
GET /message/_search
{
"query": {
"bool": {
"should": [
{
"query_string": {
"default_field": "data",
"query": "shengri"
}
},
{
"query_string": {
"default_field": "data.pinyin",
"query": "shengri"
}
}
]
}
}
}
四、filter
查询在老寇云交流群发了关于生日快乐的内容（两种方式）
GET /message/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"data": "生日快乐"
}
},
{
"match": {
"remark": "老寇云交流群"
}
}
]
}
}
}
GET /message/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"data": "生日快乐"
}
}
],
"filter": [
{
"term": {
"remark": "老寇云交流群"
}
}
]
}
}
}
query 与 filter 对比
filter：只会按照搜索条件过滤出所需要的数据，不会计算相关度分数，因此对相关度没有影响，同时内置自动缓存最常使用filter的数据
query：会计算每个文档对搜索条件的相关度，并按相关度排序，然而需要计算相关度，所以无法缓存结果
应用场景：在进行搜索时，需要根据一些条件筛选部分数据，而且不关注其排序，建议使用filter，反之，使用query
五、定制排序规则
constant_score - 举个栗子
GET /message/_search
{
"query": {
"constant_score": {
"filter": {
"term": {
"data": "生日快乐"
}
},
"boost": 1.2
}
},
"sort": [
{
"createDate": {
"order": "desc"
}
}
]
}
六、代码实现
1.引入依赖
org.springframework.boot spring-boot-starter-webio.springfox springfox-swagger2io.springfox springfox-swagger-uiorg.elasticsearch.client elasticsearch-rest-high-level-client7.6.2 org.elasticsearch elasticsearchorg.elasticsearch.client elasticsearch-rest-clientorg.elasticsearch.client elasticsearch-rest-client7.6.2 org.elasticsearch elasticsearch7.6.2
2.增加实体类
@Data
@ApiModel(description = "查询表单实体类")
public class QueryForm implements Serializable {
private Integer pageNum = 1;
private Integer pageSize = 10;
private boolean needPage = false;
private String[] indexNames;
private List queryStringList;
private List sortFieldList;
private List highlightFieldList;
private List orSearchList;
}
@Data
public class SearchDTO implements Serializable {
private String field;
private String value;
}
3.分词查询
@Slf4j
@Component
public class ElasticsearchUtil {
@Autowired
private RestHighLevelClient restHighLevelClient;
private static final String HIGHLIGHT_PRE_TAGS = "";
private static final String HIGHLIGHT_POST_TAGS = "";
private static final String PINYIN_SUFFIX = ".pinyin";
public HttpResultUtil search(QueryForm queryForm) throws IOException {
long startTime = System.currentTimeMillis();
final String[] indexName = queryForm.getIndexNames();
final List orSearchList = queryForm.getOrSearchList();
final List sortFieldList = queryForm.getSortFieldList();
final List highlightFieldList = queryForm.getHighlightFieldList();
final List queryStringList = queryForm.getQueryStringList();
final Integer pageNum = queryForm.getPageNum();
final Integer pageSize = queryForm.getPageSize();
BoolQueryBuilder boolQueryBuilder = QueryBuilders.boolQuery();
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
//用于搜索文档，聚合，定制查询有关操作
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices(indexName);
//or查询
BoolQueryBuilder orQuery = QueryBuilders.boolQuery();
for (SearchDTO dto : orSearchList) {
orQuery.should(QueryBuilders.termQuery(dto.getField(),dto.getValue()));
}
boolQueryBuilder.must(orQuery);
//分词查询
BoolQueryBuilder analysisQuery = QueryBuilders.boolQuery();
for (SearchDTO dto : queryStringList) {
final String field = dto.getField();
//清除左右空格
String keyword = dto.getValue().trim();
//处理特殊字符
keyword = QueryParser.escape(keyword);
analysisQuery.should(QueryBuilders.queryStringQuery(keyword).field(field).field(field.concat(PINYIN_SUFFIX)));
}
boolQueryBuilder.must(analysisQuery);
//高亮显示数据
HighlightBuilder highlightBuilder = new HighlightBuilder();
//设置关键字显示颜色
highlightBuilder.preTags(HIGHLIGHT_PRE_TAGS);
highlightBuilder.postTags(HIGHLIGHT_POST_TAGS);
//设置显示的关键字
for (String field : highlightFieldList) {
highlightBuilder.field(field, 0, 0).field(field.concat(PINYIN_SUFFIX), 0, 0);
}
highlightBuilder.requireFieldMatch(false);
//分页
int start = 0;
int end = 10000;
if (queryForm.isNeedPage()) {
start = (pageNum - 1) * pageSize;
end = pageSize;
}
//设置高亮
searchSourceBuilder.highlighter(highlightBuilder);
searchSourceBuilder.from(start);
searchSourceBuilder.size(end);
//追踪分数开启
searchSourceBuilder.trackScores(true);
//注解
searchSourceBuilder.explain(true);
//排序
for (SearchDTO dto : sortFieldList) {
SortOrder sortOrder;
final String desc = "desc";
final String value = dto.getValue();
final String field = dto.getField();
if (desc.equalsIgnoreCase(value)) {
sortOrder = SortOrder.DESC;
} else {
sortOrder = SortOrder.ASC;
}
searchSourceBuilder.sort(field,sortOrder);
}
searchSourceBuilder.query(boolQueryBuilder);
searchRequest.source(searchSourceBuilder);
SearchHits hits = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT).getHits();
List> data = new ArrayList<>();
for (SearchHit hit : hits){
Map sourceData = hit.getSourceAsMap();
Map highlightFields = hit.getHighlightFields();
for (String key : highlightFields.keySet()){
sourceData.put(key,highlightFields.get(key).getFragments()[0].string());
}
data.add(sourceData);
}
long endTime = System.currentTimeMillis();
HttpResultUtil result = new HttpResultUtil();
SearchVO vo = new SearchVO();
final String searchData = queryStringList.stream().map(i -> i.getValue()).collect(Collectors.joining(","));
final List searchFieldList = queryStringList.stream().map(i -> i.getField()).collect(Collectors.toList());
vo.setRecords(handlerData(data,searchFieldList));
vo.setTotal(hits.getTotalHits().value);
vo.setPageNum(queryForm.getPageSize());
vo.setPageSize(queryForm.getPageSize());
result.setMsg("搜索 " + searchData + " 找到 " + vo.getTotal() + " 个与之相关的内容，耗时：" + (endTime - startTime) +"ms");
//处理数据
result.setData(vo);
return result;
}
private List> handlerData(List> data,List fieldList) {
log.info("查询结果：{}",data);
if (CollectionUtils.isEmpty(data)) {
return Lists.newArrayList();
}
if (CollectionUtils.isEmpty(fieldList)) {
return data;
}
for (Map map : data) {
for (String field : fieldList) {
if (map.containsKey(field.concat(PINYIN_SUFFIX))) {
String result1 = map.get(field).toString();
String result2 = map.get(field.concat(PINYIN_SUFFIX)).toString();
//将同义词合并
for (;;) {
int start = result1.indexOf(HIGHLIGHT_PRE_TAGS);
int end = result1.indexOf(HIGHLIGHT_POST_TAGS);
if (start == -1 || end == -1) {
break;
}
String replaceKeyword = result1.substring(start, end).replace(HIGHLIGHT_PRE_TAGS, "");
result2 = result2.replaceAll(replaceKeyword, HIGHLIGHT_PRE_TAGS + replaceKeyword + HIGHLIGHT_POST_TAGS);
result1 = result1.substring(end + 1);
}
map.put(field, result2);
map.remove(field.concat(PINYIN_SUFFIX));
}
}
}
return data;
}
}
4.api测试
@RestController
@RequestMapping("/api")
@Api(tags = "Elasticsearch API 服务")
public class ElasticsearchController {
@Autowired
private ElasticsearchUtil elasticsearchUtil;
@PostMapping("/search")
@ApiOperation("ES关键字搜索-高亮显示")
@ResponseBody
public HttpResultUtil search(@RequestBody @Validated final QueryForm queryForm, BindingResult bindingResult) throws IOException {
if (bindingResult.hasErrors()) {
return new HttpResultUtil().error(bindingResult.getFieldError().getDefaultMessage());
}
return elasticsearchUtil.search(queryForm);
}
}
大功告成
