栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 云计算 > 云平台

【路径】1.query阶段:分行分片前的请求InitialSearchPhase

云平台 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

【路径】1.query阶段:分行分片前的请求InitialSearchPhase

一:分行分片前的请求

1.Netty4HttpServerTransport
dispatchRequest:491, Netty4HttpServerTransport (org.elasticsearch.http.netty4)
2.RestController
dispatchRequest:174, RestController (org.elasticsearch.rest)
tryAllHandlers:337, RestController (org.elasticsearch.rest)
dispatchRequest:240, RestController (org.elasticsearch.rest)
3.BaseRestHandler
handleRequest:97, BaseRestHandler (org.elasticsearch.rest)
4.RestSearchAction
accept:-1, 1593864517 (org.elasticsearch.rest.action.search.RestSearchAction$$Lambda 1697 ) l a m b d a 1697) lambda 1697)lambdaprepareRequest$2:93, RestSearchAction (org.elasticsearch.rest.action.search)
在RestSearchAction#prepareRequest方法中将请求体解析为SearchRequest 数据结构:
解析请求

public RestChannelConsumer prepareRequest(.. .) {
    SearchRequest searchRequest = new SearchRequest();
    request.withContentOrSourceParamParserOrNull (parser ->
        parseSearchRequest (searchRequest, request, parser, setSize));
}

5.AbstractClient
search:463, AbstractClient (org.elasticsearch.client.support)
execute:336, AbstractClient (org.elasticsearch.client.support)
6.NodeClient
doExecute:72, NodeClient (org.elasticsearch.client.node)
executeLocally:83, NodeClient (org.elasticsearch.client.node)
7.TransportAction
execute:80, TransportAction (org.elasticsearch.action.support)
execute:141, TransportAction (org.elasticsearch.action.support)
proceed:178, TransportAction$RequestFilterChain (org.elasticsearch.action.support)

8.TransportSearchAction
doExecute:62, TransportSearchAction (org.elasticsearch.action.search)
doExecute:207, TransportSearchAction (org.elasticsearch.action.search)

9.Rewriteable
rewriteAndFetch:86, Rewriteable (org.elasticsearch.index.query)
rewriteAndFetch:113, Rewriteable (org.elasticsearch.index.query)

10.ActionListener
onResponse:59, ActionListener$1 (org.elasticsearch.action)

10.返回8.TransportSearchAction
accept:-1, 320835983 (org.elasticsearch.action.search.TransportSearchAction$$Lambda 1699 ) l a m b d a 1699) lambda 1699)lambdadoExecute$4:187, TransportSearchAction (org.elasticsearch.action.search)
executeSearch:329, TransportSearchAction (org.elasticsearch.action.search)
构造目的shard列表
将请求涉及的本集群shard列表和远程集群的shard列表(远程集群用于跨集群访问)合并:

private void executeSearch(.. .) {
    GroupShardsI terator localShardsIterator =
        clusterService.operationRouting().searchShards (clusterState,
        concreteIndices, routingMap, searchRequest.preference (),
        searchService.getResponseCollectorService(), nodeSearchCounts);
    GroupShardsIterator  shardIterators =
        mergeShards Iterators (localShardsIterator, localIndices, remoteShardIterators);
}

二:query阶段:执行分片子查询TransportService#sendRequestInternal

1.AbstractSearchAsyncAction
start:115, AbstractSearchAsyncAction (org.elasticsearch.action.search)
executePhase:145, AbstractSearchAsyncAction (org.elasticsearch.action.search)

2.InitialSearchPhase
run:158, InitialSearchPhase (org.elasticsearch.action.search)
performPhaseOnShard:216, InitialSearchPhase (org.elasticsearch.action.search)
遍历所有shard发送请求
请求是基于shard遍历的,如果列表中有N个shard位于同一个节点,则向其发送N次请求,并不会把请求合并为一个。

public final void run() throws IOException {
    if (shardsIts.size() > 0) {
        //最大发分片请求数可以通过max_ concurrent_ shard_ requests参数配置(v6.0新增)
        int maxConcurrentShardRequests = Math.min(this.maxConcurrentShardRequests, shardsIts.size());
        for(int index = 0; index < maxConcurrentShardRequests; index++) {
            final SearchShardIterator shardRoutings = shardsIts.get (index);
            //执行shard级请求
            performPhaseOnShard (index, shardRoutings, shardRoutings.nextOrNull());
        }
    }
}

3.SearchQueryThenFetchAsyncAction 转发分片
executePhaseOnShard:52, SearchQueryThenFetchAsyncAction (org.elasticsearch.action.search)
shardsIts为本次搜索涉及的所有分片,shardRoutings.nextOrNull()从某 个分片的所有副本中选择一个,例如,从website 中选择主分片。

转发请求同时定义一个Listener,用于处理Response:

private void performPhaseOnShard(. ..) {
    executePhaseOnShard(.. .) {
        //收到执行成功的回复
        public void innerOnResponse (FirstResult result) {
            maybeFork (thread, () -> onShardResult (result,shardIt) );
        }
        //收到执行失败的回复
        public void onFailure (Exception t) {
            maybeFork(thread, () -> onShardFailure (shardIndex, shard, shard. currentNodeId(),shardIt, t));
        }
    });
}

收集返回结果
本过程在search线程池中执行:

private void onShardResult (FirstResult result, SearchShardIterator shardIt) {
    onShardSuccess(result);
    success fulShardExecution(shardIt);
}

onShardSuccess对收集到的结果进行合并。successfulShardExecution方法检查是否所有请求都已收到回复,是否进入

private void successfulShardExecution (SearchShardIterator shardsIt) {
    //计数器累加.
    final int xTotalOps = totalOps.addAndGet (remainingOpsOnIterator);
    //检查是否收到全部回复
    if (xTotalOps == expectedTotalOps) {
        onPhaseDone ();
    } else if (xTota1Ops > expectedTotal0ps) {
        throw new AssertionError(. ..);
    }
}

4.SearchTransportService
sendExecuteQuery:151, SearchTransportService (org.elasticsearch.action.search)
5.TransportService
sendChildRequest:562, TransportService (org.elasticsearch.transport)
sendChildRequest:571, TransportService (org.elasticsearch.transport)
sendRequest:530, TransportService (org.elasticsearch.transport)
sendRequest:-1, 2065587797 (org.elasticsearch.transport.TransportService$$Lambda$1106)
sendRequestInternal:611, TransportService (org.elasticsearch.transport)
6.发送到本地
sendRequest:138, TransportService$3 (org.elasticsearch.transport)
access$000:68, TransportService (org.elasticsearch.transport)
发到节点上的分片
sendLocalRequest:673, TransportService (org.elasticsearch.transport)
7.RequestHandlerRegistry
processMessageReceived:66, RequestHandlerRegistry (org.elasticsearch.transport)
响应Query请求
以常见的Query请求为例,其action为:indices:data/read/search [phase/query]
主要过程就是执行查询,然后发送Response

transportService.registerRequestHandler (QUERY_ACTION_NAME, ShardSearchTransportRequest::new, ThreadPool.Names.SAME,
    new TaskAwareTransportRequestHandler () {
        //收到Query请求
        public void messageReceived(. ..) throws Exception {
            //执行查询
            searchService.executeQueryPhase (request, (SearchTask) task, new ActionListener() {
            //处理查询成功的情况
                public void onResponse (SearchPhaseResult searchPhaseResult) {
                    channel . sendResponse (searchPhaseResult) ;
                }
                //处理查询失败的情况
                public void onFailure (Exception е) {
                    channel.sendResponse(e);
                }
            });
        }
    });

8.SearchTransportService
messageReceived:358, SearchTransportService$6 (org.elasticsearch.action.search)
taskManager.#register注册任务
messageReceived:364, SearchTransportService$6 (org.elasticsearch.action.search)
**9.SearchService **
executeQueryPhase:271, SearchService (org.elasticsearch.search)
rewriteShardRequest:990, SearchService (org.elasticsearch.search)
rewriteAndFetch:86, Rewriteable (org.elasticsearch.index.query)
10.Rewriteable
rewriteAndFetch:113, Rewriteable (org.elasticsearch.index.query)
rewriteResponse.onResponse(builder)

11 返回rewriteShardRequest的listener.onResponse
executeQueryPhase(request, task)

 public void executeQueryPhase(ShardSearchRequest request, SearchTask task, ActionListener listener) {
        rewriteShardRequest(request, new ActionListener() {
            @Override
            public void onResponse(ShardSearchRequest request) {
                try {
                    if (!Objects.equals(request.shardId().getIndex().getName(), ".kibana")) {
                        System.out.println("监听search之后的结果SearchService#SearchService");
                    }
                    listener.onResponse(executeQueryPhase(request, task));
                } catch (Exception e) {
                    onFailure(e);
                }
            }

            @Override
            public void onFailure(Exception e) {
                listener.onFailure(e);
            }
        });
    }

onResponse:271, SearchService$2 (org.elasticsearch.search)
onResponse:278, SearchService$2 (org.elasticsearch.search)
executeQueryPhase:308, SearchService (org.elasticsearch.search)
loadOrExecuteQueryPhase:266, SearchService (org.elasticsearch.search)
execute:101, QueryPhase (org.elasticsearch.search.query)
execute:129, QueryPhase (org.elasticsearch.search.query)

执行真正的lunence查询

Query query = searchContext.query();

static boolean execute(SearchContext searchContext, final IndexSearcher searcher,
                           Consumer checkCancellationSetter, @Nullable Sort indexSort) throws QueryPhaseExecutionException {
        QuerySearchResult queryResult = searchContext.queryResult();
        queryResult.searchTimedOut(false);
        try {
            queryResult.from(searchContext.from());// 设置分页
            queryResult.size(searchContext.size());
            Query query = searchContext.query();
            assert query == searcher.rewrite(query); // already rewritten
            }

          if (shouldCollect) {
                 searcher.search(query, queryCollector);
             }
     }
转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/896046.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号