继上一篇 SpringBoot-Admin 微服务监控+健康检查+钉钉告警,附代码配置
项目部署上线后,会偶尔出现下面的异常,导致告警消息并非是真正服务实例出现异常,
{
"statusInfo": {
"status": "OFFLINE",
"details": {
"exception": "org.springframework.web.reactive.function.client.WebClientRequestException",
"message": "Connection prematurely closed BEFORE response; nested exception is reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response"
}
}
}
经查看服务器日志,同样出现此类异常信息
Caused by: reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response
2021-12-01 08:06:18.715 [reactor-http-epoll-1] WARN r.n.http.client.HttpClientConnect -[id:9eb3d572-22, L:/10.2.1.2:47286 ! R:10.2.1.1/10.2.1.1:8030] The connection observed an error
reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response
2021-12-01 08:06:18.716 [reactor-http-epoll-1] INFO d.c.b.a.s.services.StatusUpdater -Couldn't retrieve status for Instance(id=0b337b0012c5, version=19, registration=Registration(name=ORDER, managementUrl=http://10.2.1.1:8030/actuator, healthUrl=http://10.2.1.1:8030/actuator/health, serviceUrl=http://10.2.1.1:8030, source=discovery), registered=true, statusInfo=StatusInfo(status=UP, details={}), statusTimestamp=2021-11-30T17:05:18.806Z, info=Info(values={}), endpoints=Endpoints(endpoints={health=Endpoint(id=health, url=http://10.2.1.1:8030/actuator/health)}), buildVersion=null, tags=Tags(values={}))
org.springframework.web.reactive.function.client.WebClientRequestException: Connection prematurely closed BEFORE response; nested exception is reactor.netty.http.client.PrematureCloseException: Connection prematurely closed BEFORE response
然后分析大概是reactor.netty 配置问题导致。
先查看当前项目使用的组件版本:
Spring Boot version: 2.4.2 Reactor netty version: 1.0.3 spring-boot-admin version: 2.4.3
git hub上也已经关注了这个问题:
https://github.com/codecentric/spring-boot-admin/issues/1711
目前尝试的
解决方案1:
将reactor netty版本修改为0.9.10.RELEASE
de.codecentric spring-boot-admin-starter-server io.projectreactor.netty reactor-netty io.projectreactor.netty reactor-netty 0.9.10.RELEASE
注意:方案1经过测试,仍然会出现上面异常,故方案1不可解决。
解决方案2:
添加JVM启动参数,更改从连接池中取连接的策略,由FIFO变更为LIFO(reactor.netty.resources.ConnectionProvider),确保拿到的连接永远是最新的连接;
-Dreactor.netty.pool.leasingStrategy=lifo
注意:方案2经过测试,仍然会出现上面异常,故方案2不可解决。
目前采用的变通方案是将健康检查周期调整为2分钟=120秒一次。这样能尽可能的减少误报告警消息。
配置如下:
spring: boot: admin: monitor: status-interval: 60000 status-lifetime: 60000
现在在跟进github上原作者的相关动作。
https://github.com/codecentric/spring-boot-admin/issues/1711
https://github.com/reactor/reactor-netty/pull/1905
参考:
https://codecentric.github.io/spring-boot-admin
https://zhuanlan.zhihu.com/p/272902531
https://blog.csdn.net/yaomingyang/article/details/114330502



