在做数据导入时,服务器报以下异常:
Caused by: org.apache.dubbo.remoting.transport.ExceedPayloadLimitException: Data length too large: 10794838, max payload: 8388608, channel: NettyChannel [channel=[id: 0x5aa64030, L:/172.31.8.190:51674 - R:/172.31.9.63:9090]]
at org.apache.dubbo.remoting.transport.AbstractCodec.checkPayload(AbstractCodec.java:50)
at org.apache.dubbo.remoting.exchange.codec.ExchangeCodec.encodeRequest(ExchangeCodec.java:247)
at org.apache.dubbo.remoting.exchange.codec.ExchangeCodec.encode(ExchangeCodec.java:69)
at org.apache.dubbo.rpc.protocol.dubbo.DubboCountCodec.encode(DubboCountCodec.java:40)
at org.apache.dubbo.remoting.transport.netty4.NettyCodecAdapter$InternalEncoder.encode(NettyCodecAdapter.java:69)
at io.netty.handler.codec.MessageToByteEncoder.write(MessageToByteEncoder.java:107)
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709)
at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:792)
at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:702)
at io.netty.handler.timeout.IdleStateHandler.write(IdleStateHandler.java:304)
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:709)
at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:792)
at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:702)
at io.netty.channel.ChannelDuplexHandler.write(ChannelDuplexHandler.java:115)
at org.apache.dubbo.remoting.transport.netty4.NettyClientHandler.write(NettyClientHandler.java:88)
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:764)
at io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1071)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
原因
dubbo对单次传输数据大小做了限制,默认8M
为什么dubbo要对传输数据大小做限制在dubbo2的版本,dubbo协议基于HTTP1.1实现。HTTP1.1是串行传输数据的,所以会有HOL问题(Head-of-line blocking),后面的请求会被前面的请求阻塞,导致并发能力下降。这也是为什么dubbo适合小数据传输的。
但是在dubbo3的版本,协议换成 了Triple,Triple协议基于HTTP2.0实现。HTTP2.0是并行传输数据的,并且使用NIO模型,且是全双工的,支持服务端主动push的方式(一次请求允许多次响应)。因此一定程度上解决了dubbo2大文件传输的HOL问题,但是仍然不建议用dubbo做大文件传输。
解决方案- 设置payload默认大小,不建议,会影响吞吐
- 分批导入
- 如果无法分批,建议使用其它传输方式。



