spark2.4.6与scala2.11.12运行程序过程中,遇到了netty版本冲突或包缺失导致程序出现NoSuchMethodError异常,报错如下:
Exception in thread “main” java.lang.NoSuchMethodError: io.netty.buffer.PooledByteBufAllocator.defaultNumHeapArena()I20/07/13 12:50:55 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(lenovo); groups with view permissions: Set(); users with modify permissions: Set(lenovo); groups with modify permissions: Set() Exception in thread "main" java.lang.NoSuchMethodError: io.netty.buffer.PooledByteBufAllocator.defaultNumHeapArena()I解决方法:
1.在系统命令行中进入到项目根目录,也就是pom文件所在的目录,运行命令mvn dependency:tree查看jar包依赖(用ctrl+F快速定位到io.netty),发现初步定位就是这两个jar包导致的冲突
D:SoftwarejavaWorkSpacehrbutest>mvn dependency:tree
2.使用 标签剔除 spark-core 中所有的低版本的netty,在项目中的pom文件中的指定标签中添加依赖
org.apache.hadoop hadoop-hdfs${hadoop.version} io.netty netty
3.然后添加高版本netty依赖,最后运行程序,就不会报错了
io.netty netty-all4.1.18.Final



