栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

Kafka in Windows error:java.nio.file.FileSystemException:另一个程序正在使用此文件,进程无法访问

Kafka in Windows error:java.nio.file.FileSystemException:另一个程序正在使用此文件,进程无法访问

问题描述

在windows环境下部署的Kafka,在运行一段时间后挂掉。
查看日志logs/server.log 发现如下报错:

 ERROR Failed to clean up log for __consumer_offsets-44 in dir D:kafkakafka_2.13-2.8.0kafkakafka_2.13-2.8.0kafka-logs due to IOException (kafka.server.LogDirFailureChannel)
java.nio.file.FileSystemException: D:kafkakafka_2.13-2.8.0kafkakafka_2.13-2.8.0kafka-logs__consumer_offsets-440000000000000000000.timeindex.cleaned -> D:kafkakafka_2.13-2.8.0kafkakafka_2.13-2.8.0kafka-logs__consumer_offsets-440000000000000000000.timeindex.swap: 另一个程序正在使用此文件,进程无法访问。

	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
	at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:387)
	at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
	at java.nio.file.Files.move(Files.java:1395)
	at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:904)
	at kafka.log.AbstractIndex.renameTo(AbstractIndex.scala:210)
	at kafka.log.LazyIndex$IndexValue.renameTo(LazyIndex.scala:155)
	at kafka.log.LazyIndex.$anonfun$renameTo$1(LazyIndex.scala:79)
	at kafka.log.LazyIndex.renameTo(LazyIndex.scala:79)
	at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:496)
	at kafka.log.Log.$anonfun$replaceSegments$4(Log.scala:2402)
	at kafka.log.Log.$anonfun$replaceSegments$4$adapted(Log.scala:2402)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at kafka.log.Log.replaceSegments(Log.scala:2402)
	at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:613)
	at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:538)
	at kafka.log.Cleaner.$anonfun$doClean$6$adapted(LogCleaner.scala:537)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at kafka.log.Cleaner.doClean(LogCleaner.scala:537)
	at kafka.log.Cleaner.clean(LogCleaner.scala:511)
	at kafka.log.LogCleaner$CleanerThread.cleanLog(LogCleaner.scala:380)
	at kafka.log.LogCleaner$CleanerThread.cleanFilthiestLog(LogCleaner.scala:352)
	at kafka.log.LogCleaner$CleanerThread.tryCleanFilthiestLog(LogCleaner.scala:332)
	at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:321)
	at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
	Suppressed: java.nio.file.FileSystemException: D:kafkakafka_2.13-2.8.0kafkakafka_2.13-2.8.0kafka-logs__consumer_offsets-440000000000000000000.timeindex.cleaned -> D:kafkakafka_2.13-2.8.0kafkakafka_2.13-2.8.0kafka-logs__consumer_offsets-440000000000000000000.timeindex.swap: 另一个程序正在使用此文件,进程无法访问。

		at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:86)
		at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
		at sun.nio.fs.WindowsFileCopy.move(WindowsFileCopy.java:301)
		at sun.nio.fs.WindowsFileSystemProvider.move(WindowsFileSystemProvider.java:287)
		at java.nio.file.Files.move(Files.java:1395)
		at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:901)
		... 20 more
原因

当消费者偏移日志清理,或者日志文件滚动时,由于文件重命名失败,Kafka broker 将会shut down。这是因为在windows中不能修改正在使用中的文件 造成的。

解决方案

在server.properties 配置文件中:

  1. 修改log.retention.hours=-1
  2. 文件末尾增加log.cleaner.enable=false
  3. 重启即可
建议

GitHub中有类似issue并且有PR,但是没有被合并,笔者用的是kafka_2.13-2.8.0 版本是有此问题的。不建议在win环境使用Kafka。

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/663337.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号