1、背景:
本人kafka使用小白,使用kafka produce生产数据的时候,从后台看kafka topic有没有数据很不方便,从网上找了找,发现有个开源工具kafka eagle比较好使。
2、软件准备:
官方网站链接:EFAK
刚开始下载了一个官方最新版的,v2.0.9版本,但是安装一直失败,报jvm崩溃,刚开始也不知道什么原因,选择了一个比较低的版本v1.4.8版本
3、安装流程:
(1)从官网获取安装包,一直解压,/opt/software/kafka_eagle/kafka-eagle-bin-1.4.8/kafka-eagle-web-1.4.8
(2)修改conf/system-config.properties.我这里只需要监控一个集群,主要修改了以下几个参数,其他cluster2 cluster3相关的参数,我都给注释掉了
kafka.eagle.zk.cluster.alias=cluster1 cluster1.zk.list=10.144.17.170:2181,10.144.17.171:2181,10.144.17.172:2181 kafka.eagle.driver=com.mysql.jdbc.Driver kafka.eagle.url=jdbc:mysql://*****:3306/kafka_ke?createDatabaseIfNotExist=true&characterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull kafka.eagle.username=root kafka.eagle.password=root
(3)执行安装 sh bin/ke.sh start
首次安装的时候,直接进程崩溃,看看jvm error,发现有如下问题:
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j java.net.Inet6AddressImpl.lookupAllHostAddr(Ljava/lang/String;)[Ljava/net/InetAddress;+0 j java.net.InetAddress$2.lookupAllHostAddr(Ljava/lang/String;)[Ljava/net/InetAddress;+4 j java.net.InetAddress.getAddressesFromNameService(Ljava/lang/String;Ljava/net/InetAddress;)[Ljava/net/InetAddress;+51 j java.net.InetAddress.getAllByName0(Ljava/lang/String;Ljava/net/InetAddress;Z)[Ljava/net/InetAddress;+29 j java.net.InetAddress.getAllByName(Ljava/lang/String;Ljava/net/InetAddress;)[Ljava/net/InetAddress;+383 j java.net.InetAddress.getAllByName(Ljava/lang/String;)[Ljava/net/InetAddress;+2 j java.net.InetAddress.getByName(Ljava/lang/String;)Ljava/net/InetAddress;+1 j java.net.InetSocketAddress.(Ljava/lang/String;I)V+15 j sun.net.NetworkClient.doConnect(Ljava/lang/String;I)Ljava/net/Socket;+92 j sun.net.www.http.HttpClient.openServer(Ljava/lang/String;I)V+4 j sun.net.www.http.HttpClient.openServer()V+114 j sun.net.www.http.HttpClient. (Ljava/net/URL;Ljava/net/Proxy;I)V+125 j sun.net.www.http.HttpClient.New(Ljava/net/URL;Ljava/net/Proxy;IZLsun/net/www/protocol/http/HttpURLConnection;)Lsun/net/www/http/HttpClient;+259 j sun.net.www.http.HttpClient.New(Ljava/net/URL;Ljava/net/Proxy;ILsun/net/www/protocol/http/HttpURLConnection;)Lsun/net/www/http/HttpClient;+5 j sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Ljava/net/URL;Ljava/net/Proxy;I)Lsun/net/www/http/HttpClient;+4 j sun.net.www.protocol.http.HttpURLConnection.plainConnect0()V+357 j sun.net.www.protocol.http.HttpURLConnection.plainConnect()V+71 j sun.net.www.protocol.http.HttpURLConnection.connect()V+20 j sun.net.www.protocol.http.HttpURLConnection.getInputStream0()Ljava/io/InputStream;+195 j sun.net.www.protocol.http.HttpURLConnection.getInputStream()Ljava/io/InputStream;+52 j net.sf.ehcache.util.UpdateChecker.getUpdateProperties(Ljava/net/URL;)Ljava/util/Properties;+13 j net.sf.ehcache.util.UpdateChecker.doCheck()V+26 j net.sf.ehcache.util.UpdateChecker.checkForUpdate()V+9 j net.sf.ehcache.util.UpdateChecker.run()V+1 j java.util.TimerThread.mainLoop()V+221 j java.util.TimerThread.run()V+1 v ~StubRoutines::call_stub
从网上找了很多答案,说是encache.xml配置的有问题,需要关闭updateCheck。
1) 在kafka-ke安装目录下搜encache.xml,找到这个配置文件shiro-ehcache.xml,添加 updateCheck="false", 修改如下:
2) 编辑ke.sh文件,把下面几行注释掉,不然每次都从war包中重新解压
#rm -rf $KE_HOME/kms/webapps/ke
#rm -rf $KE_HOME/kms/work
#mkdir -p $KE_HOME/kms/webapps/ke
#cd $KE_HOME/kms/webapps/ke
#${JAVA_HOME}/bin/jar -xvf $KE_HOME/kms/webapps/ke.war
(4)修改完成后,再次执行 sh bin/ke.sh start,结果发现新的错误,查看日志
Java frames: (J=compiled Java code, j=interpreted, Vv=VM code) j java.net.Inet6AddressImpl.lookupAllHostAddr(Ljava/lang/String;)[Ljava/net/InetAddress;+0 j java.net.InetAddress$2.lookupAllHostAddr(Ljava/lang/String;)[Ljava/net/InetAddress;+4 j java.net.InetAddress.getAddressesFromNameService(Ljava/lang/String;Ljava/net/InetAddress;)[Ljava/net/InetAddress;+51 j java.net.InetAddress.getAllByName0(Ljava/lang/String;Ljava/net/InetAddress;Z)[Ljava/net/InetAddress;+29 j java.net.InetAddress.getAllByName(Ljava/lang/String;Ljava/net/InetAddress;)[Ljava/net/InetAddress;+383 j java.net.InetAddress.getAllByName(Ljava/lang/String;)[Ljava/net/InetAddress;+2 j java.net.InetAddress.getByName(Ljava/lang/String;)Ljava/net/InetAddress;+1 j java.net.InetSocketAddress.(Ljava/lang/String;I)V+15 j sun.net.NetworkClient.doConnect(Ljava/lang/String;I)Ljava/net/Socket;+92 j sun.net.www.http.HttpClient.openServer(Ljava/lang/String;I)V+4 j sun.net.www.http.HttpClient.openServer()V+114 j sun.net.www.http.HttpClient. (Ljava/net/URL;Ljava/net/Proxy;I)V+125 j sun.net.www.http.HttpClient.New(Ljava/net/URL;Ljava/net/Proxy;IZLsun/net/www/protocol/http/HttpURLConnection;)Lsun/net/www/http/HttpClient;+259 j sun.net.www.http.HttpClient.New(Ljava/net/URL;Ljava/net/Proxy;ILsun/net/www/protocol/http/HttpURLConnection;)Lsun/net/www/http/HttpClient;+5 j sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Ljava/net/URL;Ljava/net/Proxy;I)Lsun/net/www/http/HttpClient;+4 j sun.net.www.protocol.http.HttpURLConnection.plainConnect0()V+357 j sun.net.www.protocol.http.HttpURLConnection.plainConnect()V+71 j sun.net.www.protocol.http.HttpURLConnection.connect()V+20 j sun.net.www.protocol.http.HttpURLConnection.getInputStream0()Ljava/io/InputStream;+195 j sun.net.www.protocol.http.HttpURLConnection.getInputStream()Ljava/io/InputStream;+52 j org.quartz.utils.UpdateChecker.getUpdateProperties(Ljava/net/URL;)Ljava/util/Properties;+13 j org.quartz.utils.UpdateChecker.doCheck()V+17 j org.quartz.utils.UpdateChecker.checkForUpdate()V+1 j org.quartz.utils.UpdateChecker.run()V+1 j java.util.TimerThread.mainLoop()V+221 j java.util.TimerThread.run()V+1 v ~StubRoutines::call_stub
发现和上次的很像,但是这个是org.quartz.utils.UpdateChecker,还得关掉这里。
进入/opt/software/kafka_eagle/kafka-eagle-bin-1.4.8/kafka-eagle-web-1.4.8/kms/webapps/ke/WEB-INF/classes目录下,修改alert-quartz.xml、mbean-quartz.xml、metrics-quartz.xml、topic-quartz.xml
在这几个配置文件中找到如下配置,主要是skipUpdateCheck配置,这四个文件我都修改了。
true
(5)再次安装时,发现页面总算能登进去了,但是没有kafka的信息。
4、修改kafka配置
进入$KAFKA_HOME/bin目录下,修改kafka-run-class.sh文件,在文件的首行增加 JMX_PORT=9988
然后重启所有的kafka节点
5、再次重启kafka-eagle程序,最后发现可以监控到kafka信息。



