下载解压kafka_2.11-0.11.0.2.tgz
1.下载kafka，链接 https://kafka.apache.org/downloads;
2.配置kafka的JMX PORT,修改bin/kafka-server-start.sh
添加一行 export JMX_PORT="9999" # 这里9999为设置的jmx端口
3.下载jmx_exporter,链接 https://github.com/prometheus/jmx_exporter, 将jar包(jmx_prometheus_javaagent-0.16.1.jar)放在kafka家目录下
4.创建文件kafka-agent.yaml, kafka家目录下, 内容为你需要的kafka metrics
hostPort: 127.0.0.1:9999 lowercaseOutputName: true whitelistObjectNames: - "kafka.controller:type=KafkaController,name=OfflinePartitionsCount" - "kafka.controller:type=KafkaController,name=ActiveControllerCount" - "kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec" - "kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec" - "kafka.server:type=ReplicaManager,name=UnderReplicatedPartitions" - "kafka.server:type=ReplicaFetcherManager,name=MaxLag,clientId=Replica"
5.再次修改bin/kafka-server-start.sh
添加一行
export KAFKA_OPTS="-javaagent:/opt/software/kafka_2.11-0/jmx_prometheus_javaagent-0.16.1.jar=9990:/opt/software/kafka_2.11-0/kafka-agent.yaml"
注：jmx_prometheus_javaagent-0.9.jar=9990:, 9990为jmx_exporter提供数据的端口，之后我们访问http://xxx.xxx.xxx:9990/metrics就能获得需要的metrics数据
6.使用prometheus抓取数据源，编辑prometheus目录下的配置文件prometheus.yml
7.结合grafana来做监控看板进行监控
