部署 Kafka 和 zookeeper 集群（均三节点）

首发地址：https://www.pengtao.vip/archives/bu-shu-kafkahe-zookeeperji-qun–jun-san-jie-dian-

本次部署为三台服务器部署zookeeper和Kafka集群。

自行安装 JDK，启动zookeeper需要Java环境

1、创建Zk server的集群安装目录

[root@rac2 /]# mkdir /data
[root@rac2 /]# mkdir -p /data/zookeeper

2、上传zookeeper软件并解压到相应目录

[root@rac2 data]# tar -zxvf apache-zookeeper-3.7.0-bin.tar.gz  -C /data/zookeeper/

3、创建日志目录

#创建快照日志存放目录：
mkdir -p /data/zookeeper/dataDir

#创建事务日志存放目录：
mkdir -p /data/zookeeper/dataLogDir

4、修改zookeeper配置文件

[root@rac2 data]# cd /data/zookeeper/conf/
[root@rac2 conf]# mv zoo_sample.cfg zoo.cfg
[root@rac2 conf]# vim zoo.cfg

#配置内容如下，其他两个节点一样
[root@rac2 conf]# cat zoo.cfg |grep -v ^#|grep -v ^$
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/data/zookeeper/dataDir
dataLogDir=/data/zookeeper/dataLogDir
server.1=192.168.23.161:2888:3888
server.2=192.168.23.162:2888:3888
server.3=192.168.23.163:2888:3888
clientPort=2181
maxClientCnxns=60

在我们配置的dataDir指定的目录下面，创建一个myid文件，里面内容为一个数字，用来标识当前主机，对应zoo.cfg文件中配置的server.X中X的数字。

[root@rac1 conf]# echo "1" > /data/zookeeper/dataDir/myid   #主机1
[root@rac2 conf]# echo "2" > /data/zookeeper/dataDir/myid   #主机2
[root@rac3 conf]# echo "3" > /data/zookeeper/dataDir/myid   #主机3

5、关闭防火墙

systemctl stop firewalld
systemctl disable firewalld

6、启动zookeeper集群，每个节点都执行

/data/zookeeper/bin/zkServer.sh start

7、查看Zookeeper集群状态

[root@rac1 data]# zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /data/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

[root@rac2 data]# zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /data/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: follower

[root@rac3 data]# zookeeper/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /data/zookeeper/bin/../conf/zoo.cfg
Client port found: 2181. Client address: localhost. Client SSL: false.
Mode: leader

1、 下载Kafka

下载Kafka安装文件到data下

wget https://downloads.apache.org/kafka/3.0.0/kafka_2.13-3.0.0.tgz

2、解压

tar zxf kafka_2.13-3.0.0.tgz

3、修改配置文件

[root@rac1 kafka_2.13-3.0.0]# cat config/server.properties |grep -v ^#|grep -v ^$
broker.id=0   #另外两个修改为1，2
listeners=PLAINTEXT://192.168.23.161:9092   #对应本机ip
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/tmp/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.23.161:2181,192.168.23.162:2181,192.168.23.163:2181
delete.topic.enble=true
zookeeper.connection.timeout.ms=18000
group.initial.rebalance.delay.ms=0

4、启动kafka（在保证zookeeper集群没问题的前提下启动）

[root@rac2 kafka_2.13-3.0.0]# nohup bin/kafka-server-start.sh  config/server.properties &

5、功能测试

#主机1创建一个topic
[root@rac1 kafka_2.13-3.0.0]# bin/kafka-topics.sh --create --topic producer --bootstrap-server 192.168.23.161:9092,192.168.23.162:9092,192.168.23.163:9092 --partitions 1 --replication-factor 1

#查看所有topic
[root@rac2 kafka_2.13-3.0.0]# bin/kafka-topics.sh --list --bootstrap-server 192.168.23.161:9092

#主机2启动生产者
[root@rac2 kafka_2.13-3.0.0]# bin/kafka-console-producer.sh --broker-list 192.168.23.161:9092,192.168.23.162:9092,192.168.23.163:9092 --topic producer
>999
>777

#主机3启动消费者
[root@rac3 kafka_2.13-3.0.0]# bin/kafka-console-consumer.sh --bootstrap-server 192.168.23.161:9092,192.168.23.162:9092,192.168.23.163:9092 --topic producer --from-beginning
999
777

可以正常生产和消费，部署完成。