| 节点名 | ip地址 | OS版本 | 部署服务 | broker信息 |
|---|---|---|---|---|
| kafka01 | 192.168.96.10 | CentOS Linux release 7.9.2009 (Core) | jdk、zookeeper、kafka | broker : 0 |
| kafka02 | 192.168.96.11 | CentOS Linux release 7.9.2009 (Core) | jdk、zookeeper、kafka | broker : 1 |
| kafka03 | 192.168.96.12 | CentOS Linux release 7.9.2009 (Core) | jdk、zookeeper、kafka | broker : 2 |
jdk 1.8 版本：https://www.oracle.com/java/technologies/downloads/#java8zookeeper 3.7.0 版本: https://dlcdn.apache.org/zookeeper/zookeeper-3.7.0/apache-zookeeper-3.7.0-bin.tar.gzkafka 2.13-3.1.0 版本: https://dlcdn.apache.org/kafka/3.1.0/kafka_2.13-3.1.0.tgz
开始部署： 一、部署jdk
- 解压并重命名服务
cd /opt/software/ tar -zxvf jdk-8u321-linux-x64.tar.gz mv jdk1.8.0_321 /usr/local/jdk1.8.0
- 添加环境变量
vi /etc/profile
# 在文件尾部追加以下内容
export JAVA_HOME=/usr/local/jdk1.8.0
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
- 重载环境变量，使新配置生效
source /etc/profile
- 验证
java -version
继续按照以上方法部署另外2台服务器
二、部署zookeeper- 在线下载安装包
cd /opt/software/ wget https://dlcdn.apache.org/zookeeper/zookeeper-3.7.0/apache-zookeeper-3.7.0-bin.tar.gz --no-check-certificate
- 解压并重命名服务
tar -zxvf apache-zookeeper-3.7.0-bin.tar.gz mv apache-zookeeper-3.7.0-bin /usr/local/zookeeper-3.7.0
- 创建zookeeper服务的data目录
mkdir /usr/local/zookeeper-3.7.0/data
- 创建myid文件（zookeeper用于惟一标识自己的id）
echo 1 > /usr/local/zookeeper-3.7.0/data/myid
- 生成zoo.cfg配置文件
cp /usr/local/zookeeper-3.7.0/conf/zoo_sample.cfg /usr/local/zookeeper-3.7.0/conf/zoo.cfg
- 修改zoo.cfg配置文件
vi /usr/local/zookeeper-3.7.0/conf/zoo.cfg tickTime=2000 initLimit=10 syncLimit=5 dataDir=/usr/local/zookeeper-3.7.0/data clientPort=2181 server.1=192.168.96.10:2888:3888 server.2=192.168.96.11:2888:3888 server.3=192.168.96.12:2888:3888
- 使用scp同步发送到另外2台服务器
scp -r /usr/local/zookeeper-3.7.0 root@192.168.96.11:/usr/local/ scp -r /usr/local/zookeeper-3.7.0 root@192.168.96.12:/usr/local/
- 配置另外2个节点的myid文件
192.168.96.11 服务器
echo 2 > /usr/local/zookeeper-3.7.0/data/myid
192.168.96.12 服务器
echo 3 > /usr/local/zookeeper-3.7.0/data/myid
- 最后，分别在3台服务器上启动zk服务
/usr/local/zookeeper-3.7.0/bin/zkServer.sh start
- 验证
jps -m三、部署kafka
- 下载软件包并解压、重命名服务
cd /opt/software/ wget https://dlcdn.apache.org/kafka/3.1.0/kafka_2.13-3.1.0.tgz tar -zxvf kafka_2.13-3.1.0.tgz --no-check-certificate mv kafka_2.13-3.1.0 /usr/local/kafka-2.13
- 配置kafka
vi /usr/local/kafka-2.13/config/server.properties
kafka01服务器
broker.id=0 listeners=PLAINTEXT://192.168.96.10:9092 log.dirs=/tmp/kafka-logs zookeeper.connect=192.168.96.10:2181,192.168.96.11:2181,192.168.96.12:2181
- 使用scp同步发送到另外2台服务器
scp -r /usr/local/kafka-2.13 root@192.168.96.11:/usr/local/ scp -r /usr/local/kafka-2.13 root@192.168.96.12:/usr/local/
- 分别修改kafka02、kafka03的配置，如下
kafka02服务器
broker.id=1 listeners=PLAINTEXT://192.168.96.11:9092
kafka03服务器
broker.id=2 listeners=PLAINTEXT://192.168.96.12:9092
- 分别在3台服务器上，启动kafka集群
cd /usr/local/kafka-2.13 && bin/kafka-server-start.sh config/server.properties &四、验证测试
创建topic
bin/kafka-topics.sh --create --topic quickstart-events --bootstrap-server 192.168.96.10:9092
查看topic
bin/kafka-topics.sh --describe --topic quickstart-events --bootstrap-server 192.168.96.10:9092
生产者：发送消息
bin/kafka-console-producer.sh --topic quickstart-events --bootstrap-server 192.168.96.10:9092
消费者：处理消息
bin/kafka-console-consumer.sh --topic quickstart-events --from-beginning --bootstrap-server 192.168.96.10:9092
