栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

Kafka容错性测试

Kafka容错性测试

目录

一.创建3个配置文件二.启动zk和kafka三.创建topic、消费者、生产者四.三个broker五.kill掉一个,依然能工作六.kill掉两个,依然能工作七.重启一个,Isr会恢复

一.创建3个配置文件

server.properties:

broker.id=0
listeners=PLAINTEXT://hadoop000:9092
log.dirs=/home/hadoop/app/tmp/kafka-log

server-1.properties:

broker.id=1
listeners=PLAINTEXT://hadoop000:9093
log.dirs=/home/hadoop/app/tmp/kafka-log-1

server-2.properties:

broker.id=2
listeners=PLAINTEXT://hadoop000:9094
log.dirs=/home/hadoop/app/tmp/kafka-log-2
二.启动zk和kafka
[hadoop@hadoop000 bin]$ ./zkServer.sh start
[hadoop@hadoop000 config]$ kafka-server-start.sh -daemon $KAFKA_HOME/config/server.properties
[hadoop@hadoop000 config]$ kafka-server-start.sh -daemon $KAFKA_HOME/config/server-1.properties
[hadoop@hadoop000 config]$ kafka-server-start.sh -daemon $KAFKA_HOME/config/server-2.properties

[hadoop@hadoop000 config]$ jps -m
10304 QuorumPeerMain /home/hadoop/app/zookeeper-3.4.5-cdh5.15.1/bin/../conf/zoo.cfg
10611 Kafka /home/hadoop/app/kafka_2.11-2.2.1/config/server.properties
10967 Kafka /home/hadoop/app/kafka_2.11-2.2.1/config/server-1.properties
11333 Kafka /home/hadoop/app/kafka_2.11-2.2.1/config/server-2.properties
12837 ConsoleConsumer --bootstrap-server hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
13401 ConsoleProducer --broker-list hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
13966 Jps -m
三.创建topic、消费者、生产者
[hadoop@hadoop000 config]$ kafka-topics.sh --create --bootstrap-server hadoop000:9092,hadoop000:9093,hadoop000:9094 --replication-factor 3 --partitions 1 --topic access-topic-prod

[hadoop@hadoop000 bin]$ kafka-console-consumer.sh --bootstrap-server hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod

[hadoop@hadoop000 ~]$ kafka-console-producer.sh --broker-list hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
四.三个broker
[hadoop@hadoop000 config]$ kafka-topics.sh --describe --bootstrap-server hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
Topic:access-topic-prod	PartitionCount:1	ReplicationFactor:3	Configs:segment.bytes=1073741824
	Topic: access-topic-prod	Partition: 0	Leader: 2	Replicas: 2,1,0	Isr: 2,1,0
五.kill掉一个,依然能工作
[hadoop@hadoop000 config]$ kill -9 11333

[hadoop@hadoop000 config]$ jps -m
10304 QuorumPeerMain /home/hadoop/app/zookeeper-3.4.5-cdh5.15.1/bin/../conf/zoo.cfg
10611 Kafka /home/hadoop/app/kafka_2.11-2.2.1/config/server.properties
10967 Kafka /home/hadoop/app/kafka_2.11-2.2.1/config/server-1.properties
12837 ConsoleConsumer --bootstrap-server hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
13401 ConsoleProducer --broker-list hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
13979 Jps -m

[hadoop@hadoop000 config]$ kafka-topics.sh --describe --bootstrap-server hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
[2022-03-12 18:17:11,192] WARN [AdminClient clientId=adminclient-1] Connection to node -3 (hadoop000/192.168.198.128:9094) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
Topic:access-topic-prod	PartitionCount:1	ReplicationFactor:3	Configs:segment.bytes=1073741824
	Topic: access-topic-prod	Partition: 0	Leader: 1	Replicas: 2,1,0	Isr: 1,0
六.kill掉两个,依然能工作
[hadoop@hadoop000 config]$ kill -9 10967

[hadoop@hadoop000 config]$ jps -m
10304 QuorumPeerMain /home/hadoop/app/zookeeper-3.4.5-cdh5.15.1/bin/../conf/zoo.cfg
10611 Kafka /home/hadoop/app/kafka_2.11-2.2.1/config/server.properties
12837 ConsoleConsumer --bootstrap-server hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
13401 ConsoleProducer --broker-list hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
15148 Jps -m

[hadoop@hadoop000 config]$ kafka-topics.sh --describe --bootstrap-server hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
[2022-03-12 18:18:38,990] WARN [AdminClient clientId=adminclient-1] Connection to node -3 (hadoop000/192.168.198.128:9094) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient)
Topic:access-topic-prod	PartitionCount:1	ReplicationFactor:3	Configs:segment.bytes=1073741824
	Topic: access-topic-prod	Partition: 0	Leader: 0	Replicas: 2,1,0	Isr: 0
七.重启一个,Isr会恢复
[hadoop@hadoop000 bin]$ kafka-topics.sh --describe --bootstrap-server hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
Topic:access-topic-prod	PartitionCount:1	ReplicationFactor:3	Configs:segment.bytes=1073741824
	Topic: access-topic-prod	Partition: 0	Leader: 1	Replicas: 2,1,0	Isr: 1,0
	
[hadoop@hadoop000 bin]$ kafka-server-start.sh -daemon $KAFKA_HOME/config/server-2.properties

[hadoop@hadoop000 bin]$ kafka-topics.sh --describe --bootstrap-server hadoop000:9092,hadoop000:9093,hadoop000:9094 --topic access-topic-prod
Topic:access-topic-prod	PartitionCount:1	ReplicationFactor:3	Configs:segment.bytes=1073741824
	Topic: access-topic-prod	Partition: 0	Leader: 1	Replicas: 2,1,0	Isr: 1,0,2
转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/761088.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号