Kafka节点服役和退役

• 基础配置

vim /etc/sysconfig/network-scripts/ifcfg-ens33

BOOTPROTO=static
IPADDR=192.168.10.105
GATEWAY=192.168.10.2
DNS1=192.168.10.2

vim /etc/hostname

hadoop105

• 重启hadoop105
• 修改hadoop105中kafak的broker.id为3
• 删除hadoop105中kafka下的datas和logs
• 启动hadoop102、hadoop103、hadoop104上的kafka集群
• 单独启动hadoop105中的kafka

• 创建一个要均衡的主题

vim topics-to-move.json

{
    "topics": [
        {
            "topic": "first"
        }
    ],
    "version": 1
}

• 生成一个负载均衡的计划

bin/kafka-reassign-partitions.sh 
--bootstrap-server hadoop102:9092 
--topics-to-move-json-file topics-to-move.json 
--broker-list "0,1,2,3" 
--generate

• 创建副本存储计划（所有副本存储在broker0、broker1、broker2、broker3中）

vim increase-replication-factor.json

就是上一步生成的内容

{
    "version":1,
    "partitions":[
        {
            "topic":"first",
            "partition":0,
            "replic as":[2,3,0],
            "log_dirs":["any","any","any"]
        },
        {
            "topic":"first",
            "partition":1,
            "replicas":[3,0,1],
            "log_dirs":["any","any","any"]
        },
        {
            "topic":"first",
            "partition":2,
            "replicas":[0,1,2],
            "log_dirs":["any","any","any"]
        }
    ]
}

• 执行副本存储计划

bin/kafka-reassign-partitions.sh 
--bootstrap-server hadoop102:9092 
--reassignment-json-file increase-replication-factor.json 
--execute

• 验证副本存储计划

bin/kafka-reassign-partitions.hs 
--bootstrap-server hadoop102:9092 
--reassignment-json-file increase-replication-factor.json 
--verify

先按照退役一台节点，生成执行计划，然后按照服役时操作流程执行负载均衡

• 创建一个要均衡的主题

vim topics-to-move.json

{
    "topics": [
        {"topic": "first"}
    ],
    "version": 1
}

• 创建执行计划

bin/kafka-reassign-partitions.sh 
--bootstrap-server hadoop102:9092 
--topics-to-move-json-file topics-to-move.json 
--broker-list "0,1,2" 
--generate

• 创建副本存储计划(所有副本存储在broker0、broker1、broker2中)

vim /increase-replication-factor.json

就是上一步生成的数据

{
    "version":1,
    "partitions":[
        {
            "topic":"first",
            "partition":0,
            "replic as":[2,0,1],
            "log_dirs":["any","any","any"]
        },
        {
            "topic":"first",
            "partition":1,
            "replicas":[0,1,2],
            "log_dirs":["any","any","any"]
        },
        {
            "topic":"first",
            "partition":2,
            "replicas":[1,2,0],
            "log_dirs":["any","any","any"]
        }
    ]
}

• 执行副本存储计划

bin/kafak-reassign-partitions.sh 
--bootstrap-server hadoop102:9092 
--reassignment-json-file increase-replication-factor.json 
--execute

• 验证副本存储计划

bin/kafka-reassign-partitions.sh 
--bootstrap-server hadoop102:9092 
--reassignment-json-file increase-replication-factor.json 
--verify

• 执行停止命令

# 在hadoop105上执行停止命令即可
bin/kafka-server-stop.sh