栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

docker-compose搭建zookeeper、kafka集群

docker-compose搭建zookeeper、kafka集群

1.拉取镜像 1.1.拉取kafka镜像

执行命令:docker pull wurstmeister/kafka

1.2.拉取zookeeper镜像

执行命令: docker pull wurstmeister/zookeeper

1.3.拉取kafka Manager镜像

执行命令:docker pull sheepkiller/kafka-manager

2.创建docker-compose文件
version: "3"
services:
    zoo1:
        image: zookeeper
        restart: always
        container_name: zoo1
        ports:
            - "2181:2181"
        environment:
            ZOO_MY_ID: 1
            ZOO_SERVERS: server.1=0.0.0.0:2888:3888 server.2=zoo2:2888:3888 server.3=zoo3:2888:3888
        volumes:
           - "/home/software/data/zookeeper/zoo1/logs:/datalog"
           - "/home/software/data/zookeeper/zoo1/data:/data"
           - "/home/software/data/zookeeper/zoo1/conf:/conf"
    zoo2:
        image: zookeeper
        restart: always
        container_name: zoo2
        ports:
            - "2182:2181"
        environment:
            ZOO_MY_ID: 2
            ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=0.0.0.0:2888:3888 server.3=zoo3:2888:3888
        volumes:
           - "/home/software/data/zookeeper/zoo2/logs:/datalog"
           - "/home/software/data/zookeeper/zoo2/data:/data"
           - "/home/software/data/zookeeper/zoo2/conf:/conf"
    zoo3:
        image: zookeeper
        restart: always
        container_name: zoo3
        ports:
            - "2183:2181"
        environment:
            ZOO_MY_ID: 3
            ZOO_SERVERS: server.1=zoo1:2888:3888 server.2=zoo2:2888:3888 server.3=0.0.0.0:2888:3888
        volumes:
           - "/home/software/data/zookeeper/zoo3/logs:/datalog"
           - "/home/software/data/zookeeper/zoo3/data:/data"
           - "/home/software/data/zookeeper/zoo3/conf:/conf"
    kafka1:
        container_name: kafka1
        image: wurstmeister/kafka
        ports:
          - "9092:9092"
        depends_on:
          - zoo1
          - zoo2
          - zoo3
        environment:
          KAFKA_ADVERTISED_HOST_NAME: kafka1                    ## 修改:宿主机IP
          KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://主机ip:9092    ## 修改:宿主机IP
          KAFKA_ZOOKEEPER_CONNECT: 主机ip:2181,主机ip:2182,主机ip:2183
          KAFKA_ADVERTISED_PORT: 9092
          KAFKA_BROKER_ID: 1
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
          
    kafka2:
        container_name: kafka2
        image: wurstmeister/kafka
        ports:
          - "9093:9093"
        depends_on:
          - zoo1
          - zoo2
          - zoo3
        environment:
          KAFKA_ADVERTISED_HOST_NAME: kafka2                    ## 修改:宿主机IP
          KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://主机ip:9093    ## 修改:宿主机IP
          KAFKA_ZOOKEEPER_CONNECT: 主机ip:2181,主机ip:2182,主机ip:2183
          KAFKA_ADVERTISED_PORT: 9093
          KAFKA_BROKER_ID: 2
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

    kafka3:
        container_name: kafka3
        image: wurstmeister/kafka
        ports:
          - "9094:9094"
        depends_on:
          - zoo1
          - zoo2
          - zoo3
        environment:
          KAFKA_ADVERTISED_HOST_NAME: kafka3                    ## 修改:宿主机IP
          KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://主机ip:9094    ## 修改:宿主机IP
          KAFKA_ZOOKEEPER_CONNECT: 主机ip:2181,主机ip:2182,主机ip:2183
          KAFKA_ADVERTISED_PORT: 9094
          KAFKA_BROKER_ID: 3
          KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

    kafkamanager:
        container_name: kafka-manager
        image: sheepkiller/kafka-manager
        ports:
          - "9000:9000"
        environment:
          ZK_HOSTS: ZK_SERVER_IP:2181,ZK_SERVER_IP:2182,ZK_SERVER_IP:2183

参数说明:
KAFKA_ADVERTISED_HOST_NAME:docker宿主机IP,可以设置多个;
KAFKA_CREATE_TOPICS:启动时默认创建的topic;TestComposeTopic:2:1表示创建topic为TestComposeTopic、2个分区、1个副本;
KAFKA_ZOOKEEPER_CONNECT:连接ZK,可以连接ZK集群;
KAFKA_BROKER_ID:kafka的唯一标识符;
KAFKA_ADVERTISED_PORT:监听kafka的端口号;
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR:topic的副本数,不能超过broker的数量。
注意:KAFKA_ADVERTISED_LISTENERS 和 KAFKA_LISTENERS必须要有,否则可能无法正常使用。

3.运行docker-compose

窗口运行:docker-compose up
后台运行:docker-compose up -d

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/746252.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号