栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

记录安装kafka的过程

记录安装kafka的过程

平时工作中用到了kafka这个消息中间件,但是自己没有动手安装过,所以自己在闲暇时尝试安装了一次kafka,本文就是记录一下本次安装遇到的问题与解决办法。

    先在服务器上opt下创建zookeeper和kafka文件夹
cd opt
mkdir  -p  /opt/zookeeper
mkdir  -p  /opt/kafka
    先找到国内的zookeeper和kafka的镜像下载下来:
cd /opt/zookeeper
wget https://dlcdn.apache.org/zookeeper/zookeeper-3.6.3/apache-zookeeper-3.6.3-bin.tar.gz 

cd /opt/kafka
wget http://archive.apache.org/dist/kafka/2.4.0/kafka_2.11-2.4.0.tgz
    下载好之后就行解压,先对zookeeper进行配置修改
cd /opt/zookeeper/
tar -zxvf apache-zookeeper-3.6.3-bin.tar.gz 

sudo mv apache-zookeeper-3.5.6-bin zookeeper ##把文件名修改为zookeeper

cd /opt/zookeeper/zookeeper/conf
cp zoo_sample.cfg zoo.cfg
sudo vi zoo.cfg


dataDir 定义:zookeeper保存数据的目录;
dataLogDir= zookeeper将写数据的日志文件保存在这个目录里;

    启动zookeeper
cd /opt/zookeeper/zookeeper/bin
./zkServer.sh start  ###启动

./zkServer.sh status ###启动后要查看是否成功启动
    对kafka包进行解压及配置修改
cd /opt/kafka
tar -xzvf kafka_2.11-2.4.0.gz

cd kafka_2.11-2.4.0
mkdir logs ## 创建日志

cd /opt/kafka/kafka_2.11-2.4.0/config
sudo vi server.properties

    以后台启动的方式启动kafka
nohup ./kafka-server-start.sh ../config/server.properties 1>/dev/null 2>&1 &
sudo lsof -i:9092 ##查看是否成功启动

    创建一个topic
[root@mode1 kafka_2.11-2.4.0] bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic zxx
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
Created topic zxx.

[root@mode1 kafka_2.11-2.4.0] bin/kafktopics.sh --list --zookeeper localhost:2181 ##查看所有topic
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
zlh
zxx
    生产者与消费者
[root@mode1 kafka_2.11-2.4.0] bin/kafka-console-producer.sh --broker-list xx.xx.x.x:9092 --topic zlh ##启动一
##个生产者,往里边写入数据
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should
 configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
>ll
>zz
>hh
>好
>131

#在服务器启动消费者

[root@mode1 kafka_2.11-2.4.0]# bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic zlh --from-beginning
OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N
ll
zz
hh
好
131

在自己mac电脑上启动一个消费者,外网连接kafka

➜  kafka_2.11-2.4.0 bin/kafka-console-consumer.sh --bootstrap-server xx.xx.xx.xx:9092 --topic zlh --from-beginning
ll
zz
hh
好
131

至此,在服务器搭建kafka已基本完成,接下来可以在代码中使用kafka来进行玩了

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/734797.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号