springcloud Stream整合Kafka入门

安装

http://mirror.bit.edu.cn/apache/kafka/2.8.1/kafka_2.13-2.8.1.tgz

tar zxvf kafka_2.13-2.8.1.tgz

启动

启动zk

[root@log-system bin]# ./zookeeper-server-start.sh -daemon ../config/zookeeper.properties

启动kafka

[root@log-system bin]# ./kafka-server-start.sh -daemon ../config/server.properties

查看启动情况

[root@10-211-55-6 bin]# netstat -tunlp|egrep "(2181|9092)"
tcp6       0      0 :::9092                 :::*                    LISTEN      31445/java          
tcp6       0      0 :::2181                 :::*                    LISTEN      10379/java          
[root@10-211-55-6 bin]# 

创建topic

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic topic-pear

zookeeper：zookeeper host 和ip

replication-factor：副本数

partitions ：分区数

topic ：消息订阅名称

发布发布消息

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic topic-pear

消费消息

./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic topic-pear --from-beginning

问题 A default binder has been requested

生产者配置默认binder

spring:
  cloud:
    stream:
      default:
        binder: kafka
        producer:
          useNativeEncoding: true
        default-binder: kafka

at org.apache.kafka.common.serialization.ByteArraySerializer.serialize(ByteArraySerializer.java:19) ~[kafka-clients-2.3.1.jar:na]

生产者增加配置

spring.cloud.stream.bindings.stream-kafka-output.producer.use-native-encoding=true
spring.cloud.stream.bindings.stream-kafka-output.producer.partitionCount=5
spring.cloud.stream.bindings.stream-kafka-output.producer.configuration.key.serializer=org.apache.kafka.common.serialization.StringSerializer
spring.cloud.stream.bindings.stream-kafka-output.producer.configuration.value.serializer=org.apache.kafka.common.serialization.ByteArraySerializer

安装可视化管理

https://github.com/xxd763795151/kafka-console-ui

spring-cloud-stream-kafka-demo

demo: https://github.com/learn-group-demo/neo-spring-cloud-stream.git