ELK + logstash + rabbitmq 构建数据采集

centos8 + docker

docker hub 上复制 pull 命令，我pull的版本是8.1.2

docker pull logstash:8.1.2

本地创建logstash的config文件夹

mkdir ~/docker/logstash/config
mkdir ~/docker/logstash/config/conf.d

在config文件中添加logstash.yml文件

http.host: "0.0.0.0"
path.config: /usr/share/logstash/config/conf.d/*.conf
path.logs: /usr/share/logstash/logs

xpack.monitoring.enabled: true
xpack.monitoring.elasticsearch.hosts: [ "http://127.0.0.1:9200" ]

在conf.d文件中添加输入输出配置文件my_config.conf

input {
    rabbitmq {
        host => "127.0.0.1"
        port => 5672
        vhost => "virtual host"
        user => "user name"
        password => "password"
        exchange => "log"
        queue => "log_info"
        durable => true
        codec => "json"
    }
}

output {
    stdout { }
    elasticsearch {
        hosts => ["127.0.0.1:9200"]
        index => "rabbitmq_log"
    }
}

添加文件权限，否则后面启动时会提示 “/usr/share/logstash/config/logstash.yml permission denied"（权限不足的问题困扰了我一天，git和stack overflow 都没有合适的答案，最后在CSDN上找到了解答）

chmod -R 777 logstash

启动容器

sudo docker run -dit -p 5044:5044 -p 9600:9600 --name logstash --privileged=true -e ES_JAVA_OPTS="-Xms512m -Xmx512m" -v /home/wbxbuilds/docker/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml -v /home/wbxbuilds/docker/logstash/config/conf.d:/usr/share/logstash/config/conf.d -v /home/wbxbuilds/docker/logstash/logs:/usr/share/logstash/logs logstash:8.1.2

从dockerhub上拷贝pull命令，我下载的时7.0.0版本，网上说ELK的版本要和logstash一致，但是我目前还没遇到问题

创建文件夹

mkdir ~/docker/elasticsearch/config
mkdir ~/docker/elasticsearch/data
mkdir ~/docker/elasticsearch/plugins

添加配置文件elasticsearch.yml到config目录下

http.host: 0.0.0.0
http.cors.enabled: true
http.cors.allow-origin: "*"

给文件添加权限（被上面整怕了）

chmod -R 777 elasticsearch

启动容器

sudo docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -v /home/wbxbuilds/docker/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml -v /home/wbxbuilds/docker/elasticsearch/data:/usr/share/elasticsearch/data -v /home/wbxbuilds/docker/elasticsearch/plugins:/usr/share/elasticsearch/plugins elasticsearch:7.17.0