ELK+Filebeat+Redis部署应用指导书（二）

原创作者：运维工程师 谢晋

• ELK+Filebeat+Redis的部署
• • 安装JDK
  • Elasticsearch安装
• Redis安装
• Nginx安装

Elasticsearch安装需要JDK版本大于等于1.8

# yum install java-1.8.0-openjdk.x86_64 java-1.8.0-openjdk-devel.x86_64 -y

1. 安装Elasticsearch软件包
   在根目录下创建data目录

# mkdir /data

上传Elasticsearch 6.5.4的tar包并解压到data目录下

# tar -xvf elasticsearch-6.5.4.tar.gz -C /data/

Elasticsearch用户创建
创建单独的ES用户

# useradd elk

给用户增加相应目录的访问权限

# chown elk:elk -R /data/elasticsearch-6.5.4

切换用户

# su - elk

2. 修改Elasticsearch配置文件
   将下列参数进行修改

$ cd /data/elasticsearch-6.5.4/config/
$ vim elasticsearch.yml
node.name: node-1
path.data: /data/elasticsearch-6.5.4/data
path.logs: /data/elasticsearch-6.5.4/logs
network.host: 0.0.0.0
http.port: 9200

3. 调整jvm启动参数

$ vi jvm.options
-Xms128m
-Xmx128m

-Xms初始堆大小 物理内存的1/64(<1GB) 默认(MinHeapFreeRatio参数可以调整)空余堆内存小于40%时，JVM就会增大堆直到-Xmx的最大限制
-Xmx最大堆大小物理内存的1/4(<1GB) 默认(MaxHeapFreeRatio参数可以调整)空余堆内存大于70%时，JVM会减少堆直到 -Xms的最小限制

4. 调整系统内核

# vim /etc/sysctl.conf
vm.max_map_count = 262144
# sysctl -p

5. 修改连接数

# vim /etc/security/limits.conf
* soft nofile 65536
* hard nofile 131072
* soft memlock 2048
* hard memlock 2048

6. 启动Elasticsearch
   进入bin目录下启动服务

# cd /data/elasticsearch-6.5.4/bin
# ./elasticsearch -d

测试是否成功

# curl 10.11.11.30 :9200
{
  "name" : "node-1",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "Qx1tPJxgSReip6f4Zw7Pgw",
  "version" : {
    "number" : "6.5.4",
    "build_flavor" : "default",
    "build_type" : "tar",
    "build_hash" : "d2ef93d",
    "build_date" : "2018-12-17T21:17:40.758843Z",
    "build_snapshot" : false,
    "lucene_version" : "7.5.0",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

1. 安装gcc

# yum -y install gcc
# yum -y install centos-release-scl
# yum -y install devtoolset-9-gcc devtoolset-9-gcc-c++ devtoolset-9-binutils
# scl enable devtoolset-9 bash
# echo "/source /opt/rh/devtoolset-9/enable" >>/etc/profile

2. 下载并安装Redis软件包

# mkdir /data
# wget http://download.redis.io/releases/redis-6.0.4.tar.gz
# tar -xvf redis-6.0.4.tar.gz
# cd redis-6.0.4/
# make && make install PREFIX=/usr/local/redis

3. 后台启动Redis

# cp /data/redis-6.0.4/redis.conf /usr/local/redis/bin/
# vim /usr/local/redis/bin/redis.conf 
Bind 10.11.11.32   #redis地址
daemonize no
# cd /usr/local/redis/bin/
# ./redis-server redis.conf
16692:C 23 Dec 2021 11:21:57.292 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
16692:C 23 Dec 2021 11:21:57.292 # Redis version=6.0.4, bits=64, commit=00000000, modified=0, pid=16692, just started
16692:C 23 Dec 2021 11:21:57.292 # Configuration loaded
16692:M 23 Dec 2021 11:21:57.293 * Increased maximum number of open files to 10032 (it was originally set to 1024).
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 6.0.4 (00000000/0) 64 bit
  .-`` .-```.  ```/    _.,_ ''-._                                   
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 16692
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           http://redis.io        
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

1. 配置yum源

# vim /etc/yum.repos.d/nginx.repo
[nginx-stable]
name=nginx stable repo
baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
gpgcheck=0
enabled=1
gpgkey=https://nginx.org/keys/nginx_signing.key
[nginx-mainline]
name=nginx mainline repo
baseurl=http://nginx.org/packages/mailline/centos/$releasever/$basearch/
gpgcheck=0
enabled=0
gpgkey=https://nginx.org/keys/nginx_signing.key

2. 安装Nginx

# yum install nginx httpd-tools -y

3. 将Nginx日志格式设置为json
   在nginx.conf中添加json日志格式配置

# vim /etc/nginx/nginx.conf
http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;
    log_format json '{ "time_local": "$time_local", '
                       '"remote_addr": "$remote_addr", '
                       '"referer": "$http_referer", '
                       '"request": "$request", '
                       '"status": $status, '
                       '"bytes": $body_bytes_sent, '
                       '"agent": "$http_user_agent", '
                       '"x_forwarded": "$http_x_forwarded_for", '
                       '"up_addr": "$upstream_addr", '
                       '"up_host": "$upstream_http_host", '
                       '"upstream_time": "$upstream_response_time", '
                       '"request_time": "$request_time"'
                          '}';
    access_log  /var/log/nginx/access.log  json;

4. 启动Nginx

# systemctl start nginx

查看是否启动成功

至此Elasticsearch、redis、nginx安装完成。