栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

ELK安装及配置

ELK安装及配置

一. Logstash6的安装
cd /opt
wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.2.tar.gz
tar -zxvf logstash-6.5.2.tar.gz
mv logstash-6.5.2 logstash
cd /opt/logstash/config
cp logstash-sample.conf syslog.conf
vi syslog.conf
# 定义日志源
input {  
  syslog {
    type => "system-syslog"  	# 定义类型
    port => 10514    			# 定义监听端口
  }
}
# 定义日志输出
output {  
  stdout {
    codec => rubydebug  		# 将日志输出到当前的终端上显示
  }
}
# 验证配置文件
/opt/logstash/bin/logstash --path.settings /opt/logstash/config/ -f /opt/logstash/config/syslog.conf --config.test_and_exit

正确输出如下:
Sending Logstash logs to /opt/software/logstash/logs which is now configured via log4j2.properties
[2018-11-23T09:28:36,184][WARN ][logstash.config.source.multilocal] Ignoring the 'pipelines.yml' file because modules or command line options are specified
Configuration OK
[2018-11-23T09:28:38,630][INFO ][logstash.runner          ] Using config.test_and_exit mode. Config Validation Result: OK. Exiting Logstash


--config.test_and_exit 指定检测完之后就退出,不然就会直接启动了
# 配置服务器的ip以及配置的监听端口
vim /etc/rsyslog.conf

# 增加自己的服务器IP
#### RULES ####
*.* @@xxx.xxx.xxx.xxx:10514


ps. 上面xxx.xxx.xxx.xxx需要替换成自己服务器的IP地址
# 重启rsyslog, 让配置生效
systemctl restart rsyslog

# 指定配置文件,启动logstash
cd /opt/logstash/bin
./logstash --path.settings /opt/software/logstash/config/ -f /opt/software/logstash/config/syslog.conf

# 打开新终端检查一下10514端口是否已被监听
netstat -lntp |grep 10514
tcp6       0      0 :::10514                :::*                    LISTEN      496336/java

# 然后在别的机器ssh登录到这台机器上,测试一下有没有日志输出:
{
          "severity" => 6,
         "timestamp" => "Sep 30 14:00:12",
               "pid" => "497144",
              "type" => "system-syslog",
         "logsource" => "VM-0-11-centos",
    "facility_label" => "security/authorization",
              "host" => "81.69.248.111",
           "program" => "sshd",
           "message" => "Disconnected from authenticating user root 188.166.180.17 port 42278 [preauth]n",
        "@timestamp" => 2021-09-30T06:00:12.000Z,
          "@version" => "1",
          "priority" => 86,
    "severity_label" => "Informational",
          "facility" => 10
}

如上,终端中以JSON的格式打印了收集到的日志,表示测试成功了。

二. 安装java
  1. 下载jdk: (13条消息) JDK 8 下载(jdk-8u212-windows-x64.exe 和 jdk-8u212-linux-x64.tar.gz)_baidu_20457783的博客-CSDN博客_jdk8下载

  2. 安装:

[root@VM-0-11-centos opt]# cd /opt
[root@VM-0-11-centos opt]# tar -zxvf jdk-8u212-linux-x64.tar.gz
[root@VM-0-11-centos opt]# vim /etc/profile

在/etc/profile文件最下面加入如下配置内容:
JAVA_HOME=/opt/jdk1.8.0_212
CLASSPATH=$JAVA_HOME/lib/
PATH=$PATH:$JAVA_HOME/bin
export PATH JAVA_HOME CLASSPATH

[root@VM-0-11-centos opt]# source /etc/profile 
[root@VM-0-11-centos opt]# java -version
三. 安装elasticsearch
rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch
在基于RedHat的发行版的/etc/yum.repos.d/目录中创建一个名为elasticsearch.repo的文件,其中包括:
[elasticsearch-6.x]
name=Elasticsearch repository for 6.x packages
baseurl=https://artifacts.elastic.co/packages/6.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

yum makecache
yum install elasticsearch -y

要将Elasticsearch配置为在系统引导时自动启动,请运行以下命令:

sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable elasticsearch.service
  • Elasticsearch可以按如下方式启动和停止:
sudo systemctl start elasticsearch.service
sudo systemctl stop elasticsearch.service

这些命令不会提供有关Elasticsearch是否已成功启动的反馈。相反,此信息将写入位于/ var / log / elasticsearch /中的日志文件中。 默认情况下,Elasticsearch服务不会记录systemd日志中的信息。要启用journalctl日志记录,必须从elasticsearch中的ExecStart命令行中删除–quiet选项。服务文件。

# 注释20行的 --quiet 
vim /etc/systemd/system/multi-user.target.wants/elasticsearch.service
# 使用tail查看journal:
sudo journalctl -f

# 要列出elasticsearch服务的日记帐分录:
sudo journalctl --unit elasticsearch

# 要从给定时间开始列出elasticsearch服务的日记帐分录:
sudo journalctl --unit elasticsearch --since  "2017-1-4 10:17:16"

# since 表示指定时间之前的记录

# 检查Elasticsearch是否正在运行
curl -XGET 'http://localhost:9200/?pretty'

{
  "name" : "WjrL56v",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "4fcDHU0eSNGZTe4ecjB4xQ",
  "version" : {
    "number" : "6.8.19",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "33e6bd0",
    "build_date" : "2021-09-15T16:38:13.498661Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.3",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

# 配置 Elasticsearch

# Elasticsearch 从默认的/etc/elasticsearch/elasticsearch.yml加载配置文件, 
 配置文件的格式参考: 
https://www.elastic.co/guide/en/elasticsearch/reference/current/settings.html

node.name: node-1
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
network.host: 192.168.0.209  # 默认localhost,自定义为ip
http.port: 9200


# RPM还具有系统配置文件(/etc/sysconfig/elasticsearch),允许您设置以下参数:
[root@linuxprobe elasticsearch]# egrep -v "^#|^$" /etc/sysconfig/elasticsearch 
ES_HOME=/usr/share/elasticsearch
JAVA_HOME=/usr/java/jdk1.8.0_111
CONF_DIR=/etc/elasticsearch
DATA_DIR=/var/lib/elasticsearch
LOG_DIR=/var/log/elasticsearch
PID_DIR=/var/run/elasticsearch
四. 安装Kibana
cd /opt
wget https://artifacts.elastic.co/downloads/kibana/kibana-6.2.4-x86_64.rpm
rpm -ivh kibana-6.2.4-x86_64.rpm
vim /etc/kibana/kibana.yml

# 取消2、7、21行的注释,行号不一定准确,但一定是下面这几项
server.port: 5601
server.host: "localhost"
elasticsearch.url: "http://localhost:9200"

# 启动
systemctl enable kibana
systemctl start kibana
netstat -plntu

通过netstat -plntu查看kibana是否启动成功,如果有端口号为5601的输出那就代表kibana启动成功了

参考文章: https://cloud.tencent.com/developer/article/1445264
https://www.cnblogs.com/cheesebar/p/9126171.html
https://blog.csdn.net/jeikerxiao/article/details/84403437

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/281154.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号