- SaltStack进阶
- 1. masterless
- 1.1 应用场景
- 1.2 masterless配置
- 1.2.1 修改配置文件minion
- 1.2.2 关闭salt-minion服务
- 1.2.3 salt-call
- 2. salt-master高可用
- 2.1 salt-master高可用配置
- 2.1.1 master主机上安装salt-master
- 2.1.2 安装完成后修改minion端的配置文件
- 2.1.3 开启master主机上的salt-master和minion主机上的salt-minion
- 2.1.4 进行test.ping检测
- 2.1.5 传输密钥到master2
- 2.1.6 再去修改minion的配置文件把master的IP改为master2的IP
- 2.1.7 等待证书生成后，授权证书，去master2主机上进行test.ping检测
- 2.1.8 当两台master都能ping通之后，最后再进行高可用设置
- 2.19 测试
- salt-master HA配置步骤总结
- 3 salt-syndic分布式架构
- 3.1 salt-syndic架构图
- 3.2 salt-syndic的优劣势
- 3.3 salt-syndic部署
- 3.3.1 环境说明
- 3.3.2 安装salt-master与salt-syndic
- 3.3.3 配置master
- 3.3.4 配置syndic
- 3.3.5 配置minion
- 3.3.6 在syndic上接受minion主机的key
- 3.3.7 在master上接受syndic主机的key
- 3.3.8 执行状态文件
-
master 与 minion 网络不通或通信有延迟，即网络不稳定
-
想在 minion 端直接执行状态
传统的 SaltStack 是需要通过 master 来执行状态控制 minion 从而实现状态的管理，但是当网络不稳定的时候，当想在minion本地执行状态的时候，当在只有一台主机的时候，想执行状态该怎么办呢？这就需要用到 masterless 了。
有了masterless，即使你只有一台主机，也能玩saltstack，而不需要你有N台主机架构。
1.2 masterless配置 1.2.1 修改配置文件minion- 注释master行
- 取消注释file_client并设其值为local
- 设置file_roots
- 设置pillar_roots
[root@minion ~]# vim /etc/salt/minion
....此处省略N行
# resolved, then the minion will fail to start.
# master: salt //注释此行
....此处省略N行
file_client: local //取消此行注释并将值设为local
....此处省略N行
file_roots: //设置file_roots，base环境的路径，可添加其他环境路径
base:
- /srv/salt/
....此处省略N行
pillar_roots: //设置pillar_roots，base环境变量的路径，可添加其他环境变量路径
base:
- /srv/pillar/base
1.2.2 关闭salt-minion服务
使用 masterless 模式时是不需要启动任何服务的，包括salt-master和salt-minion。
[root@node1 ~]# systemctl disable --now salt-minion
Removed /etc/systemd/system/multi-user.target.wants/salt-minion.service.
[root@node1 ~]# systemctl status salt-minion
● salt-minion.service - The Salt Minion
Loaded: loaded (/usr/lib/systemd/system/salt-minion.service; disabled; vend>
Active: inactive (dead)
Docs: man:salt-minion(1)
file:///usr/share/doc/salt/html/contents.html
https://docs.saltproject.io/en/latest/contents.html
11月 29 19:58:56 node1 salt-minion[2883]: [ERROR ] Master hostname: 'salt' n>
11月 29 19:59:41 node1 salt-minion[2883]: [ERROR ] DNS lookup or connection >
11月 29 19:59:41 node1 salt-minion[2883]: [ERROR ] Master hostname: 'salt' n>
11月 29 20:00:26 node1 salt-minion[2883]: [ERROR ] DNS lookup or connection >
11月 29 20:00:26 node1 salt-minion[2883]: [ERROR ] Master hostname: 'salt' n>
11月 29 20:00:46 node1 systemd[1]: Stopping The Salt Minion...
11月 29 20:00:46 node1 salt-minion[2883]: [WARNING ] Minion received a SIGTERM>
11月 29 20:00:46 node1 salt-minion[2883]: The Salt Minion is shutdown. Minion >
11月 29 20:00:46 node1 systemd[1]: salt-minion.service: Succeeded.
11月 29 20:00:46 node1 systemd[1]: Stopped The Salt Minion.
[root@node1 ~]#
1.2.3 salt-call
masterless模式执行模块或状态时需要使用salt-call命令，而不再是salt或者salt-ssh。需要注意的是要使用salt-call的–local选项。
用万能模块执行命令
[root@node1 ~]# salt-call --local cmd.run 'date'
local:
Mon Nov 29 20:14:30 CST 2021
[root@node1 ~]#
[root@node1 ~]# salt-call --local cmd.run 'ls -l /root'
local:
total 61168
-rw-------. 1 root root 1084 Sep 30 09:21 anaconda-ks.cfg
-rw-r--r-- 1 root root 53599174 Nov 27 11:59 grafana-7.3.4-1.x86_64.rpm
-rw-r--r--. 1 root root 9030402 Nov 25 23:04 node_exporter-1.3.0.linux-amd64.tar.gz
[root@node1 ~]# ^C
[root@node1 ~]#
执行状态文件
[root@node1 ~]# salt-call --local state.sls init.history.main
local:
----------
ID: /etc/profile
Function: file.append
Result: True
Comment: Appended 1 lines
Started: 20:23:38.371590
Duration: 12.966 ms
Changes:
----------
diff:
---
+++
@@ -83,3 +83,4 @@
. /etc/bashrc
fi
fi
+export HISTTIMEFORMAT="%F %T `whoami` "
Summary for local
------------
Succeeded: 1 (changed=1)
Failed: 0
------------
Total states run: 1
Total run time: 12.966 ms
2. salt-master高可用
2.1 salt-master高可用配置
涉及到高可用时，数据的同步是个永恒的话题，我们必须保证高可用的2个master间使用的数据是一致的，包括：
-
/etc/salt/master配置文件
-
/etc/salt/pki目录下的所有key
-
/srv/下的salt和pillar目录下的所有文件
保障这些数据同步的方案有：
-
nfs挂载
-
rsync同步
-
使用gitlab进行版本控制
安全相关：
为保证数据的同步与防止丢失，可将状态文件通过gitlab进行版本控制管理。
环境说明：
| 主机名 | IP | 职务 |
|---|---|---|
| master | 192.168.200.145 | 主master |
| master2 | 192.168.200.151 | 备master |
| node1 | 192.168.200.144 | minion |
环境准备
2.1.1 master主机上安装salt-master//配置salt源 [root@master2 ~]# rpm --import https://repo.saltproject.io/py3/redhat/8/x86_64/latest/SALTSTACK-GPG-KEY.pub [root@master2 ~]# curl -fsSL https://repo.saltproject.io/py3/redhat/8/x86_64/latest.repo | tee /etc/yum.repos.d/salt.repo 下载salt-master [root@master2 ~]# yum -y install salt-master
在minion上安装salt-minion
//配置salt源 [root@minion ~]# rpm --import https://repo.saltproject.io/py3/redhat/8/x86_64/latest/SALTSTACK-GPG-KEY.pub [root@minion ~]# curl -fsSL https://repo.saltproject.io/py3/redhat/8/x86_64/latest.repo | tee /etc/yum.repos.d/salt.repo //安装salt-minion [root@minion ~]# yum -y install salt-minion2.1.2 安装完成后修改minion端的配置文件
...... master: 192.168.200.145 //指定主master # Set http proxy information for the minion when doing requests ......2.1.3 开启master主机上的salt-master和minion主机上的salt-minion
[root@master ~]# systemctl restart salt-master [root@master ~]# ss -antl State Recv-Q Send-Q Local Address:Port Peer Address:Port Process LISTEN 0 128 0.0.0.0:22 0.0.0.0:* LISTEN 0 128 0.0.0.0:4505 0.0.0.0:* LISTEN 0 128 0.0.0.0:4506 0.0.0.0:* LISTEN 0 128 [::]:22 [::]:* [root@master ~]# [root@node1 ~]# systemctl start salt-minion State Recv-Q Send-Q Local Address:Port Peer Address:Port Process LISTEN 0 128 0.0.0.0:22 0.0.0.0:* LISTEN 0 128 [::]:22 [::]:*2.1.4 进行test.ping检测
[root@master ~]# salt-key -L
Accepted Keys:
Denied Keys:
Unaccepted Keys:
master
node1
Rejected Keys:
[root@master ~]#
[root@master ~]# salt-key -ya node1
The following keys are going to be accepted:
Unaccepted Keys:
node1
Key for minion node1 accepted.
[root@master ~]# salt-key -L
Accepted Keys:
node1
Denied Keys:
Unaccepted Keys:
master
Rejected Keys:
[root@master ~]#
[root@master ~]# salt '*' test.ping
node1:
True
[root@master ~]#
2.1.5 传输密钥到master2
当主master与minion端ping通后，再将master主机上的/etc/salt/pki/master目录中的公钥与私钥传（master.pem master.pub）输到备（masters主机）的/etc/salt/pki/master目录中。
[root@master ~]# cd /etc/salt/pki/master/ [root@master master]# ls master.pem minions minions_denied minions_rejected master.pub minions_autosign minions_pre [root@master master]# scp /etc/salt/pki/master/master.pem 192.168.200.151:/etc/salt/pki/master/ The authenticity of host '192.168.200.151 (192.168.200.151)' can't be established. ECDSA key fingerprint is SHA256:4VPYR1bpyvu6BYTzkAkAZWVS/UttttbRezAdzsbF9N0. Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '192.168.200.151' (ECDSA) to the list of known hosts. root@192.168.200.151's password: master.pem 100% 1679 1.2MB/s 00:00 [root@master master]# [root@master master]# scp /etc/salt/pki/master/master.pub 192.168.200.151:/etc/salt/pki/master root@192.168.200.151's password: master.pub 100% 451 342.0KB/s 00:00 [root@master master]#2.1.6 再去修改minion的配置文件把master的IP改为master2的IP
..... # Set the location of the salt master server. If the master server cannot be # resolved, then the minion will fail to start. #master: salt master: 192.168.58.30 //指定备masters # Set http proxy information for the minion when doing requests ...... //修改完成后重启salt-minion root@node1 ~]# systemctl restart salt-minion2.1.7 等待证书生成后，授权证书，去master2主机上进行test.ping检测
[root@master2 master]# salt-key -L
Accepted Keys:
Denied Keys:
Unaccepted Keys:
node1
Rejected Keys:
[root@master2 master]# salt-key -ya node1
The following keys are going to be accepted:
Unaccepted Keys:
node1
Key for minion node1 accepted.
[root@masters master]# salt '192.168.200.151' test.ping
192.168.200.151:
True
2.1.8 当两台master都能ping通之后，最后再进行高可用设置
minion端配置
..... # Set the location of the salt master server. If the master server cannot be # resolved, then the minion will fail to start. #master: salt master: - 192.168.200.145 //指定主master - 192.168.200.151 //指定备master2 # Set http proxy information for the minion when doing requests ......
配置故障转移
[root@minion ~]# vim /etc/salt/minion # beacons) without a master connection master_type: failover //高可用（故障转移） ---------- # connection events. # master_alive_interval: 10 //主机等待的时间间隔 //配置完成后重启salt-minion [root@minion ~]# systemctl restart salt-minion2.19 测试
在主(master)上test.ping测试
[root@master master]# salt '192.168.200.145' test.ping
192.168.200.145:
True
在备(master2)上test.ping测试
[root@masters master]# salt '192.168.200.151' test.ping
192.168.200.151:
Minion did not return. [No response]
The minions may not have all finished running and any remaining minions will return upon completion. To look up the return data for this job later, run the following command:
salt-run jobs.lookup_jid 20211129105415725809
ERROR: Minions returned with non-zero exit code
//当两台主机都在运行状态时，minion只能连接到主master上
模拟主master宕机
[root@master ~]# systemctl stop salt-master [root@master ~]# ss -anlt State Recv-Q Send-Q Local Address:Port Peer Address:Port LISTEN 0 128 0.0.0.0:22 0.0.0.0:* LISTEN 0 128 [::]:22
在备master2主机上进行test.ping检测
[root@masters master]# salt '192.168.200.151' test.ping
192.168.200.151:
True
去salt-minion查看minion服务状态
[root@node1 pki]# systemctl status salt-minion
● salt-minion.service - The Salt Minion
Loaded: loaded (/usr/lib/systemd/system/salt-minion.service; enabled; vendor preset: disabled)
Active: active (running) since Tue 2021-11-30 19:10:01 CST; 5min ago
Docs: man:salt-minion(1)
file:///usr/share/doc/salt/html/contents.html
https://docs.saltproject.io/en/latest/contents.html
Main PID: 1822 (salt-minion)
Tasks: 6 (limit: 49290)
Memory: 87.0M
CGroup: /system.slice/salt-minion.service
├─1822 /usr/bin/python3.6 /usr/bin/salt-minion
├─1829 /usr/bin/python3.6 /usr/bin/salt-minion
└─1831 /usr/bin/python3.6 /usr/bin/salt-minion
11月 30 19:10:01 node1 systemd[1]: Starting The Salt Minion...
11月 30 19:10:01 node1 systemd[1]: Started The Salt Minion.
11月 30 19:10:54 node1 salt-minion[1822]: [CRITICAL] 'master_type' set to 'failover' but 'retry_dns' is not 0. Setting 'retry_dns' to 0 to failover to the next master on DNS errors.
11月 30 19:14:18 node1 salt-minion[1822]: [WARNING ] Master ip address changed from 192.168.200.145 to 192.168.200.151
11月 30 19:14:18 node1 salt-minion[1822]: [WARNING ] Master ip address changed from 192.168.200.145 to 192.168.200.151
[root@node1 pki]#
//可以看到这里master主节点IP正在转换成备节点IP
salt-master HA配置步骤总结
-
1.创建备服务器
-
2.将主服务器上密钥复制到备服务器中
-
3.启动备服务器
-
4.配置 minions 以连接到备 master
-
5.重启minions
-
6.接受备上的密钥
优势：
- 可以通过syndic实现更复杂的salt架构
- 减轻master的负担
劣势：
- syndic的/srv目录下的salt和pillar目录内容要与最顶层的master下的一致，所以要进行数据同步，同步方案同salt-master高可用
- 最顶层的master不知道自己有几个syndic，它只知道自己有多少个minion，并不知道这些minion是由哪些syndic来管理的
| 主机IP | 角色 | 安装的应用 |
|---|---|---|
| 192.168.200.145 | Master | salt-master |
| 192.168.200.151 | Syndic | salt-master salt-syndic |
| 192.168.200.144 | Minion | salt-minion |
| 192.168.200.154 | Minion | salt-minion |
在192.168.200.151主机上安装salt-master与salt-syndic，安装前需自行配置salt源
[root@syndic ~]# yum -y install salt-master salt-syndic3.3.3 配置master
修改master的master配置文件
- 取消注释order_master
- 将order_master的值设为True
[root@master ~]# vim /etc/salt/master ...... # Set the order_masters setting to True if this master will command lower # masters' syndic interfaces. order_masters: True //取消注释，修改此处的值为True ...... [root@master ~]# systemctl restart salt-master3.3.4 配置syndic
修改syndic所在主机的master配置文件
- 取消注释syndic_master
- 将syndic_master的值设为master的IP
[root@syndic ~]# vim /etc/salt/master ...... # If this master will be running a salt syndic daemon, syndic_master tells # this master where to receive commands from. syndic_master: masterofmasters //此行取消注释 ...... [root@syndic ~]# systemctl enable --now salt-master Created symlink /etc/systemd/system/multi-user.target.wants/salt-master.service → /usr/lib/systemd/system/salt-master.service. [root@syndic ~]# systemctl enable --now salt-syndic Created symlink /etc/systemd/system/multi-user.target.wants/salt-syndic.service → /usr/lib/systemd/system/salt-syndic.service. [root@syndic ~]# systemctl restart salt-master [root@syndic ~]# systemctl restart salt-syndic [root@syndic ~]#3.3.5 配置minion
配置minion，将master指向syndic所在主机
[root@node1 ~]# vi /etc/salt/minion ...... # Set the location of the salt master server. If the master server cannot be # resolved, then the minion will fail to start. #master: salt master: 192.168.200.151 //把IP修改为sydinc主机上的IP ...... # Set http proxy information for the minion when doing requests [root@node1 ~]# systemctl restart salt-minion
[root@node2 ~]# vi /etc/salt/minion ...... # Set the location of the salt master server. If the master server cannot be # resolved, then the minion will fail to start. #master: salt master: 192.168.200.151 //把IP修改为sydinc主机上的IP ...... # Set http proxy information for the minion when doing requests [root@node2 ~]# systemctl restart salt-minion
在所有minion上做同样的操作，注意，要设置minion配置文件中的id参数，指向minion自身的ip地址或主机名，必须能够唯一标识minion本机。
3.3.6 在syndic上接受minion主机的key[root@syndic ~]# salt-key -L Accepted Keys: Denied Keys: Unaccepted Keys: node1 node2 Rejected Keys: [root@syndic ~]# salt-key -yA The following keys are going to be accepted: Unaccepted Keys: node1 node2 Key for minion node1 accepted. Key for minion node2 accepted. [root@syndic ~]# salt-key -L Accepted Keys: node1 node2 Denied Keys: Unaccepted Keys: Rejected Keys: [root@syndic ~]#3.3.7 在master上接受syndic主机的key
[root@master master]# salt-key -L Accepted Keys: Denied Keys: Unaccepted Keys: syndic Rejected Keys: [root@master master]# salt-key -yA The following keys are going to be accepted: Unaccepted Keys: syndic Key for minion syndic accepted. [root@master master]# salt-key -L Accepted Keys: syndic Denied Keys: Unaccepted Keys: Rejected Keys:
master上执行模块或状态检验有几个minion应答
[root@master master]# salt '*' test.ping
node2:
True
node1:
True
3.3.8 执行状态文件
//先将master上的/srv目录传输到syndic主机中
[root@master ~]# scp -r /srv/* 192.168.200.151:/srv/
//再到syndic主机中添加环境变量
[root@syndic ~]# vim /etc/salt/master
......
#
file_roots:
base:
- /srv/salt/base
prod:
- /srv/salt/prod
.......
pillar_roots:
base:
- /srv/pillar/base
prod:
- /srv/pillar/prod
[root@master ~]# salt '*' state.sls init.history.main
node1:
----------
ID: /etc/profile
Function: file.append
Result: True
Comment: Appended 1 lines
Started: 22:04:03.656139
Duration: 16.797 ms
Changes:
----------
diff:
---
+++
@@ -83,3 +83,4 @@
. /etc/bashrc
fi
fi
+export HISTTIMEFORMAT="%F %T `whoami` "
Summary for node1
------------
Succeeded: 1 (changed=1)
Failed: 0
------------
Total states run: 1
Total run time: 16.797 ms
