Linux高级实战专题篇：keepalived高可用（keepalived+LVS集群部署；keepalived+mysql集群部署；mysql主从复制自动化脚本）

keepalived高可用一，keepalived简介 1，keepalived是什么

keepalived是集群管理中保证集群高可用(HA)的一个服务软件，其功能类似于heartbeat，用来防止单点故障。

keepalived是以VRRP协议为实现基础的,当backup收不到vrrp包时就认为master宕掉了，这时就需要根据VRRP
的优先级来选举一个backup当master。这样我们就可以保证集群的高可用。

VRRP本身是数通方向的协议。
keepalived是以VRRP协议为实现基础的，VRRP全称Virtual Router Redundancy Protocol，即虚拟路由冗余协议。
虚拟路由冗余协议，可以认为是实现路由器高可用的协议，即将N台提供相同功能的路由器组成一个路由器组，这个组
里面有一个master和多个backup，master上面有一个对外提供服务的vip（该路由器所在局域网内其他机器的默认路
由为该vip），master会发组播，当backup收不到vrrp包时就认为master宕掉了，这时就需要根据VRRP的优先级来选
举一个backup当master。这样的话就可以保证路由器的高可用了。

keepalived主要有三个模块，分别是core、check和vrrp。core模块为keepalived的核心，负责主进程的启动、维护
以及全局配置文件的加载和解析。check负责健康检查，包括常见的各种检查方式。vrrp模块是来实现VRRP协议的

HA 高可用    
               client
                    |
            192.168.1.8   vip  vip给谁，要看谁是master，可飘逸的我们叫做资源
                     |
              
masterA---hearbeat--- backupB
 dip          心跳检测         dip
 
主要原因：网络 
脑裂：backup强资源，master不认为自己会死，他俩抢着为客户端服务
解决方案：shoot the other in the head 爆头  master    
        
如何判断谁是master：看vip，vip在谁上谁就是master        
        
什么是脑裂?
脑裂（split-brain）：指在一个高可用（HA）系统中，当联系着的两个节点断开联系时，本来为一个整体的系统，
分裂为两个独立节点，这时两个节点开始争抢共享资源，结果会导致系统混乱，数据损坏。
        
对于无状态服务的HA，无所谓脑裂不脑裂；但对有状态服务(比如MySQL)的HA，必须要严格防止脑裂。        
究竟是有状态服务，还是无状态服务，其判断依据——两个来自相同发起者的请求在服务器端是否具备上下文关系。   
        
举例：
   在商城里购买一件商品。需要经过放入购物车、确认订单、付款等多个步骤。由于HTTP协议本身是无状态的，所以
为了实现有状态服务，就需要通过一些额外的方案。比如最常见的session，将用户挑选的商品（购物车），保存到
session中，当付款的时候，再从购物车里取出商品信息

二，keepalived高可用+nginx负载均衡 1，实验过程

1、选择两台nginx服务器作为代理服务器/负载均衡器。
2、给两台代理服务器安装keepalived制作高可用生成VIP
3、准备两台nginx服务器作为web-server

2，实验部署

web-server-1：10.0.0.42
web-server-2：10.0.0.141
安装nginx，并且保证nginx正常运行
分别要在web-server上创建一个测试界面
echo "web-server-1" > /usr/share/nginx/html/index.html
echo "web-server-2" > /usr/share/nginx/html/index.html
检测web-server是否正常被访问

3，负载均衡部署（两台都做）

master：10.0.0.154
backup：10.0.0.27

master和backup都要做以下操作：
vim /etc/nginx/nginx.conf 
在http模块下第25行下面添加以下内容

  upstream xingdian {
        server 10.0.0.42:80;
        server 10.0.0.141:80;
        }

vim /etc/nginx/conf.d/default.conf 修改一下内容
    server {
    listen 80;
    server_name  localhost;
    location / {
        proxy_pass http://xingdian;
         proxy_redirect default;
         proxy_set_header Host $http_host;
         proxy_set_header X-Real-IP $remote_addr;
         proxy_set_header REMOTE-HOST $remote_addr;
         proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
   }

保证nginx的负载均衡可用，客户端可以访问测试：

4，keepalived实现调度器HA

1. 主/备调度器安装软件（安装keepalived）
[root@master ~]# yum install -y keepalived
[root@backup ~]# yum install -y keepalived
[root@master ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak （略）
[root@nginx-proxy-master ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id director1   #辅助改为director2
}

vrrp_instance VI_1 {
    state MASTER        #定义主还是备
    interface ens33     #VIP绑定接口
    virtual_router_id 80  #整个集群的调度器一致
    priority 100         #back改为50，优先
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.0.110/24
    }
}
    
[root@backup ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
[root@backup ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id directory2
}

vrrp_instance VI_1 {   #实例名称，两台要保持相同
    state BACKUP    #设置为backup
    interface ens33  #心跳网卡
    nopreempt        #设置到back上面，不抢占资源
    virtual_router_id 80  #虚拟路由编号，主备要保持一致
    priority 50   #辅助改为50
    advert_int 1   #检查间隔，单位秒
    authentication {  秘钥认证
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        10.0.0.110/24
    }
}

5，启动主备keepalived

[root@master ~]# systemctl enable keepalived
[root@master ~]# systemctl start keepalived
[root@master ~]# ip addr
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0:  mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:a3:db:79 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.154/24 brd 10.0.0.255 scope global noprefixroute dynamic eth0
       valid_lft 2872sec preferred_lft 2872sec
    inet 10.0.0.110/24 scope global secondary eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fea3:db79/64 scope link 
       valid_lft forever preferred_lft forever

到此：
可以解决心跳故障keepalived
不能解决Nginx服务故障
4. 扩展对调度器Nginx健康检查（可选）两台都设置
思路：
让Keepalived以一定时间间隔执行一个外部脚本，脚本的功能是当Nginx
失败，则关闭本机的Keepalived
(1) script
[root@nginx-proxy-master ~]# vim 
/etc/keepalived/check_nginx_status.sh
#!/bin/bash												        
usr/bin/curl -I http://localhost &>/dev/null	
if [ $? -ne 0 ];then										    
#	/etc/init.d/keepalived stop
	systemctl stop keepalived
fi															        	
[root@nginx-proxy-master ~]# chmod a+x /etc/keepalived/check_nginx_status.sh

(2). keepalived使用script
! Configuration File for keepalived

global_defs {
   router_id director1
}
vrrp_script check_nginx {     #健康检测模块调用
   script "/etc/keepalived/check_nginx_status.sh"  #指定脚本
   interval 5  #检查频率,秒
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 80
    priority 100
    advert_int 1
    authentication {  
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.246.16/24
    }
    track_script { 引用脚本
        check_nginx
    }
}

注：必须先启动nginx，再启动keepalived

拓展学习：
#!/bin/bash
#+检查nginx进程是否存在
counter=$(ps -C nginx --no-heading|wc -l)    #此行有服务名
if [ "${counter}" = "0" ]; then
#尝试启动一次nginx，停止5秒后再次检测
    service nginx start                                          #启动服务
    sleep 5
    counter=$(ps -C httpd --no-heading|wc -l) #此行有服务名
    if [ "${counter}" = "0" ]; then
#如果启动没成功，就杀掉keepalive触发主备切换
        service keepalived stop
    fi
fi



#!/bin/bash
#haproxy健康检查脚本
a=`ps -C haproxy --no-heading|wc -l`
if [ "$a" -eq "0" ];then
        systemctl start haproxy
        sleep 5
        a=`ps -C haproxy --no-heading|wc -l`
        if [ "$a" -eq "0"  ];then
                systemctl stop keepalived
        fi
fi

四，实战部署：KeepAlived高可用+LVS

KeepAlived在该项目中的功能：
1. 管理IPVS的路由表（包括对RealServer做健康检查）
2. 实现调度器的HA
官方文档：http://www.keepalived.org

Keepalived所执行的外部脚本命令建议使用绝对路径

1，主/备调度器安装软件

[root@lvs-keepalived-master ~]# yum -y install ipvsadm keepalived 
[root@lvs-keepalived-slave ~]# yum -y install ipvsadm keepalived
2. Keepalived
lvs-master
[root@ha-proxy-master ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lvs-keepalived-master    #辅助改为lvs-backup
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33                #VIP绑定接口
    virtual_router_id 80         #VRID 同一组集群，主备一致          
    priority 100            #本节点优先级，辅助改为50
    advert_int 1            #检查间隔，默认为1s
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.246.160/24
    }
}

virtual_server 192.168.246.160 80 {    #LVS配置
	delay_loop 3
	lb_algo rr     #LVS调度算法
	lb_kind DR     #LVS集群模式（路由模式）
	nat_mask 255.255.255.0
	protocol TCP      #健康检查使用的协议
	real_server 192.168.246.162 80 {
		weight 1
		inhibit_on_failure   #当该节点失败时，把权重设置为0，而不是从IPVS中删除
		TCP_CHECK {          #健康检查
			connect_port 80   #检查的端口
			connect_timeout 3  #连接超时的时间
			}
		}
	real_server 192.168.246.163 80 {
		weight 1
		inhibit_on_failure
		TCP_CHECK {
			connect_timeout 3
			connect_port 80
			}
		}
}

[root@lvs-keepalived-slave ~]# vim etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lvs-keepalived-slave
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    nopreempt                    #不抢占资源
    virtual_router_id 80
    priority 50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.246.160/24
    }
}
virtual_server 192.168.246.160 80 {
	delay_loop 3
	lb_algo rr
	lb_kind DR
	nat_mask 255.255.255.0
	persistence_timeout 20
	protocol TCP
	real_server 192.168.246.162 80 {
		weight 1
		inhibit_on_failure
		TCP_CHECK {
			connect_port 80
			connect_timeout 3
			}
		}
	real_server 192.168.246.163 80 {
		weight 1
		inhibit_on_failure
		TCP_CHECK {
			connect_timeout 3
			connect_port 80
			}
		}
}

2，lvs部署

[root@lvs-server ~]# ip addr add dev ens33 192.168.246.160/32 #设置VIP
[root@xingdian-cloud ~]# ipvsadm -S > /etc/sysconfig/ipvsadm
[root@xingdian-cloud ~]# systemctl start ipvsadm
[root@lvs-server ~]# ipvsadm -A -t 192.168.246.160:80 -s rr 
[root@lvs-server ~]# ipvsadm -a -t 192.168.246.160:80 -r 192.168.246.163 -g 
[root@lvs-server ~]# ipvsadm -a -t 192.168.246.160:80 -r 192.168.246.162 -g  
3. 启动KeepAlived（主备均启动）
[root@lvs-keepalived-master ~]# systemctl start keepalived
[root@lvs-keepalived-master ~]# systemctl enable keepalived

[root@lvs-keepalived-master ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.246.160:80 rr persistent 20
  -> 192.168.246.162:80           Route   1      0          0         
  -> 192.168.246.163:80           Route   0      0          0
[root@lvs-keepalived-master ~]# ipvsadm -Ln --stats
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port               Conns   InPkts  OutPkts  InBytes OutBytes
  -> RemoteAddress:Port
TCP  192.168.222.160:80                  0        0        0        0        0
  -> 192.168.222.128:80                  0        0        0        0        0
  -> 192.168.222.130:80                  0        0        0        0        0
[root@lvs-keepalived-master ~]# ipvsadm -Ln --rate
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port                 CPS    InPPS   OutPPS    InBPS   OutBPS
  -> RemoteAddress:Port
TCP  192.168.222.160:80                  0        0        0        0        0
  -> 192.168.222.128:80                  0        0        0        0        0
  -> 192.168.222.130:80                  0        0        0        0        0

3，所有RS配置(nginx1,nginx2)

配置好网站服务器，测试所有RS
[root@test-nginx1 ~]# yum install -y nginx
[root@test-nginx2 ~]# yum install -y nginx
[root@test-nginx1 ~]# ip addr add dev lo 192.168.246.160/32
[root@test-nginx1 ~]# echo "net.ipv4.conf.all.arp_ignore = 1" >> /etc/sysctl.conf
[root@test-nginx1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
[root@test-nginx1 ~]# sysctl -p
[root@test-nginx1 ~]# echo "web1..." >> /usr/share/nginx/html/index.html
[root@test-nginx1 ~]# systemctl start nginx

LB集群 HA高可用集群
所有分发器和Real Server都正常

五，实战部署keepalived高可用+mysql主从复制

项目环境：
VIP 192.168.246.100
mysql1 192.168.246.162      keepalived-master
mysql2 192.168.246.163      keepalived-salve

一、mysql 主主同步        （不使用共享存储，数据保存本地存储）
二、安装keepalived 
三、keepalived 主备配置文件
四、mysql状态检测脚本/root/bin/keepalived_check_mysql.sh
五、测试及诊断
注 keepalived之间使用vrrp组播方式通信使用的IP地址是224.0.0.18

1、mysql主从复制参考链接

https://blog.csdn.net/chenshuai199533/article/details/124743540

2、安装keepalived—两台机器都操作

[root@mysql-keepalived-master ~]# yum install -y keepalived

3，keepalived 主备配置文件

192.168.246.162 master配置
[root@mysql-keepalived-master ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
[root@mysql-keepalived-master ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id directory1
}
vrrp_script check_run {
   script "/etc/keepalived/keepalived_chech_mysql.sh"
   interval 5
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 89
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.246.18/24
    }
    track_script {
        check_run
    }
}

4，slave 192.168.246.163 配置

[root@mysql-keepalived-slave ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
[root@mysql-keepalived-slave ~]# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id directory2
}
vrrp_script check_run {
   script "/etc/keepalived/keepalived_check_mysql.sh"
   interval 5
}

vrrp_instance VI_1 {
    state BACKUP
    nopreempt
    interface ens33
    virtual_router_id 89
    priority 50
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.246.18/24
    }
    track_script {
        check_run
    }
}

5，mysql状态检测脚本/root/keepalived_check_mysql.sh（两台MySQL同样的脚本）

简单使用：
[root@mysql-keepalived-master ~]# vim /etc/keepalived/keepalived_chech_mysql.sh
#!/bin/bash
usr/bin/mysql -uroot -p'QianFeng@2019!' -e "show status" &>/dev/null 
if [ $? -ne 0 ] ;then 
#	service keepalived stop
	systemctl stop keepalived
fi
[root@mysql-keepalived-master ~]# chmod +x /etc/keepalived/keepalived_chech_mysql.sh

两边均启动keepalived
[root@mysql-keepalived-master ~]# systemctl start keepalived
[root@mysql-keepalived-master ~]# systemctl enable keepalived

六，mysql主从复制脚本（拓展干货）

#!/bin/bash
systemctl stop mysqld mariadb &>/dev/null
yum -y remove `rpm -qa |grep mariadb` 2>/dev/null
yum -y remove `rpm -qa |grep myaql` 2>/dev/null
rm -rf /etc/my.cnf  /var/lib/mysql  /var/log/mysql*
userdel -r mysql &>/dev/null
ping -c1 www.baidu.com &>/dev/null
if [ $? -eq 0 ];then
	echo "network is successful"
	yum -y install wget &>/dev/null
	if [ $? -eq 0 ];then
		wget https://dev.mysql.com/get/mysql80-community-release-el7-6.noarch.rpm
		rpm -ivh /root/mysql80-community-release-el7-6.noarch.rpm
		yum clean all
		yum makecache fast
	else
		echo "wget下载失败,请检查网络！！！"
		exit
	fi
else
	echo "network is failed"
	exit
fi

yum -y install yum-utils
yum-config-manager  --enable mysql57-community
yum-config-manager  --disable mysql80-community
sed -ri 's/gpgcheck=1/gpgcheck=0/' /etc/yum.repos.d/mysql-community.repo
yum -y install mysql mysql-server
systemctl start mysqld
ss -lntp |grep 3306
if [ $? -eq 0 ];then
	echo "mysql启动成功"
else
	echo "mysql启动失败"
fi
read -p "请输入您的初始化密码：" pass
a=`cat /var/log/mysqld.log |grep root@localhost |awk '{print $NF}'`
mysqladmin -uroot -p$a password  $pass
mysql -uroot -pQianfeng@123 -e 'show databases' >/dev/null
if [ $? -eq 0 ];then
	echo "mysql新密码设置成功"
else
	echo "mysql新密码设置失败"
fi
m=/etc/my.cnf
cat >> $m </dev/null
if [ $? -eq 0 ];then
	echo "$name用户授权已经创建"
else
	echo "$name用户授权创建失败！！！"

#从服务器部署
b=/etc/my.cnf
cat >> $b << eof
log-bin
server-id=1
gtid_mode = on
enforce_gtid_consistency=1
relay_log_recovery        = on
master-info-repository=TABLE
relay-log-info-repository=TABLE
eof
systemctl restart mysqld
read -p "请输入主服务器的IP地址：" ipaddr
read -p "请输入授权的用户名称：" name1
read -p "请输入授权的密码：" mima
c="change master to  master_host='$ipaddr',master_user='$name1',master_password='$mima',master_auto_position=1;"
mysql -uroot -pQianfeng@123 -e '$c'
mysql -uroot -pQianfeng@123 -e 'start $name1'
mysql -uroot -pQianfeng@123 -e 'show $name1 statusG'

Linux高级实战专题篇：keepalived高可用（keepalived+LVS集群部署；keepalived+mysql集群部署；mysql主从复制自动化脚本）

Linux相关栏目本月热门文章