keepalived高可用

Keepalived是HA cluster的一种实现方案，其以vrrp协议为实现基础，在N台具有相同功能的路由器或服务器中选出一个master或多个backup，这些路由器或服务器会共用一个对外提供服务的VIP。在此HA cluster集群中，master主机会发送组播来跟backup主机进行通信确认双方是否存活。当backup主机在指定的时间内没有收到vrrp组播报文则认为master主机宕机了，此时就会在所有的backup主机中根据vrrp协议来选举出新的master主机来保证服务的高可用性。
keepalived主要有三个模块，分别是ipvs wrapper、checkers和vrrp stack。ipvs wrapper模块为keepalived的核心，负责主进程的启动、维护以及全局配置文件的加载和解析。checkers负责健康检查，包括常见的各种检查方式。vrrp stack模块是来实现VRRP协议的。

keepalived 有三个重要的功能，分别是：
管理LVS负载均衡软件
实现LVS集群节点的健康检查
作为系统网络服务的高可用性（failover）

Keepalived高可用五福之间故障转移，是通过VRRP来实现的。在Keepalived服务工作时，主Master节点会不断地向节点发送（多播的方式）心跳消息，用来告诉备Backup节点自己还活着。当主节点发生故障时，无法给备节点发送心跳消息，如果别节点无法继续监测来自主节点的心跳。就会调用自身的接管系统，接管主节点的ip资源和服务。当节点恢复的时候，备节点又会释放主节点故障时自身接管的ip资源和服务，恢复到原来的备用角色。

环境说明centos8和centos8

keepalived安装 master 和 slave

//关闭防火墙与SELINUX
systemctl stop firewalld
systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
setenforce 0
 sed -ri 's/^(SELINUX=).*/1disabled/g' /etc/selinux/config
 //安装基础包
dnf -y install epel-release vim wget
dnf -y -y install keepalived
//查看安装生成文件
rpm -ql keepalived
/etc/keepalived  //配置文件
/etc/keepalived/keepalived.conf
/etc/sysconfig/keepalived //主配置文件
/usr/bin/genhash
/usr/lib/.build-id
/usr/lib/.build-id/0a
/usr/lib/.build-id/0a/410997e11c666114ca6d785e58ff0cc248744e
/usr/lib/.build-id/6f
/usr/lib/.build-id/6f/ba0d6bad6cb5ff7b074e703849ed93bebf4a0f
/usr/lib/systemd/system/keepalived.service //服务控制文件

安装httpd master 和 slave

dnf -y install httpd
systemctl enable --now httpd
Created symlink /etc/systemd/system/multi-user.target.wants/httpd.service → /usr/lib/systemd/system/httpd.service.
//web区别
[root@master ~]# echo web1 > /var/www/html/index.html

master 和 slave web页面

配置主keepalived

[root@master keepalived]# cat keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb01
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.143.250
    }
}

virtual_server 192.168.143.250 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.143.101 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.143.102 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[root@master keepalived]# systemctl enable --now keepalived
Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /usr/lib/systemd/system/keepalived.service.

配置备keepalived

[root@slave keepalived]# cat keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb02
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.143.250
    }
}

virtual_server 192.168.143.250 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.143.101 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.143.102 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[root@slave keepalived]# systemctl enable --now keepalived.service 
Created symlink /etc/systemd/system/multi-user.target.wants/keepalived.service → /usr/lib/systemd/system/keepalived.service.

查看VIP在哪里

[root@master ~]# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33:  mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:39:f6:1c brd ff:ff:ff:ff:ff:ff
    inet 192.168.143.101/24 brd 192.168.143.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.143.250/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe39:f61c/64 scope link 
       valid_lft forever preferred_lft forever

[root@slave keepalived]# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33:  mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:07:05:66 brd ff:ff:ff:ff:ff:ff
    inet 192.168.143.102/24 brd 192.168.143.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe07:566/64 scope link 
       valid_lft forever preferred_lft forever

修改内核参数，开启监听VIP功能 master 和 slave
此步可做可不做，该功能可用于仅监听VIP的时候

echo 'net.ipv4.ip_nonlocal_bind = 1' >>/etc/sysctl.conf
sysctl -p
net.ipv4.ip_nonlocal_bind = 1

让keepalived监控httpd负载均衡机

//在master上编写脚本
[root@master]# mkdir /scripts
[root@master]# cd /scripts
[root@master scripts]# cat check_apache.sh 
#!/bin/bash
httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep 'bhttpdb'|wc -l)
if [ $httpd_status -lt 1 ];then
    systemctl stop keepalived
fi
[root@master scripts]# chmod +x check_apache.sh
[root@master scripts]# cat notify.sh
#! /bin/bash
VIP=$2
sendmail (){
        subject="${VIP}'s server keepalived state is translate"
        content="`date +'%F %T'`: `hostname`'s state change to master"
        echo $content | mail -s "$subject" 1720615473@qq.com
}
case "$1" in
  master)
        httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep 'bhttpdb'|wc -l)
        if [ $httpd_status -lt 1 ];then
            systemctl start httpd
        fi
        sendmail
  ;;
  backup)
        httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep 'bhttpdb'|wc -l)
        if [ $httpd_status -gt 0 ];then
            systemctl stop httpd
        fi
  ;;
  *)
        echo "Usage:$0 master|backup VIP"
  ;;
esac
[root@master scripts]# chmod +x notify.sh
//在slave上编写脚本
[root@slave ~]# mkdir /scripts
[root@slave ~]# cd /scripts/
[root@slave scripts]# cat notify.sh
#! /bin/bash
VIP=$2
sendmail (){
        subject="${VIP}'s server keepalived state is translate"
        content="`date +'%F %T'`: `hostname`'s state change to master"
        echo $content | mail -s "$subject" 2327633411@qq.com
}
case "$1" in
  master)
        httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep 'bhttpdb'|wc -l)
        if [ $httpd_status -lt 1 ];then
            systemctl start httpd
        fi
        sendmail
  ;;
  backup)
        httpd_status=$(ps -ef|grep -Ev "grep|$0"|grep 'bhttpdb'|wc -l)
        if [ $httpd_status -gt 0 ];then
            systemctl stop httpd
        fi
  ;;
  *)
        echo "Usage:$0 master|backup VIP"
  ;;
esac
[root@slave scripts]# chmod +x notify.sh

配置keepalived加入监控脚本的配置

//配置主keepalived
[root@master ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb01
}

vrrp_script apache_check {
    script "/scripts/check_apache.sh"
    interval 1
    weight -20
}

vrrp_instance VI_1 {
    state MASTER
    interface ens33
    virtual_router_id 51
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.143.250
    }
    track_script {
        httpd_check
    }
    notify_master "/scripts/notify.sh master 192.168.143.250"
    notify_backup "/scripts/notify.sh backup 192.168.143.250"
}

virtual_server 192.168.143.250 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.143.101 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.143.102 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[root@master ~]# systemctl restart keepalived.service
//配置备keepalived
[root@slave ~]# cat /etc/keepalived/keepalived.conf
! Configuration File for keepalived

global_defs {
   router_id lb02
}

vrrp_instance VI_1 {
    state BACKUP
    interface ens33
    virtual_router_id 51
    priority 90
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.143.250
    }
    notify_master "/scripts/notify.sh master 192.168.143.250"
    notify_backup "/scripts/notify.sh backup 192.168.143.250"
}

virtual_server 192.168.143.250 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    persistence_timeout 50
    protocol TCP

    real_server 192.168.143.101 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }

    real_server 192.168.143.102 80 {
        weight 1
        TCP_CHECK {
            connect_port 80
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
[root@slave ~]# systemctl restart keepalived.service

web页面测试

当主服务器keepalive服务停止或宕机vip访问自动到备用服务器

[root@master ~]# systemctl stop keepalived.service 
[root@master ~]# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33:  mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:39:f6:1c brd ff:ff:ff:ff:ff:ff
    inet 192.168.143.101/24 brd 192.168.143.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe39:f61c/64 scope link 
       valid_lft forever preferred_lft forever
[root@master ~]# 
[root@slave ~]# ip a
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33:  mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:07:05:66 brd ff:ff:ff:ff:ff:ff
    inet 192.168.143.102/24 brd 192.168.143.255 scope global noprefixroute ens33
       valid_lft forever preferred_lft forever
    inet 192.168.143.250/32 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe07:566/64 scope link 
       valid_lft forever preferred_lft forever
[root@slave ~]#