3台机器，3台master
在 192.168.254.162 上执行
hostnamectl set-hostname k8s-master01
在 192.168.254.163 上执行
hostnamectl set-hostname k8s-master02
在 192.168.254.164 上执行
hostnamectl set-hostname k8s-master03
在三台服务器上执行
cat >> /etc/hosts <2.关闭防火墙、dnsmasq、postfix 在三台服务器上执行
systemctl disable firewalld --now systemctl disable dnsmasq --now systemctl disable postfix --now3.关闭selinux在三台服务器上执行
sed -i 's/^SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config setenforce 04.关闭swap在三台服务器上执行
swapoff -a && sysctl -w vm.swappiness=0 sed -i '/swap/s/^/#/' /etc/fstab cat /etc/fstab注释 /etc/fstab 中的swap那一行
5.安装时间同步工具在三台服务器上执行
yum install -y chrony systemctl enable chronyd --now systemctl status chronyd6.修改时区在三台服务器上执行
ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime echo 'Asia/Shanghai' > /etc/timezone7.修改ulimit在三台服务器上执行
cat >> /etc/security/limits.conf </etc/security/limits.d/20-nproc.conf < 8.导入sshkey到各个node 在三台服务器上执行
ssh-keygen for i in k8s-master01 k8s-master02 k8s-master03 do ssh-copy-id $i done9.配置阿里云源在三台服务器上执行
yum -y install wget mkdir -p /etc/yum.repos.d/bak mv /etc/yum.repos.d/*.repo /etc/yum.repos.d/bak wget http://mirrors.aliyun.com/repo/Centos-7.repo -O /etc/yum.repos.d/Centos-7.repo yum clean all yum makecache10.导入阿里云的k8s镜像源在三台服务器上执行
cat > /etc/yum.repos.d/kubernetes.repo <11.升级系统 在三台服务器上执行
yum update -y --exclude=kernel*12.升级内核在三台服务器上执行
#安装内核包 rpm --import https://www.elrepo.org/RPM-GPG-KEY-elrepo.org rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-5.el7.elrepo.noarch.rpm yum --enablerepo=elrepo-kernel -y install kernel-ml#设置默认内核 awk -F' '$1=="menuentry " {print i++ " : " $2}' /etc/grub2.cfg0 : CentOS Linux (5.15.6-1.el7.elrepo.x86_64) 7 (Core)
1 : CentOS Linux (3.10.0-957.el7.x86_64) 7 (Core)
2 : CentOS Linux (0-rescue-e00de991102846e7b51670dd8e6de5e5) 7 (Core)# CentOS Linux (5.15.3-1.el7.elrepo.x86_64) 7 (Core) 这个是awk命令查出来的 grub2-set-default 'CentOS Linux (5.15.6-1.el7.elrepo.x86_64) 7 (Core)'13.安装ipvs在三台服务器上执行
yum install ipvsadm ipset sysstat conntrack libseccomp -y modprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack cat > /etc/modules-load.d/ipvs.conf <14.内核参数调整 在三台服务器上执行
cat > /etc/sysctl.d/k8s.conf <15.重启系统 在三台服务器上执行
reboot基础组件安装 1.挂载磁盘在三台服务器上执行
fdisk -l|grep dev|grep -v mapperDisk /dev/vda: 64.4 GB, 64424509440 bytes, 125829120 sectors
/dev/vda1 * 2048 2099199 1048576 83 Linux
/dev/vda2 2099200 83886079 40893440 8e Linux LVM
Disk /dev/vdb: 1610.6 GB, 1610612736000 bytes, 3145728000 sectorsvgcreate datavg /dev/vdb vgdisplay datavg|grep PEPE Size 4.00 MiB
Total PE 383999
Alloc PE / Size 0 / 0
Free PE / Size 383999 / 1.46 TiBlvcreate -l 383999 datavg -n datalv mkfs.xfs /dev/datavg/datalv cat >> /etc/fstab <2.docker安装 在三台服务器上执行
yum install -y yum-utils yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo yum -y install docker-ce docker-ce-cli containerd.io mkdir -p /etc/docker cat > /etc/docker/daemon.json <3.kubeadm 安装 在三台服务器上执行
yum -y install kubeadm-1.21.2 kubelet-1.21.2 kubectl-1.21.24.设置kubelet随机启动在三台服务器上执行
systemctl daemon-reload systemctl enable --now kubelet echo "1">/proc/sys/net/bridge/bridge-nf-call-iptables echo "1">/proc/sys/net/bridge/bridge-nf-call-ip6tables高可用组件安装 1. 安装haproxy、keepalived在三台服务器上执行
yum install haproxy keepalived -y2. 配置haproxy并启动在三台服务器上执行
cat > /etc/haproxy/haproxy.cfg <3. 设置keepalived配置文件 master01 cat > /etc/keepalived/keepalived.conf <master02 cat > /etc/keepalived/keepalived.conf <master03 cat > /etc/keepalived/keepalived.conf <添加健康检查脚本 在三台服务器上执行
cat > /etc/keepalived/check_apiserver.sh <4. 启动 systemctl enable --now keepalived systemctl status keepalived ip a|grep -w inetkubeadm初始化集群 1. 初始化master01在192.168.254.162上执行
#下载镜像 kubeadm config images pull --image-repository daocloud.io/daocloud --kubernetes-version v1.21.0 #初始化集群 kubeadm init --control-plane-endpoint "192.168.254.165:16443" --image-repository daocloud.io/daocloud --kubernetes-version v1.21.0 --pod-network-cidr=10.244.0.0/16 --upload-certs如果失败，按错误提示解决问题，reset后重新初始化
kubeadm reset如果成功会出现如下的信息：
2. 按照提示执行初始化命令Your Kubernetes control-plane has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown ( i d − u ) : (id -u): (id−u):(id -g) $HOME/.kube/configAlternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run “kubectl apply -f [podnetwork].yaml” with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/You can now join any number of the control-plane node running the following command on each as root:
kubeadm join 192.168.254.165:16443 --token xfrknv.dj2jum2zd3xvgsvm
–discovery-token-ca-cert-hash sha256:a39195c78f8f9a6657755ea6ac746007da558739c5320c0c5e1a2fb281a9e8f0
–control-plane --certificate-key 74723f6534e1d4bdf96a3eb342457e191db7c85fe9a1c80f8e725a3195b43097Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
“kubeadm init phase upload-certs --upload-certs” to reload certs afterward.Then you can join any number of worker nodes by running the following on each as root:
kubeadm join 192.168.254.165:16443 --token xfrknv.dj2jum2zd3xvgsvm
–discovery-token-ca-cert-hash sha256:a39195c78f8f9a6657755ea6ac746007da558739c5320c0c5e1a2fb281a9e8f0在192.168.254.162上执行
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config3. 加入集群并执行初始化命令在192.168.254.163和192.168.254.164上执行
kubeadm join 192.168.254.165:16443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:64cb2b4c7f61ed8e577293e60c0b34547fcaf71100a8090e3ac8619314aa2e6f --control-plane --certificate-key 561e70a3ee47ab1c5668af0b7a039ae1c996878ced8251ae85ce2e9a8fa46e48 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config4. 安装flannel在192.168.254.162上执行
#详见 https://github.com/flannel-io/flannel kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml5. 安装metrics-server在192.168.254.162上执行
#详见 https://github.com/kubernetes-sigs/metrics-server#requirements wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml #修改components.yaml文件 sed -i '/image:/i - --kubelet-insecure-tls' components.yaml sed -i '/image:/c image: bitnami/metrics-server:0.6.1' components.yaml kubectl apply -f components.yaml6. 安装kuboard在192.168.254.162上执行
#详见 https://kuboard.cn/install/v3/install-in-k8s.html kubectl apply -f https://addons.kuboard.cn/kuboard/kuboard-v3.yaml7. token过期处理在192.168.254.162上执行
kubeadm token create --print-join-commandkubeadm join 192.168.254.165:16443 --token 05knyn.cohcy5ahayzru6kw --discovery-token-ca-cert-hash sha256:b38bca487dadaf2cdc89bf33aa0011819eb41fac030e42c733dffbe55440900b
#slave节点加入的命令为： kubeadm join 192.168.254.165:16443 --token 05knyn.cohcy5ahayzru6kw --discovery-token-ca-cert-hash sha256:b38bca487dadaf2cdc89bf33aa0011819eb41fac030e42c733dffbe55440900b在192.168.254.162上执行
kubeadm init phase upload-certs --upload-certsI1124 12:08:13.125648 5736 version.go:254] remote version is much newer: v1.22.4; falling back to: stable-1.21
[upload-certs] Storing the certificates in Secret “kubeadm-certs” in the “kube-system” Namespace
[upload-certs] Using certificate key:
967a05512e3db41793746dc1c1a3e39ccf3e52b6b626dd8b2e32d62909c0d25f#master节点加入的命令为： kubeadm join 192.168.254.165:16443 --token 05knyn.cohcy5ahayzru6kw --discovery-token-ca-cert-hash sha256:b38bca487dadaf2cdc89bf33aa0011819eb41fac030e42c733dffbe55440900b --control-plane --certificate-key 967a05512e3db41793746dc1c1a3e39ccf3e52b6b626dd8b2e32d62909c0d25f
