安装好K8S，没搭建K8S的可查看：kubeadm方式搭建K8S集群、RKE搭建K8S【强烈推荐】
需要知道自己的内网IP，为了后面NFS和PV的配置#通过命令查看内网IP，我的网段为172.31.0.0/24 kubectl get nodes -o wide #================================================================================================ NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP 172.31.0.110 Ready controlplane,etcd,worker 7d22h v1.22.9 172.31.0.1101.创建NFS存储172.31.0.115 Ready worker 7d22h v1.22.9 172.31.0.115 172.31.0.82 Ready worker 7d22h v1.22.9 172.31.0.82 #================================================================================================
创建NFS存储主要是为了给Redis提供稳定的后端存储，当Redis的Pod重启或迁移后，依然能获得原先的数据。这里，我们先要创建NFS，然后通过使用PV为Redis挂载一个远程的NFS路径。
yum -y install nfs-utils rpcbind（nfs-utils主包提供文件系统、rpcbind提供rpc协议）
新增/etc/exports文件，用于设置需要共享的路径：
#编辑/etc/exports文件，添加内容 vi /etc/exports #================================================================================================ /usr/local/kubernetes/redis/pv1 172.31.0.0/24(rw,sync,no_root_squash) /usr/local/kubernetes/redis/pv2 172.31.0.0/24(rw,sync,no_root_squash) /usr/local/kubernetes/redis/pv3 172.31.0.0/24(rw,sync,no_root_squash) /usr/local/kubernetes/redis/pv4 172.31.0.0/24(rw,sync,no_root_squash) /usr/local/kubernetes/redis/pv5 172.31.0.0/24(rw,sync,no_root_squash) /usr/local/kubernetes/redis/pv6 172.31.0.0/24(rw,sync,no_root_squash) #================================================================================================
创建对应文件夹并授权
mkdir -p /usr/local/kubernetes/redis/pv{1..6}
chmod 777 /usr/local/kubernetes/redis/pv{1..6}
启动服务
#启动nfs和rpcbind服务 systemctl enable nfs systemctl enable rpcbind systemctl start nfs systemctl start rpcbind exportfs -v #================================================================================================ /usr/local/kubernetes/redis/pv1 172.31.0.0/24(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) /usr/local/kubernetes/redis/pv2 172.31.0.0/24(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) /usr/local/kubernetes/redis/pv3 172.31.0.0/24(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) /usr/local/kubernetes/redis/pv4 172.31.0.0/24(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) /usr/local/kubernetes/redis/pv5 172.31.0.0/24(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) /usr/local/kubernetes/redis/pv6 172.31.0.0/24(sync,wdelay,hide,no_subtree_check,sec=sys,rw,secure,no_root_squash,no_all_squash) #================================================================================================
所有节点安装nfs客户端
#在所有节点进行安装操作 yum -y install nfs-utils #验证显示被客户端挂载的目录名 showmount -e 172.31.0.1102.创建PV【server为master节点的内网IP】
vi pv.yaml
#================================================================================================
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv1
spec:
storageClassName: redis
capacity:
storage: 2Gi
accessModes:
- ReadWriteMany
nfs:
server: 172.31.0.110
path: "/usr/local/kubernetes/redis/pv1"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-vp2
spec:
storageClassName: redis
capacity:
storage: 2Gi
accessModes:
- ReadWriteMany
nfs:
server: 172.31.0.110
path: "/usr/local/kubernetes/redis/pv2"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv3
spec:
storageClassName: redis
capacity:
storage: 2Gi
accessModes:
- ReadWriteMany
nfs:
server: 172.31.0.110
path: "/usr/local/kubernetes/redis/pv3"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv4
spec:
storageClassName: redis
capacity:
storage: 2Gi
accessModes:
- ReadWriteMany
nfs:
server: 172.31.0.110
path: "/usr/local/kubernetes/redis/pv4"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv5
spec:
storageClassName: redis
capacity:
storage: 2Gi
accessModes:
- ReadWriteMany
nfs:
server: 172.31.0.110
path: "/usr/local/kubernetes/redis/pv5"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: nfs-pv6
spec:
storageClassName: redis
capacity:
storage: 2Gi
accessModes:
- ReadWriteMany
nfs:
server: 172.31.0.110
path: "/usr/local/kubernetes/redis/pv6"
#================================================================================================
#执行创建pv
kubectl create -f pv.yaml
kubectl get pv
3.创建Configmap
#创建redis文件夹并切换到文件夹下 mkdir /etc/redis && cd /etc/redis #创建redis.conf文件 vi redis.conf #================================================================================================ protected-mode no appendonly yes cluster-enabled yes cluster-config-file /var/lib/redis/nodes.conf cluster-node-timeout 5000 dir /var/lib/redis port 6379 #================================================================================================ #将reids.conf转为configMap文件 kubectl create configmap redis-conf --from-file=redis.conf #查看创建的configmap: kubectl describe cm redis-conf #================================================================================================ Name: redis-conf Namespace: default Labels:4.创建headless serviceAnnotations: Data ==== redis.conf: ---- protected-mode noprotected-mode no appendonly yes cluster-enabled yes cluster-config-file /var/lib/redis/nodes.conf cluster-node-timeout 5000 dir /var/lib/redis port 6379 Events: #================================================================================================
Headless service是StatefulSet实现稳定网络标识的基础
#创建headless-service.yaml 文件
vi headless-service.yaml
#================================================================================================
apiVersion: v1
kind: Service
metadata:
name: redis-service
labels:
app: redis
spec:
ports:
- name: redis-port
port: 6379
clusterIP: None
selector:
app: redis
appCluster: redis-cluster
#================================================================================================
#创建
kubectl create -f headless-service.yaml
#查看
kubectl get svc
#可以看到，服务名称为redis-service，其CLUSTER-IP为None，表示这是一个“无头”服务
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.43.0.1 443/TCP 7d19h
redis-service ClusterIP None 6379/TCP 5s
5.创建Redis StatefulSet
#创建redis.yaml
vi redis.yaml
#================================================================================================
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: redis-app
spec:
serviceName: "redis-service"
replicas: 6
selector:
matchLabels:
app: redis
appCluster: redis-cluster
template:
metadata:
labels:
app: redis
appCluster: redis-cluster
spec:
containers:
- name: redis
image: "redis:6.0.6"
command: ["/bin/bash", "-ce", "tail -f /dev/null"]
command: ["redis-server"]
args:
- "/etc/redis/redis.conf"
- "--protected-mode"
- "no"
ports:
- name: redis
containerPort: 6379
protocol: "TCP"
- name: cluster
containerPort: 16379
protocol: "TCP"
volumeMounts:
- name: "redis-conf"
mountPath: "/etc/redis"
- name: "redis-data"
mountPath: "/var/lib/redis"
volumes:
- name: "redis-conf"
configMap:
name: "redis-conf"
items:
- key: "redis.conf"
path: "redis.conf"
volumeClaimTemplates:
- metadata:
name: redis-data
spec:
accessModes: [ "ReadWriteMany" ]
storageClassName: "redis"
resources:
requests:
storage: 1Gi
#================================================================================================
#创建新资源
kubectl create -f redis.yaml
#查看redis pods
kubectl get pods -o wide
#================================================================================================
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS
redis-app-0 1/1 Running 0 4m57s 10.42.1.31 172.31.0.115
redis-app-1 1/1 Running 0 4m56s 10.42.2.23 172.31.0.82
redis-app-2 1/1 Running 0 4m54s 10.42.0.21 172.31.0.110
redis-app-3 1/1 Running 0 4m51s 10.42.1.32 172.31.0.115
redis-app-4 1/1 Running 0 4m50s 10.42.2.24 172.31.0.82
redis-app-5 1/1 Running 0 4m48s 10.42.0.22 172.31.0.110
#================================================================================================
#查看PV
kubectl get pv
#================================================================================================
NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS
nfs-pv1 2Gi RWX Retain Bound default/redis-data-redis-app-2 redis
nfs-pv3 2Gi RWX Retain Bound default/redis-data-redis-app-0 redis
nfs-pv4 2Gi RWX Retain Bound default/redis-data-redis-app-4 redis
nfs-pv5 2Gi RWX Retain Bound default/redis-data-redis-app-5 redis
nfs-pv6 2Gi RWX Retain Bound default/redis-data-redis-app-1 redis
nfs-vp2 2Gi RWX Retain Bound default/redis-data-redis-app-3 redis
#================================================================================================
6.初始化Redis集群
初始化集群采用redis-trib这个工具，直接yum安装redis-trib这个软件，执行初始化
yum -y install redis-trib redis-trib create --replicas 1 10.42.1.31:6379 10.42.2.23:6379 10.42.0.21:6379 10.42.1.32:6379 10.42.2.24:6379 10.42.0.22:6379
或docker 容器构建
docker pull inem0o/redis-trib docker run --rm -ti inem0o/redis-trib create --replicas 1 10.42.1.31:6379 10.42.2.23:6379 10.42.0.21:6379 10.42.1.32:6379 10.42.2.24:6379 10.42.0.22:6379
构建成功
#===============================================进入容器=========================================
kubectl exec -it redis-app-0 -- redis-cli -c
#===============================================查看集群=========================================
127.0.0.1:6379> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:82
cluster_stats_messages_pong_sent:80
cluster_stats_messages_sent:162
cluster_stats_messages_ping_received:75
cluster_stats_messages_pong_received:82
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:162
#===============================================查看角色=========================================
127.0.0.1:6379> role
1) "master"
2) (integer) 84
3) 1) 1) "10.42.1.36"
2) "6379"
3) "84"
#===============================================查看节点=========================================
127.0.0.1:6379> cluster nodes
fc63373625ab65f480c5343ffca4f7d650af8ee6 10.42.0.25:6379@16379 master - 0 1653709741112 3 connected 10923-16383
c444518310f7be61ace5895605f24fe23fd4625b 10.42.2.28:6379@16379 slave 635a262b8e0b2e340acf68262a73870099f40be6 0 1653709740000 2 connected
e57fff1006a56eea193179f0ffcf91ee91e8b800 10.42.1.36:6379@16379 slave 1fe5682e876057cc1e2346b4434008abb7ed2ab8 0 1653709740512 1 connected
635a262b8e0b2e340acf68262a73870099f40be6 10.42.2.27:6379@16379 master - 0 1653709741000 2 connected 5461-10922
289c8aae1b7093b3ae58e7be9bc6bc75abd8dcef 10.42.0.26:6379@16379 slave fc63373625ab65f480c5343ffca4f7d650af8ee6 0 1653709740612 3 connected
1fe5682e876057cc1e2346b4434008abb7ed2ab8 10.42.1.35:6379@16379 myself,master - 0 1653709739000 1 connected 0-5460
7、创建用于访问Service
vi redis-access-service.yaml
apiVersion: v1
kind: Service
metadata:
name: redis-access-service
labels:
app: redis
spec:
ports:
- name: redis-port
protocol: "TCP"
port: 6379
targetPort: 6379
nodePort: 6379
selector:
app: redis
appCluster: redis-cluster
type: NodePort
kubectl get svc redis-access-service -o wide
注意：这里我采用的是NodePort直接开放外部端口使用，想采用默认的ClusterId，然后通过Ingress开放端口进行访问也行。
