k8s1.24 FAQ

k8s1.24版本关于runc的问题

问题报错：
May 12 13:48:30 k8snode2 containerd:
time=“2022-05-12T13:48:30.848749359+08:00” level=error msg=“RunPodSandbox for &PodSandboxMetadata{Name:kube-flannel-ds-8lvs6,Uid:1c0ac5b8-74eb-49b2-842c-bcc00355ddff,Namespace:kube-system,Attempt:0,} failed,
error” error=“failed to create containerd task: failed to create shim task: OCI runtime create failed: unable to retrieve OCI runtime error (open
/run/containerd/io.containerd.runtime.v2.task/k8s.io/029eabb43894ea21e4ac75a43a4f144031ed8946fe748006e8748d1d65843092/log.json: no such file or directory): runc did not terminate successfully: exit status 127: unknown”

解决方法

官网下载 runc 执行文件替换linux原有的 /user/local/sbin/runc, 赋予执行权限 chmod +x /user/local/sbin/runc

- containerd配置文件需要修改的地方

/etc/containerd/config.toml 

[plugins."io.containerd.grpc.v1.cri"]
    sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.6"
    .....
    [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
      SystemdCgroup = true
	......
    [plugins."io.containerd.grpc.v1.cri".registry]
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
                endpoint = ["https://registry-1.docker.io"]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."myharbor.io"]
                endpoint = ["http://myharbor.io"]
      [plugins."io.containerd.grpc.v1.cri".registry.configs]
        [plugins."io.containerd.grpc.v1.cri".registry.configs."myharbor.io".tls]
                insecure_skip_verify = true
        [plugins."io.containerd.grpc.v1.cri".registry.configs."myharbor.io".auth]
      [plugins."io.containerd.grpc.v1.cri".registry.headers]

- cni插件未初始化网络错误，解决办法

mkdir -p /etc/cni/net.d/
cat >/etc/cni/net.d/10-flannel.conflist </run/flannel/subnet.env < 
- 出现kubelet匿名还是集群角色无法认证的问题 
May 12 15:58:22 k8snode1 kubelet: W0512 15:58:22.313065    3950 reflector.go:324] vendor/k8s.io/client-go/informers/factory.go:134: failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
May 12 15:58:22 k8snode1 kubelet: E0512 15:58:22.313093    3950 reflector.go:138] vendor/k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.Service: failed to list *v1.Service: services is forbidden: User "system:anonymous" cannot list resource "services" in API group "" at the cluster scope
May 12 15:58:22 k8snode1 kubelet: E0512 15:58:22.361994    3950 kubelet.go:2419] "Error getting node" err="node "k8snode1" not found"
 
解决办法 
node节点关闭开启的kubelet
# systemctl stop kubelet

master节点
# kubectl delete clusterrolebinding kubelet-bootstrap
clusterrolebinding.rbac.authorization.k8s.io "kubelet-bootstrap" deleted

# kubectl create clusterrolebinding kubelet-bootstrap --clusterrole=system:node-bootstrapper --user=kubelet-bootstrap
clusterrolebinding.rbac.authorization.k8s.io/kubelet-bootstrap created

#node节点开启kubelet
systemctl start kubelet

master节点
# kubectl get csr
 
- 问题:kube-proxy匿名授权 
May 12 19:12:32 k8snode1 kube-proxy: E0512 19:12:32.438837   34504 reflector.go:138] vendor/k8s.io/client-go/informers/factory.go:134: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: endpointslices.discovery.k8s.io is forbidden: User "system:anonymous" cannot list resource "endpointslices" in API group "discovery.k8s.io" at the cluster scope
 
解决方法 
kubectl config set-cluster kubernetes 
  --certificate-authority=/data/k8s/ssl/ca.pem 
  --embed-certs=true 
  --server="https://192.168.8.203:6443" 
  --kubeconfig=kube-proxy.kubeconfig

kubectl config set-credentials kube-proxy 
  --client-certificate=/data/k8s/ssl/kube-proxy.pem 
  --client-key=/data/k8s/ssl/kube-proxy-key.pem 
  --embed-certs=true 
  --kubeconfig=kube-proxy.kubeconfig

kubectl config set-context default 
  --cluster=kubernetes 
  --user=kube-proxy 
  --kubeconfig=kube-proxy.kubeconfig

kubectl config use-context default --kubeconfig=kube-proxy.kubeconfig

systemctl restart kube-proxy

 
- 问题：k8s拉取私有镜像仓库失败处理方法 
～/.docker/config.json
让k8s集群具有拉取私有仓库的权限
kubectl create secret generic harbor-registry --from-file=.dockerconfigjson=/root/.docker/config.json --type=kubernetes.io/dockerconfigjson
 
问题：failed to set bridge addr: could not add IP address to “cni0”: permission denied" 
报错信息：
May 14 14:35:27 k8snode4 kubelet: E0514 14:35:27.058422    5161 pod_workers.go:951] "Error syncing pod, skipping" err="failed to "CreatePodSandbox" for "eip-nfs-nfsshare-556ccffc6d-jk8rk_kube-system(7c2fcb44-7df3-4804-a5bb-f5f7dc60b2b6)" with CreatePodSandboxError: "Failed to create sandbox for pod \"eip-nfs-nfsshare-556ccffc6d-jk8rk_kube-system(7c2fcb44-7df3-4804-a5bb-f5f7dc60b2b6)\": rpc error: code = Unknown desc = failed to setup network for sandbox \"24743a13eb6ff7bf972c0f7a25bb6d5860ccafc1eed49b07d6ed6131edaa793d\": plugin type=\"bridge\" failed (add): failed to set bridge addr: could not add IP address to \"cni0\": permission denied"" pod="kube-system/eip-nfs-nfsshare-556ccffc6d-jk8rk" podUID=7c2fcb44-7df3-4804-a5bb-f5f7dc60b2b6
 
处理方法 
导致这个问题是由于cni0网段地址不一致导致网络加入没有权限
mkdir /usr/share/oci-umount/oci-umount.d -p
mkdir /run/flannel/
cat >/run/flannel/subnet.env <

k8s1.24 FAQ

Linux相关栏目本月热门文章