microk8s默认启用了高可用插件ha-cluster,单机安装时需要禁用此插件,单机安装时使用的网络是flannel。
解决方案 可用版本ubuntu@microk8s:~$ snap info microk8s name: microk8s summary: Kubernetes for workstations and appliances publisher: Canonical✓ store-url: https://snapcraft.io/microk8s contact: https://github.com/ubuntu/microk8s license: Apache-2.0 description: | MicroK8s is a small, fast, secure, single node Kubernetes that installs on just about any Linux box. Use it for offline development, prototyping, testing, or use it on a VM as a small, cheap, reliable k8s for CI/CD. It's also a great k8s for appliances - develop your IoT apps for k8s and deploy them to MicroK8s on your boxes. snap-id: EaXqgt1lyCaxKaQCU349mlodBkDCXRcg channels: 1.24/stable: v1.24.0 2022-05-13 (3272) 230MB classic 1.24/candidate: v1.24.0 2022-05-13 (3272) 230MB classic 1.24/beta: v1.24.0 2022-05-13 (3272) 230MB classic 1.24/edge: v1.24.1 2022-05-26 (3349) 230MB classic latest/stable: v1.24.0 2022-05-13 (3272) 230MB classic latest/candidate: v1.24.0 2022-05-13 (3273) 230MB classic latest/beta: v1.24.0 2022-05-13 (3273) 230MB classic latest/edge: v1.24.1 2022-05-27 (3360) 230MB classic dqlite/stable: – dqlite/candidate: – dqlite/beta: – dqlite/edge: v1.16.2 2019-11-07 (1038) 189MB classic 1.23/stable: v1.23.6 2022-04-29 (3204) 218MB classic 1.23/candidate: v1.23.6 2022-04-28 (3204) 218MB classic 1.23/beta: v1.23.6 2022-04-28 (3204) 218MB classic 1.23/edge: v1.23.7 2022-05-26 (3335) 218MB classic 1.22/stable: v1.22.9 2022-05-06 (3203) 193MB classic 1.22/candidate: v1.22.9 2022-04-28 (3203) 193MB classic 1.22/beta: v1.22.9 2022-04-28 (3203) 193MB classic 1.22/edge: v1.22.10 2022-05-26 (3331) 193MB classic 1.21/stable: v1.21.12 2022-05-06 (3202) 191MB classic 1.21/candidate: v1.21.12 2022-04-29 (3202) 191MB classic 1.21/beta: v1.21.12 2022-04-29 (3202) 191MB classic 1.21/edge: v1.21.13 2022-05-25 (3297) 191MB classic 1.20/stable: v1.20.13 2021-12-08 (2760) 221MB classic 1.20/candidate: v1.20.13 2021-12-07 (2760) 221MB classic 1.20/beta: v1.20.13 2021-12-07 (2760) 221MB classic 1.20/edge: v1.20.14 2022-01-11 (2843) 217MB classic 1.19/stable: v1.19.15 2021-09-30 (2530) 216MB classic 1.19/candidate: v1.19.15 2021-09-29 (2530) 216MB classic 1.19/beta: v1.19.15 2021-09-29 (2530) 216MB classic 1.19/edge: v1.19.16 2022-01-07 (2820) 212MB classic 1.18/stable: v1.18.20 2021-07-12 (2271) 198MB classic 1.18/candidate: v1.18.20 2021-07-12 (2271) 198MB classic 1.18/beta: v1.18.20 2021-07-12 (2271) 198MB classic 1.18/edge: v1.18.20 2021-11-03 (2647) 198MB classic 1.17/stable: v1.17.17 2021-01-15 (1916) 177MB classic 1.17/candidate: v1.17.17 2021-01-14 (1916) 177MB classic 1.17/beta: v1.17.17 2021-01-14 (1916) 177MB classic 1.17/edge: v1.17.17 2021-01-13 (1916) 177MB classic 1.16/stable: v1.16.15 2020-09-12 (1671) 179MB classic 1.16/candidate: v1.16.15 2020-09-04 (1671) 179MB classic 1.16/beta: v1.16.15 2020-09-04 (1671) 179MB classic 1.16/edge: v1.16.15 2020-09-02 (1671) 179MB classic 1.15/stable: v1.15.11 2020-03-27 (1301) 171MB classic 1.15/candidate: v1.15.11 2020-03-27 (1301) 171MB classic 1.15/beta: v1.15.11 2020-03-27 (1301) 171MB classic 1.15/edge: v1.15.11 2020-03-26 (1301) 171MB classic 1.14/stable: v1.14.10 2020-01-06 (1120) 217MB classic 1.14/candidate: ↑ 1.14/beta: ↑ 1.14/edge: v1.14.10 2020-03-26 (1303) 217MB classic 1.13/stable: v1.13.6 2019-06-06 (581) 237MB classic 1.13/candidate: ↑ 1.13/beta: ↑ 1.13/edge: ↑ 1.12/stable: v1.12.9 2019-06-06 (612) 259MB classic 1.12/candidate: ↑ 1.12/beta: ↑ 1.12/edge: ↑ 1.11/stable: v1.11.10 2019-05-10 (557) 258MB classic 1.11/candidate: ↑ 1.11/beta: ↑ 1.11/edge: ↑ 1.10/stable: v1.10.13 2019-04-22 (546) 222MB classic 1.10/candidate: ↑ 1.10/beta: ↑ 1.10/edge: ↑ ubuntu@k8s-node-2:~$版本安装
ubuntu@microk8s:~$ sudo snap install microk8s --classic --channel=1.24/stable microk8s (1.24/stable) v1.24.0 from Canonical✓ installed ubuntu@microk8s:~$禁高可用
microk8s当前版本安装时默认启用了ha-cluster[集群高可用]插件,单机安装时需要禁用此插件
ubuntu@microk8s-singleton:~/$ sudo microk8s.disable ha-cluster --force Infer repository core for addon ha-cluster Reverting to a non-HA setup Generating new cluster certificates. Waiting for node to start. Enabling flanneld and etcd HA disabled服务启动
ubuntu@microk8s:~$ sudo microk8s start Started. ubuntu@microk8s:~$命令简化
ubuntu@microk8s:~$ sudo snap alias microk8s.kubectl kubectl
Added:
- microk8s.kubectl as kubectl
ubuntu@microk8s:~$ sudo usermod -a -G microk8s ubuntu
ubuntu@microk8s:~$ sudo chown -f -R ubuntu ~/.kube
ubuntu@microk8s:~$ newgrp microk8s
ubuntu@microk8s:~$ kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.0-2+59bbb3530b6769", GitCommit:"59bbb3530b6769e4935a05ac0e13c9910c79253e", GitTreeState:"clean", BuildDate:"2022-05-13T06:43:45Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24+", GitVersion:"v1.24.0-2+59bbb3530b6769", GitCommit:"59bbb3530b6769e4935a05ac0e13c9910c79253e", GitTreeState:"clean", BuildDate:"2022-05-13T06:41:13Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}
环境准备
环境检查
ubuntu@microk8s:~$ kubectl get nodes NAME STATUS ROLES AGE VERSION microk8s Ready前置存储3m8s v1.24.0-2+59bbb3530b6769 ubuntu@microk8s:~$ ubuntu@microk8s:~$ kubectl get pod -A No resources found ubuntu@microk8s:~$
这一步特别重要,如果不进行这一步骤,则后续所有的操作创建的Pod状态均至ContainerCreating状态无法进行下一步操作,例如
ubuntu@microk8s:/$ kubectl get pods -A NAMESPACE NAME READY STATUS RESTARTS AGE default nginx 0/1 ContainerCreating 0 76s
开启存储插件
ubuntu@microk8s:~$ sudo microk8s.enable storage Infer repository core for addon storage DEPRECIATION WARNING: 'storage' is deprecated and will soon be removed. Please use 'hostpath-storage' instead. Infer repository core for addon hostpath-storage Enabling default storage class. WARNING: Hostpath storage is not suitable for production environments. deployment.apps/hostpath-provisioner created storageclass.storage.k8s.io/microk8s-hostpath created serviceaccount/microk8s-hostpath created clusterrole.rbac.authorization.k8s.io/microk8s-hostpath created clusterrolebinding.rbac.authorization.k8s.io/microk8s-hostpath created Storage will be available soon. ubuntu@microk8s:~$ ubuntu@microk8s:~$ kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system hostpath-provisioner-76f65f69ff-52qnp 0/1 ContainerCreating 0 22s ubuntu@microk8s:~$
长时间等待发现一直处于ContainerCreating状态,通过describe查看原因
ubuntu@microk8s:~$ kubectl describe pod hostpath-provisioner-76f65f69ff-52qnp -n kube-system
Name: hostpath-provisioner-76f65f69ff-52qnp
Namespace: kube-system
Priority: 0
Node: microk8s/192.168.64.19
Start Time: Sun, 29 May 2022 13:40:33 +0800
Labels: k8s-app=hostpath-provisioner
pod-template-hash=76f65f69ff
Annotations:
Status: Pending
IP:
IPs:
Controlled By: ReplicaSet/hostpath-provisioner-76f65f69ff
Containers:
hostpath-provisioner:
Container ID:
Image: cdkbot/hostpath-provisioner:1.2.0
Image ID:
Port:
Host Port:
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment:
NAMESPACE: kube-system (v1:metadata.namespace)
NODE_NAME: (v1:spec.nodeName)
PV_DIR: /var/snap/microk8s/common/default-storage
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-kz7xb (ro)
/var/snap/microk8s/common/default-storage from pv-volume (rw)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
pv-volume:
Type: HostPath (bare host directory volume)
Path: /var/snap/microk8s/common/default-storage
HostPathType:
kube-api-access-kz7xb:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional:
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors:
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 58s default-scheduler Successfully assigned kube-system/hostpath-provisioner-76f65f69ff-52qnp to microk8s
Warning FailedCreatePodSandBox 28s kubelet Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox image "k8s.gcr.io/pause:3.1": failed to pull image "k8s.gcr.io/pause:3.1": failed to pull and unpack image "k8s.gcr.io/pause:3.1": failed to resolve reference "k8s.gcr.io/pause:3.1": failed to do request: Head "https://k8s.gcr.io/v2/pause/manifests/3.1": dial tcp 142.250.157.82:443: i/o timeout
Warning MissingClusterDNS 16s (x2 over 58s) kubelet pod: "hostpath-provisioner-76f65f69ff-52qnp_kube-system(6708c267-cc81-47a8-b6c8-2022103c4288)". kubelet does not have ClusterDNS IP configured and cannot create Pod using "ClusterFirst" policy. Falling back to "Default" policy.
ubuntu@microk8s:~$
由上可以看到是因为k8s.gcr.io/pause:3.1无法下载导致。此时可以通过命令收到导入镜像,安装工具 pullk8s,此工具可以通过 hub.docker.com 的 opsdockerimage 仓库下载k8s所需的 k8s.gcr.io 或 gcr.io 镜像,无需自己翻墙,每天更新一次,包括所有image 的全平台的所有tags。
ubuntu@microk8s:~$ git clone https://github.com/OpsDocker/pullk8s.git Cloning into 'pullk8s'... remote: Enumerating objects: 11, done. remote: Counting objects: 100% (11/11), done. remote: Compressing objects: 100% (10/10), done. remote: Total 11 (delta 1), reused 3 (delta 0), pack-reused 0 Receiving objects: 100% (11/11), 10.49 KiB | 185.00 KiB/s, done. Resolving deltas: 100% (1/1), done. ubuntu@microk8s:~$ cd pullk8s/ ubuntu@microk8s:~/pullk8s$ ls LICENSE README.md pullk8s.sh ubuntu@microk8s:~/pullk8s$ chmod +x pullk8s.sh ubuntu@microk8s:~/pullk8s$ sudo cp pullk8s.sh /usr/local/bin/pullk8s ubuntu@microk8s:~/pullk8s$ sudo pullk8s pull k8s.gcr.io/pause:3.1 --microk8s Pull pause:3.1 ... Pull pause:3.1 ... /usr/local/bin/pullk8s: line 34: docker: command not found /usr/local/bin/pullk8s: line 35: docker: command not found /usr/local/bin/pullk8s: line 36: docker: command not found /usr/local/bin/pullk8s: line 40: docker: command not found ctr: unrecognized image format ubuntu@microk8s:~/pullk8s$ sudo snap install docker docker 20.10.14 from Canonical✓ installed ubuntu@microk8s:~/pullk8s$ sudo pullk8s pull k8s.gcr.io/pause:3.1 --microk8s Pull pause:3.1 ... Pull pause:3.1 ... 3.1: Pulling from opsdockerimage/pause 67ddbfb20a22: Pull complete Digest: sha256:f78411e19d84a252e53bff71a4407a5686c46983a2c2eeed83929b888179acea Status: Downloaded newer image for opsdockerimage/pause:3.1 docker.io/opsdockerimage/pause:3.1 Untagged: opsdockerimage/pause:3.1 Untagged: opsdockerimage/pause@sha256:f78411e19d84a252e53bff71a4407a5686c46983a2c2eeed83929b888179acea unpacking k8s.gcr.io/pause:3.1 (sha256:0968e31df05b727234888883ba43ccaa4ec75566113c75065af5a6124b62d93c)...done ubuntu@microk8s:~/pullk8s$ sudo docker images REPOSITORY TAG IMAGE ID CREATED SIZE k8s.gcr.io/pause 3.1 da86e6ba6ca1 4 years ago 742kB ubuntu@microk8s:~/pullk8s$
pullk8s工具依赖docker 来拉镜像,安装好docker,然后运行 pullk8s check --microk8s检查被屏蔽的 gcr.io 或 k8s.gcr.io 容器名称
再次查看Pod,此时可以看到Pod都在正常创建中,通过-w命令持续观察Pod状态
ubuntu@microk8s:~/pullk8s$ kubectl get pods -A -w NAMESPACE NAME READY STATUS RESTARTS AGE kube-system hostpath-provisioner-76f65f69ff-52qnp 0/1 ContainerCreating 0 6m48s kube-system hostpath-provisioner-76f65f69ff-52qnp 1/1 Running 0 7m基础插件
查看目前支持的插件
ubuntu@microk8s:~$ sudo microk8s status
microk8s is running
high-availability: no
addons:
enabled:
hostpath-storage # (core) Storage class; allocates storage from host directory
storage # (core) Alias to hostpath-storage add-on, deprecated
disabled:
community # (core) The community addons repository
dashboard # (core) The Kubernetes dashboard
dns # (core) CoreDNS
gpu # (core) Automatic enablement of Nvidia CUDA
ha-cluster # (core) Configure high availability on the current node
helm # (core) Helm 2 - the package manager for Kubernetes
helm3 # (core) Helm 3 - Kubernetes package manager
host-access # (core) Allow Pods connecting to Host services smoothly
ingress # (core) Ingress controller for external access
mayastor # (core) OpenEBS MayaStor
metallb # (core) Loadbalancer for your Kubernetes cluster
metrics-server # (core) K8s Metrics Server for API access to service metrics
prometheus # (core) Prometheus operator for monitoring and logging
rbac # (core) Role-Based Access Control for authorisation
registry # (core) Private image registry exposed on localhost:32000
ubuntu@microk8s:~$
开启基础插件
ubuntu@microk8s:~/pullk8s$ sudo microk8s.enable dns rbac metrics-server dashboard Infer repository core for addon dns Infer repository core for addon rbac Infer repository core for addon metrics-server Infer repository core for addon dashboard Enabling DNS Applying manifest serviceaccount/coredns created configmap/coredns created deployment.apps/coredns created service/kube-dns created clusterrole.rbac.authorization.k8s.io/coredns created clusterrolebinding.rbac.authorization.k8s.io/coredns created Restarting kubelet DNS is enabled Enabling RBAC Reconfiguring apiserver RBAC is enabled Enabling Metrics-Server serviceaccount/metrics-server created clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader created clusterrole.rbac.authorization.k8s.io/system:metrics-server created rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader created clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator created clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server created service/metrics-server created deployment.apps/metrics-server created apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io created clusterrolebinding.rbac.authorization.k8s.io/microk8s-admin created Metrics-Server is enabled Enabling Kubernetes Dashboard Infer repository core for addon metrics-server Enabling Metrics-Server serviceaccount/metrics-server unchanged clusterrole.rbac.authorization.k8s.io/system:aggregated-metrics-reader unchanged clusterrole.rbac.authorization.k8s.io/system:metrics-server unchanged rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader unchanged clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator unchanged clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server unchanged service/metrics-server unchanged deployment.apps/metrics-server configured apiservice.apiregistration.k8s.io/v1beta1.metrics.k8s.io unchanged clusterrolebinding.rbac.authorization.k8s.io/microk8s-admin unchanged Metrics-Server is enabled Applying manifest serviceaccount/kubernetes-dashboard created service/kubernetes-dashboard created secret/kubernetes-dashboard-certs created secret/kubernetes-dashboard-csrf created secret/kubernetes-dashboard-key-holder created configmap/kubernetes-dashboard-settings created role.rbac.authorization.k8s.io/kubernetes-dashboard created clusterrole.rbac.authorization.k8s.io/kubernetes-dashboard created rolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created deployment.apps/kubernetes-dashboard created service/dashboard-metrics-scraper created deployment.apps/dashboard-metrics-scraper created If RBAC is not enabled access the dashboard using the default token retrieved with: token=$(microk8s kubectl -n kube-system get secret | grep default-token | cut -d " " -f1) microk8s kubectl -n kube-system describe secret $token In an RBAC enabled setup (microk8s enable RBAC) you need to create a user with restricted permissions as shown in: https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md ubuntu@microk8s:~/pullk
查看状态
NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-66bcf65bb8-jglch 1/1 Running 0 2m49s kube-system dashboard-metrics-scraper-6b6f796c8d-zgs5l 0/1 ContainerCreating 0 7s kube-system hostpath-provisioner-76f65f69ff-52qnp 1/1 Running 1 (2m28s ago) 12m kube-system kubernetes-dashboard-765646474b-6jsmj 0/1 ContainerCreating 0 7s kube-system metrics-server-5f8f64cb86-pp8kl 0/1 ContainerCreating 0 7s ubuntu@microk8s:~/pullk8s$
相同镜像下载原因,下载依赖镜像
ubuntu@microk8s:~/$ sudo pullk8s pull coredns/coredns:1.9.0 ubuntu@microk8s:~/$ sudo pullk8s pull k8s.gcr.io/metrics-server/metrics-server:v0.5.2 ubuntu@microk8s:~/$ sudo pullk8s pull k8s.gcr.io/metrics-server/metrics-server:v0.5.2 --microk8s
再次查看Pod状态
ubuntu@microk8s:~$ kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-66bcf65bb8-jglch 1/1 Running 0 6m33s kube-system dashboard-metrics-scraper-6b6f796c8d-zgs5l 1/1 Running 0 3m51s kube-system hostpath-provisioner-76f65f69ff-52qnp 1/1 Running 1 (6m12s ago) 16m kube-system kubernetes-dashboard-765646474b-6jsmj 1/1 Running 0 3m51s kube-system metrics-server-5f8f64cb86-pp8kl 1/1 Running 0 3m51s ubuntu@microk8s:~$看板访问 网络地址
通过ip命令查看网络地址
ubuntu@microk8s:~$ ip addr 1: lo:mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp0s2: mtu 1500 qdisc fq_codel state UP group default qlen 1000 link/ether b2:74:11:12:94:98 brd ff:ff:ff:ff:ff:ff inet 192.168.64.19/24 metric 100 brd 192.168.64.255 scope global dynamic enp0s2 valid_lft 83490sec preferred_lft 83490sec inet6 fe80::b074:11ff:fe12:9498/64 scope link valid_lft forever preferred_lft forever
可以看到当前机器网络地址: 192.168.64.19
开启代理通过命令microk8s dashboard-proxy开启看板访问
ubuntu@microk8s:~$ microk8s dashboard-proxy Checking if Dashboard is running. Infer repository core for addon dashboard Waiting for Dashboard to come up. Create token for accessing the dashboard secret/microk8s-dashboard-proxy-token created Waiting for secret token (attempt 0) Dashboard will be available at https://127.0.0.1:10443 Use the following token to login: eyJhbGciOiJSUzI1NiIsImtpZCI6ImE5TGNZazZ0RUc1TWZONmlDRG1jVEJaUUR5amRWc29vZ0VQZmRhOVBhcmMifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJtaWNyb2s4cy1kYXNoYm9hcmQtcHJveHktdG9rZW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGVmYXVsdCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImNiNGJhYzhjLWY1MWYtNDhkNS04Y2QzLWFkYTUzNDQxYjI1ZiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTpkZWZhdWx0In0.ckIAEF-SXHW-BT_ZFPb-Vbp8a4KSWpc2nDFAOpaIztZeiazqKVt39JymH8qDl5dBK0u82W2BkWjZQfLYk_3OSK_At9M_ie06DUUXd3sv7liUtoj2GpZfckz4O7jDHHlnV97LPa5vpURrzvdhXI0FuRuPgITt1JhHzM6BuwrNRI2TLZreSlF6Ukqw2_UFGH2x-El9AZERDJcvtdtYGhoFLLA9C5lDmS3dJg0JAfakqciwkoIQMs94h3yAMzjsV1uxizX6ysFwqMcyqZ7Eaa9gKgjkf50PiqdG18NFS0At0XcNMV9Gonot7chikOuHr25k2mIreMpd0T3bBwRmp7CDJQ
打开浏览器访问https://192.168.64.19:10443/
安全访问浏览器访问网址https://192.168.64.19:10443/
此时点击高级,发现无法登录
在当前浏览器该chrome页面上,直接使用键盘输入这11个字符:thisisunsafe 此时发现已经正常进入
此时输入token即可正常登录
ubuntu@microk8s:~$ microk8s dashboard-proxy Checking if Dashboard is running. Infer repository core for addon dashboard Waiting for Dashboard to come up. Create token for accessing the dashboard secret/microk8s-dashboard-proxy-token unchanged Waiting for secret token (attempt 0) Dashboard will be available at https://127.0.0.1:10443 Use the following token to login: eyJhbGciOiJSUzI1NiIsImtpZCI6Imc3RHlPVTRMeTZxclpycmU0THp6V1dDVmp1Yk5iamRDX1dCLXJIUlFzOUUifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJtaWNyb2s4cy1kYXNoYm9hcmQtcHJveHktdG9rZW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGVmYXVsdCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjIyZTg1OGFkLThjMTEtNGQ4MC05ZmQ1LTk1OTRlOWI5NzJmYiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTpkZWZhdWx0In0.ofQpEj-izIyFHEv0Nv9s-XwdqNKorBL8Lfy8IImTLV7y9PVc6PcjPtplEndtEwCJrqsc_NcZg51yN2o8qmi9JPsjq1k05Q2NIDam8-770wTLEppGV_gcfX4jQ1JMFT1Ia4YNOvoCfk0WrvGzP1uO_xH_rS5FAt-pb09u17nWqqwI-vTtKrQuMt6ufMV1UYddgbB8NYe1b70JH08uF0Xhnr0_GFCq8oOqTBF3Jw98D-61zCwi5Qc2fQYUCUdE390Y2ya7uRnL-NaCaP9hSztEEzWwduY6AtR-D5YtItVHBiQJ4kaYhjBZiA3TsZrxeE6EZ9FKHX13IhL6AzYb8gvacA启用RBAC
如果前面开启了rbac插件,则进入会有错误提示
此时需要给kubernetes-dashboard赋予集群管理员权限
ubuntu@microk8s:~$ kubectl get deploy -A
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
kube-system coredns 1/1 1 1 17m
kube-system dashboard-metrics-scraper 1/1 1 1 16m
kube-system hostpath-provisioner 1/1 1 1 27m
kube-system kubernetes-dashboard 1/1 1 1 16m
kube-system metrics-server 1/1 1 1 17m
ubuntu@microk8s:~$
ubuntu@microk8s:~$ kubectl get deploy kubernetes-dashboard -n kube-system -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{},"labels":{"k8s-app":"kubernetes-dashboard"},"name":"kubernetes-dashboard","namespace":"kube-system"},"spec":{"replicas":1,"revisionHistoryLimit":10,"selector":{"matchLabels":{"k8s-app":"kubernetes-dashboard"}},"template":{"metadata":{"labels":{"k8s-app":"kubernetes-dashboard"}},"spec":{"containers":[{"args":["--auto-generate-certificates","--namespace=kube-system"],"image":"kubernetesui/dashboard:v2.3.0","imagePullPolicy":"IfNotPresent","livenessProbe":{"httpGet":{"path":"/","port":8443,"scheme":"HTTPS"},"initialDelaySeconds":30,"timeoutSeconds":30},"name":"kubernetes-dashboard","ports":[{"containerPort":8443,"protocol":"TCP"}],"securityContext":{"allowPrivilegeEscalation":false,"readOnlyRootFilesystem":true,"runAsGroup":2001,"runAsUser":1001},"volumeMounts":[{"mountPath":"/certs","name":"kubernetes-dashboard-certs"},{"mountPath":"/tmp","name":"tmp-volume"}]}],"nodeSelector":{"kubernetes.io/os":"linux"},"serviceAccountName":"kubernetes-dashboard","tolerations":[{"effect":"NoSchedule","key":"node-role.kubernetes.io/master"}],"volumes":[{"name":"kubernetes-dashboard-certs","secret":{"secretName":"kubernetes-dashboard-certs"}},{"emptyDir":{},"name":"tmp-volume"}]}}}}
creationTimestamp: "2022-05-29T05:51:49Z"
generation: 1
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
resourceVersion: "1848"
uid: abf3a423-d78a-44ce-9eb7-e418f51c01c1
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-dashboard
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
k8s-app: kubernetes-dashboard
spec:
containers:
- args:
- --auto-generate-certificates
- --namespace=kube-system
image: kubernetesui/dashboard:v2.3.0
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 3
httpGet:
path: /
port: 8443
scheme: HTTPS
initialDelaySeconds: 30
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 30
name: kubernetes-dashboard
ports:
- containerPort: 8443
protocol: TCP
resources: {}
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsGroup: 2001
runAsUser: 1001
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /certs
name: kubernetes-dashboard-certs
- mountPath: /tmp
name: tmp-volume
dnsPolicy: ClusterFirst
nodeSelector:
kubernetes.io/os: linux
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: kubernetes-dashboard
serviceAccountName: kubernetes-dashboard
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
volumes:
- name: kubernetes-dashboard-certs
secret:
defaultMode: 420
secretName: kubernetes-dashboard-certs
- emptyDir: {}
name: tmp-volume
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2022-05-29T05:53:30Z"
lastUpdateTime: "2022-05-29T05:53:30Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
- lastTransitionTime: "2022-05-29T05:52:56Z"
lastUpdateTime: "2022-05-29T05:53:30Z"
message: ReplicaSet "kubernetes-dashboard-765646474b" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
observedGeneration: 1
readyReplicas: 1
replicas: 1
updatedReplicas: 1
ubuntu@microk8s:~$
找到配置节
namespace: kube-system serviceAccountName: kubernetes-dashboard
查看集群角色列表
ubuntu@microk8s:~$ kubectl get clusterrole NAME CREATED AT admin 2022-05-29T05:50:58Z cluster-admin 2022-05-29T05:50:58Z coredns 2022-05-29T05:50:14Z edit 2022-05-29T05:50:58Z kubernetes-dashboard 2022-05-29T05:51:49Z microk8s-hostpath 2022-05-29T05:40:33Z system:aggregate-to-admin 2022-05-29T05:50:58Z ......................... ......................... .........................
查看集群角色权限
ubuntu@microk8s:~$ kubectl describe clusterrole cluster-admin
Name: cluster-admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
*.* [] [] [*]
[*] [] [*]
ubuntu@microk8s:~$ kubectl describe clusterrole admin
Name: admin
Labels: kubernetes.io/bootstrapping=rbac-defaults
Annotations: rbac.authorization.kubernetes.io/autoupdate: true
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
rolebindings.rbac.authorization.k8s.io [] [] [create delete deletecollection get list patch update watch]
roles.rbac.authorization.k8s.io [] [] [create delete deletecollection get list patch update watch]
configmaps [] [] [create delete deletecollection patch update get list watch]
events [] [] [create delete deletecollection patch update get list watch]
persistentvolumeclaims [] [] [create delete deletecollection patch update get list watch]
.........................
.........................
.........................
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
creationTimestamp: "2022-05-29T05:50:58Z"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: cluster-admin
resourceVersion: "1470"
uid: 61799386-3b02-4aeb-8ee6-db55055864af
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- '*'
- nonResourceURLs:
- '*'
verbs:
- '*'
为kubernetes-dashboard的服务账号进行赋权操作
ubuntu@microk8s:~$ kubectl create clusterrolebinding kubernetes-dashboard-clusterbingding_kube-system_kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-clusterbingding_kube-system_kubernetes-dashboard created
如果加入绑定错误,则使用kubectl delete clusterrolebinding.rbac.authorization.k8s.io/命令中的名字进行删除
此时发现错误仍然存在,此时查看pod日志
ubuntu@microk8s:~$ kubectl get pod -A NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-66bcf65bb8-jglch 1/1 Running 0 74m kube-system dashboard-metrics-scraper-6b6f796c8d-zgs5l 1/1 Running 0 72m kube-system hostpath-provisioner-76f65f69ff-52qnp 1/1 Running 1 (74m ago) 84m kube-system kubernetes-dashboard-765646474b-6jsmj 1/1 Running 0 72m kube-system metrics-server-5f8f64cb86-pp8kl 1/1 Running 0 72m ubuntu@microk8s:~$ kubectl -n kube-system logs kubernetes-dashboard-765646474b-6jsmj 2022/05/29 05:53:30 Starting overwatch ........................... ........................... ........................... 2022/05/29 06:01:42 Non-critical error occurred during resource retrieval: namespaces is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "namespaces" in API group "" at the cluster scope 2022/05/29 06:01:42 Non-critical error occurred during resource retrieval: deployments.apps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "deployments" in API group "apps" in the namespace "default" 2022/05/29 06:01:42 Non-critical error occurred during resource retrieval: pods is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "pods" in API group "" in the namespace "default"
再次进行绑定
ubuntu@microk8s:~$ kubectl create clusterrolebinding kubernetes-dashboard-clusterbingding_kube-system_default --clusterrole=cluster-admin --serviceaccount=kube-system:default clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard-clusterbingding_kube-system_default created
如果加入绑定错误,则使用kubectl delete clusterrolebinding.rbac.authorization.k8s.io/命令中的名字进行删除
此时发现问题解决
禁用RBAC如果前面没有开启rbac插件,则进入是如下页面。
切换工作空间可以查看相对应的监控信息
凭据获取ubuntu@microk8s:/var/snap/microk8s/current/args$ token=$(kubectl -n kube-system get secret | grep default-token | cut -d " " -f1) ubuntu@microk8s:/var/snap/microk8s/current/args$ kubectl -n kube-system describe secret $token ubuntu@microk8s:/var/snap/microk8s/current/args$ kubectl -n kube-system describe secret $token Name: kubernetes-dashboard-certs Namespace: kube-system Labels: k8s-app=kubernetes-dashboard Annotations:简单使用Type: Opaque Data ==== Name: kubernetes-dashboard-csrf Namespace: kube-system Labels: k8s-app=kubernetes-dashboard Annotations: Type: Opaque Data ==== csrf: 256 bytes Name: kubernetes-dashboard-key-holder Namespace: kube-system Labels: Annotations: Type: Opaque Data ==== priv: 1675 bytes pub: 459 bytes Name: microk8s-dashboard-proxy-token Namespace: kube-system Labels: Annotations: kubernetes.io/service-account.name: default kubernetes.io/service-account.uid: 22e858ad-8c11-4d80-9fd5-9594e9b972fb Type: kubernetes.io/service-account-token Data ==== token: eyJhbGciOiJSUzI1NiIsImtpZCI6Imc3RHlPVTRMeTZxclpycmU0THp6V1dDVmp1Yk5iamRDX1dCLXJIUlFzOUUifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJtaWNyb2s4cy1kYXNoYm9hcmQtcHJveHktdG9rZW4iLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGVmYXVsdCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjIyZTg1OGFkLThjMTEtNGQ4MC05ZmQ1LTk1OTRlOWI5NzJmYiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTpkZWZhdWx0In0.ofQpEj-izIyFHEv0Nv9s-XwdqNKorBL8Lfy8IImTLV7y9PVc6PcjPtplEndtEwCJrqsc_NcZg51yN2o8qmi9JPsjq1k05Q2NIDam8-770wTLEppGV_gcfX4jQ1JMFT1Ia4YNOvoCfk0WrvGzP1uO_xH_rS5FAt-pb09u17nWqqwI-vTtKrQuMt6ufMV1UYddgbB8NYe1b70JH08uF0Xhnr0_GFCq8oOqTBF3Jw98D-61zCwi5Qc2fQYUCUdE390Y2ya7uRnL-NaCaP9hSztEEzWwduY6AtR-D5YtItVHBiQJ4kaYhjBZiA3TsZrxeE6EZ9FKHX13IhL6AzYb8gvacA ca.crt: 1123 bytes namespace: 11 bytes ubuntu@microk8s:/var/snap/microk8s/current/args$
ubuntu@microk8s:~$ kubectl run nginx --image=nginx pod/nginx created ubuntu@microk8s:~$ kubectl get pod NAME READY STATUS RESTARTS AGE nginx 0/1 ContainerCreating 0 4s ubuntu@microk8s:~$ kubectl get pod -w NAME READY STATUS RESTARTS AGE nginx 0/1 ContainerCreating 0 8s nginx 1/1 Running 0 25s扩展安装
microk8s 只是最精简的安装,所以只有 api-server, controller-manager, scheduler, kubelet, cni, kube-proxy 被安装运行。额外的服务比如 kube-dns, dashboard 可以通过 microk8s.enable 启动
扩展启用ubuntu@microk8s: microk8s.enable dns dashboard扩展禁用
ubuntu@microk8s: microk8s.disable dns dashboard可用扩展
ubuntu@microk8s:~$ microk8s status
microk8s is running
high-availability: no
addons:
enabled:
dashboard # (core) The Kubernetes dashboard
dns # (core) CoreDNS
hostpath-storage # (core) Storage class; allocates storage from host directory
metrics-server # (core) K8s Metrics Server for API access to service metrics
rbac # (core) Role-Based Access Control for authorisation
storage # (core) Alias to hostpath-storage add-on, deprecated
disabled:
community # (core) The community addons repository
gpu # (core) Automatic enablement of Nvidia CUDA
ha-cluster # (core) Configure high availability on the current node
helm # (core) Helm 2 - the package manager for Kubernetes
helm3 # (core) Helm 3 - Kubernetes package manager
host-access # (core) Allow Pods connecting to Host services smoothly
ingress # (core) Ingress controller for external access
mayastor # (core) OpenEBS MayaStor
metallb # (core) Loadbalancer for your Kubernetes cluster
prometheus # (core) Prometheus operator for monitoring and logging
registry # (core) Private image registry exposed on localhost:32000
ubuntu@microk8s:~$
启用社区
启用社区仓库可以启用更多的功能,比如istio
ubuntu@microk8s:~$ sudo microk8s.enable istio
Addon istio was not found in any repository
To use the community maintained flavor enable the respective repository:
microk8s enable community
ubuntu@microk8s:~$ microk8s enable community
Infer repository core for addon community
Cloning into '/var/snap/microk8s/common/addons/community'...
done.
Community repository is now enabled
ubuntu@microk8s:~$
ubuntu@microk8s:~$ sudo microk8s.enable istio
Infer repository community for addon istio
Enabling Istio
Fetching istioctl version v1.10.3.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 21.3M 100 21.3M 0 0 222k 0 0:01:38 0:01:38 --:--:-- 550k
istio-1.10.3/
Istio扩展
启用istio之前需要事先启用dns,否则会出现错误
ubuntu@microk8s:~$ sudo microk8s.enable istio服务管理 服务重启
ubuntu@microk8s:/var/snap/microk8s/current/args$ microk8s.stop Stopped. ubuntu@microk8s:/var/snap/microk8s/current/args$ microk8s.start Started.服务重置
ubuntu@microk8s: sudo microk8s.reset服务卸载
ubuntu@microk8s: sudo snap remove microk8s服务状态
ubuntu@microk8s:/var/snap/microk8s/current/args$ microk8s.status
microk8s is running
high-availability: no
datastore master nodes: 127.0.0.1:19001
datastore standby nodes: none
addons:
enabled:
dashboard # (core) The Kubernetes dashboard
ha-cluster # (core) Configure high availability on the current node
metrics-server # (core) K8s Metrics Server for API access to service metrics
disabled:
community # (core) The community addons repository
dns # (core) CoreDNS
gpu # (core) Automatic enablement of Nvidia CUDA
helm # (core) Helm 2 - the package manager for Kubernetes
helm3 # (core) Helm 3 - Kubernetes package manager
host-access # (core) Allow Pods connecting to Host services smoothly
hostpath-storage # (core) Storage class; allocates storage from host directory
ingress # (core) Ingress controller for external access
mayastor # (core) OpenEBS MayaStor
metallb # (core) Loadbalancer for your Kubernetes cluster
prometheus # (core) Prometheus operator for monitoring and logging
rbac # (core) Role-Based Access Control for authorisation
registry # (core) Private image registry exposed on localhost:32000
storage # (core) Alias to hostpath-storage add-on, deprecated
ubuntu@microk8s:/var/snap/microk8s/current/args$
镜像加速
初始容器
初始容器也叫根容器,可以通过编辑配置文件/var/snap/microk8s/current/args/containerd-template.toml进行更改
24 # The 'plugins."io.containerd.grpc.v1.cri"' table contains all of the server options. 25 [plugins."io.containerd.grpc.v1.cri"] 26 27 stream_server_address = "127.0.0.1" 28 stream_server_port = "0" 29 enable_selinux = false 30 sandbox_image = "k8s.gcr.io/pause:3.1" 31 stats_collect_period = 10
访问阿里云镜像 搜索pause编辑为如下内容
24 # The 'plugins."io.containerd.grpc.v1.cri"' table contains all of the server options. 25 [plugins."io.containerd.grpc.v1.cri"] 26 27 stream_server_address = "127.0.0.1" 28 stream_server_port = "0" 29 enable_selinux = false 30 sandbox_image = "registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1" 31 stats_collect_period = 10
编辑配置文件/var/snap/microk8s/current/args/kubelet并添加如下参数
--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google_containers/pause:3.1底层容器
k8s-1.20发布之后,不再使用doker作为底层容器运行时,而是默认使用Container Runtime Interface(CRI)。查看运行时环境
ubuntu@microk8s:/var/snap/microk8s/current/args$ microk8s.ctr -v ctr github.com/containerd/containerd v1.5.11 ubuntu@microk8s:/var/snap/microk8s/current/args$
右上可知,当前版本使用的是containerd作为运行时环境。
镜像仓库可以通过配置microk8s内置containerd的registry.mirrors来加速镜像下载,编辑 /var/snap/microk8s/current/args/containerd-template.toml 文件
75 # 'plugins."io.containerd.grpc.v1.cri".registry' contains config related to the registry
76 [plugins."io.containerd.grpc.v1.cri".registry]
77 config_path = "${SNAP_DATA}/args/certs.d"
在 endpoint 追加新的国内 registry.mirrors
75 # 'plugins."io.containerd.grpc.v1.cri".registry' contains config related to the registry
76 [plugins."io.containerd.grpc.v1.cri".registry]
77 config_path = "${SNAP_DATA}/args/certs.d"
78 [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
79 [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
80 endpoint = [
81 "https://b7j3uwrc.mirror.aliyuncs.com",
82 "https://docker.mirrors.ustc.edu.cn",
83 "https://hub-mirror.c.163.com",
84 "https://mirror.ccs.tencentyun.com",
85 ]
此时重启服务并检查状态
ubuntu@microk8s:/var/snap/microk8s/current/args$ microk8s.stop
Stopped.
ubuntu@microk8s:/var/snap/microk8s/current/args$ microk8s.start
Started.
ubuntu@microk8s:/var/snap/microk8s/current/args$ microk8s.status
microk8s is running
high-availability: no
datastore master nodes: 127.0.0.1:19001
datastore standby nodes: none
addons:
enabled:
dashboard # (core) The Kubernetes dashboard
ha-cluster # (core) Configure high availability on the current node
metrics-server # (core) K8s Metrics Server for API access to service metrics
disabled:
community # (core) The community addons repository
dns # (core) CoreDNS
gpu # (core) Automatic enablement of Nvidia CUDA
helm # (core) Helm 2 - the package manager for Kubernetes
helm3 # (core) Helm 3 - Kubernetes package manager
host-access # (core) Allow Pods connecting to Host services smoothly
hostpath-storage # (core) Storage class; allocates storage from host directory
ingress # (core) Ingress controller for external access
mayastor # (core) OpenEBS MayaStor
metallb # (core) Loadbalancer for your Kubernetes cluster
prometheus # (core) Prometheus operator for monitoring and logging
rbac # (core) Role-Based Access Control for authorisation
registry # (core) Private image registry exposed on localhost:32000
storage # (core) Alias to hostpath-storage add-on, deprecated
ubuntu@microk8s:/var/snap/microk8s/current/args$
文档参考
https://blog.csdn.net/sinat_38453878/article/details/123345268
https://blog.csdn.net/rpfgg/article/details/116763900
文档参考https://cloud.tencent.com/developer/article/2000534



