k8s！pod容器与镜像管理和k8s私有仓库harbor搭建

• 前言
• • 1.1：pod的容器分类与镜像拉取策略
  • 1.2：k8s的harbor私有仓库部署

• pod在k8s中是：

1、最小部署单页
2、一组容器的集合
3、一个pod中的容器共享网络命名空间
4、pod是短暂的

• pod的容器分类：

1、infrastructure container：基础容器

• 维护整个pod网络空间：可以在node节点操作查看容器的网络
  [root@node01 ~]# cat /opt/k8s/cfg/kubelet

KUBELET_OPTS="--logtostderr=true 
--v=4 
--hostname-override=192.168.233.132 
--kubeconfig=/opt/k8s/cfg/kubelet.kubeconfig 
--bootstrap-kubeconfig=/opt/k8s/cfg/bootstrap.kubeconfig 
--config=/opt/k8s/cfg/kubelet.config 
--cert-dir=/opt/k8s/ssl 
--pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0"	'//是基础容器'

2.initcontainers：初始化容器

• 先于业务容器开始执行，原先pod中容器是并行开启，现在进行了改进
• 无论容器写在初始化容器前还是写在初始化容器后，最先执行的都是初始化容器。只有初始化容器执行成功后才可以启动容器。
• 初始化容器的应用场景一般是多容器，例如：mysql和业务分开两个容器。将业务设为初始化容器，并检查mysql是否启动，若mysql启动，则业务容器启动；否则业务容器等待mysql启动。

3、container：业务容器

• 业务容器就是我们创建的pod资源内的容器服务，业务容器也叫APP容器，并行启动

• 镜像拉取策略（image PullPolicy）

1、ifnotpresent：默认值，镜像在宿主机上不存在时会拉取
2、always：每次创建pod都会重新拉取一次镜像
3、never：pod永远不会主动拉取这个镜像

• 查看镜像拉取策略（master节点查看）

[root@master ~]# kubectl get pod
NAME                        READY   STATUS    RESTARTS   AGE
nginx-dbddb74b8-5s6h7       1/1     Running   1          10d
nginx-test-d55b94fd-9zmdj   1/1     Running   0          27h
nginx-test-d55b94fd-b8lkl   1/1     Running   0          27h
nginx-test-d55b94fd-w4c5k   1/1     Running   0          27h
[root@master ~]# kubectl edit deploy/nginx

• 尝试编辑一个pod并指定拉去策略

[root@master ~]# cd test/
[root@master test]# ls
nginx-service-test.yaml  nginx-test02.yaml
nginx-test01.yaml        nginx-test.yaml
[root@master test]# cat > pod1-test.yaml < apiVersion: v1
> kind: Pod
> metadata:
>     name: mypod
> spec:
>     containers:
>       - name: nginx
>         image: nginx:1.14
>         imagePullPolicy: Always
> EOF
[root@master test]# kubectl create -f pod1-test.yaml 	'//如果需要更新容器，需要删除原先的容器：kubectl delete -f pod1-test.yaml，修改yaml文件后使用apply命令重新部署：kubectl apply -f pod1-test.yaml '
pod/mypod created
[root@master test]# kubectl get pod 
NAME                        READY   STATUS    RESTARTS   AGE
mypod                       1/1     Running   0          6m
nginx-dbddb74b8-5s6h7       1/1     Running   1          10d
nginx-test-d55b94fd-9zmdj   1/1     Running   0          27h
nginx-test-d55b94fd-b8lkl   1/1     Running   0          27h
nginx-test-d55b94fd-w4c5k   1/1     Running   0          27h

• 查看容器详细信息：kubectl describe pod 名称

[root@master test]# kubectl describe pod mypod
Name:               mypod
Namespace:          default
Priority:           0
PriorityClassName:  
Node:               192.168.233.132/192.168.233.132	'//资源被创建在这个ip的node节点上'
Start Time:         Mon, 11 May 2020 19:27:58 +0800
Labels:             
Annotations:        
Status:             Running
IP:                 172.17.26.5	'//可以查看到ip'
...省略信息

• 可以在相应node节点访问容器

[root@node01 ~]# curl -I 172.17.26.5	'//可以查看到相应的信息'
HTTP/1.1 200 OK
Server: nginx/1.14.2
Date: Mon, 11 May 2020 11:35:54 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 04 Dec 2018 14:44:49 GMT
Connection: keep-alive
ETag: "5c0692e1-264"
Accept-Ranges: bytes

开局优化，修改主机名（harbor），关闭防火墙，上传docker-compose和harbor的软件包（操作简单，不在赘述），私有仓库的IP地址为：192.168.233.134

• docker和docker-compose安装

[root@harbor harbor]# yum -y install yum-utils device-mapper-persistent-data lvm2	'//安装碧瑶软件'
[root@harbor harbor]# yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo	'//设置阿里云镜像'
[root@harbor harbor]# yum -y install docker-ce	'//直接安装社区版'
[root@harbor harbor]# service docker start	'//启动服务'
Redirecting to /bin/systemctl start docker.service
[root@harbor harbor]# docker version
[root@harbor harbor]# mkdir -p /etc/docker
[root@harbor harbor]# tee /etc/docker/daemon.json <<-'EOF'
> {
>   "registry-mirrors": ["https://yu1vx79j.mirror.aliyuncs.com"]
> }
> EOF	'//镜像加速'
{
  "registry-mirrors": ["https://yu1vx79j.mirror.aliyuncs.com"]
}
[root@harbor harbor]# systemctl daemon-reload	'//重载进程'
[root@harbor harbor]# systemctl restart docker
[root@harbor ~]# rz -E
rz waiting to receive.
[root@harbor ~]# ls
anaconda-ks.cfg  docker-compose  harbor-offline-installer-v1.2.2.tgz
[root@harbor ~]# mv docker-compose  /usr/local/bin/
[root@harbor ~]# chmod +x /usr/local/bin/docker-compose 
[root@harbor ~]# docker-compose -v
docker-compose version 1.21.1, build 5a3f1a3

• 安装harbor

[root@harbor ~]# tar zxf harbor-offline-installer-v1.2.2.tgz -C /usr/local/	'//解压到指定目录'
[root@harbor ~]# cd /usr/local/harbor/
[root@harbor harbor]# ls
common                     harbor_1_1_0_template  LICENSE
docker-compose.clair.yml   harbor.cfg             NOTICE
docker-compose.notary.yml  harbor.v1.2.2.tar.gz   prepare
docker-compose.yml         install.sh             upgrade
[root@harbor harbor]# vim harbor.cfg 	'//修改配置文件'
hostname = 192.168.233.134	'//修改为监听本地地址，不可以使用localhost或者127。0.0.1'
[root@harbor harbor]# sh install.sh 

• web网站登录测试

• 所有node节点修改daemon-json文件，指定harbor仓库地址，修改完文件后记得重启Docker

[root@node01 ~]# vim /etc/docker/daemon.json 
{
  "registry-mirrors": ["https://yu1vx79j.mirror.aliyuncs.com"],'//注意这里有个逗号'
  "insecure-registries":["192.168.233.134"]
}
[root@node01 ~]# systemctl daemon-reload
[root@node01 ~]# systemctl restart docker

• 所有node节点都登录harbor仓库(在使用harbor仓库下载镜像创建资源的时候，需要保证node节点处于登陆的状态)

[root@node01 ~]# docker login 192.168.233.134
Username: admin
Password: 
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
[root@node01 ~]# 

• 下载一个Tomcat镜像
• 名称空间

root@master test]# kubectl get namespace
NAME          STATUS   AGE
default       Active   12d
kube-public   Active   12d
kube-system   Active   12d

• 指定node节点从私有仓库下载
  1、查看node节点登录harbor的凭据（所有node节点的凭据是一样的）

[root@node01 ~]# cat .docker/config.json |base64 -w 0
ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjIzMy4xMzQiOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2U0dGeVltOXlNVEl6TkRVPSIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTkuMDMuOCAobGludXgpIgoJfQp9[root@node01 ~]# 

2、master节点创建secret资源

[root@master test]# cat > registry-pull-secret.yaml < apiVersion: v1
> kind: Secret
> metadata:    
>   name: registry-pull-secret
> data:
>   .dockerconfigjson: ewoJImF1dGhzIjogewoJCSIxOTIuMTY4LjIzMy4xMzQiOiB7CgkJCSJhdXRoIjogIllXUnRhVzQ2U0dGeVltOXlNVEl6TkRVPSIKCQl9Cgl9LAoJIkh0dHBIZWFkZXJzIjogewoJCSJVc2VyLUFnZW50IjogIkRvY2tlci1DbGllbnQvMTkuMDMuOCAobGludXgpIgoJfQp9
> type: kubernetes.io/dockerconfigjson
> EOF
[root@master test]# kubectl create -f registry-pull-secret.yaml 	'//创建secret资源'
secret/registry-pull-secret created
[root@master test]# kubectl get secret	'//查看secret资源'
NAME                   TYPE                                  DATA   AGE
default-token-x8jtv    kubernetes.io/service-account-token   3      12d
registry-pull-secret   kubernetes.io/dockerconfigjson        1      3s

3、node节点下载一个nginx镜像并上传到harbor仓库

[root@node01 ~]# docker pull nginx
[root@node01 ~]# docker tag nginx 192.168.233.134/project-test/nginx
[root@node01 ~]# docker push 192.168.233.134/project-test/nginx

4、master节点创建一个yaml文件并将镜像下载地址修改为harbor

[root@master test]# cat > nginx-deploy.yaml < apiVersion: extensions/v1beta1
> kind: Deployment
> metadata:
>   name: my-nginx
> spec:
>   replicas: 2
>   template:
>     metadata:
>       labels:
>         app: my-nginx
>     spec:
>       imagePullSecrets: 	'//镜像安全'
>       - name: registry-pull-secret 
>       containers:
>       - name: my-nginx
>         image: 192.168.233.134/project-test/nginx   	'//'指定私有仓库镜像
>         ports:
>         - containerPort: 80
> ---
> apiVersion: v1
> kind: Service
> metadata:
>   name: my-nginx
> spec:
>   type: NodePort
>   ports:
>   - port: 80
>     targetPort: 80
>     nodePort: 30001
>   selector:
>     app: my-nginx
> EOF
[root@master test]# kubectl create -f nginx-deploy.yaml 
deployment.extensions/my-nginx created
service/my-nginx created
[root@master test]# kubectl get pod
NAME                        READY   STATUS    RESTARTS   AGE
my-nginx-69b8899fd6-g6lhs   1/1     Running   0          5s
my-nginx-69b8899fd6-glh6w   1/1     Running   0          5s
mypod                       1/1     Running   1          154m
nginx-dbddb74b8-5s6h7       1/1     Running   2          10d
nginx-test-d55b94fd-9zmdj   1/1     Running   1          30h
nginx-test-d55b94fd-b8lkl   1/1     Running   1          30h
nginx-test-d55b94fd-w4c5k   1/1     Running   1          30h

• 此时查看镜像仓库发现镜像被下载了两次 ，这是正确的

• 如果遇到处于Terminating状态的无法删除的容器可以强制删除

[root@master test]# kubectl get pods
NAME                              READY   STATUS        RESTARTS   AGE

my-nginx-57667b9d9-nklvj         1/1     Terminating   0          10h

my-nginx-57667b9d9-wllnp         1/1     Terminating   0          10h

'//这种情况下可以使用强制删除命令'
[root@master test]# kubectl delete pod my-nginx-57667b9d9-nklvj  --force --grace-period=0 -n default

'//使用kubectl get ns，查看命名空间'
[root@master test]# kubectl get ns
NAME          STATUS   AGE
default       Active   12d
kube-public   Active   12d
kube-system   Active   12d