云原生应用也就是面向“云”而设计的应用，在使用云原生技术后，开发者无需考虑底层的技术实现，可以充分发挥云平台的弹性和分布式优势，实现快速部署、按需伸缩、不停机交付等。
今天我们通过把wordpress 和Mysql部署到IBM公有云的托管K8S上，来实践如何部署一个应用到k8s的环境，从而为我们后面的云原生开发打下实践的基础。
本实验基于k8s 示例：使用 Persistent Volumes 部署 WordPress 和 MySQL 与 IBM IKS官方文档.
本实验与k8s官方教程的主要有以下区别：
- 本实验iaas层基于ibm cloud vpc gen2，k8s官方教程基于本地的minikube。
- 本实验持久化基于 IBM vpc block storage, 而k8s官网用的是本地storage作为持久化。
相同点：
- 本实验同样使用 dynamic PVC 自动创建相对应的PV。
- 由于本实验的iaas层基于ibm vpc gen2， 所以需要先创建对应的vpc实例, 可以使用ibm cloud gui console 快速创建vpc。
- Push image 到 IBM Registry
使用以下命令
ibmcloud cr login # 登录 IBM Registry。 ibmcloud cr namespace-add spark_test # 添加一个namespace ibmcloud cr namespace-list -v # list namespace docker pull wordpress:4.8-apache # 从dockerhub下载image 到本地 docker pull mysql:5.6 docker tag wordpress:4.8-apache us.icr.io/spark_test/wordpress:4.8-apache # tag imag docker tag mysql:5.6 us.icr.io/spark_test/mysql:5.6 docker push wordpress:4.8-apache us.icr.io/spark_test/wordpress:4.8-apache # 推送image到仓库对应的namespace下 docker push mysql:5.6 us.icr.io/spark_test/mysql:5.6
远端仓库的格式为 {zone}.us.icr.io,当登录Registry 会提示远端仓库的信息，这里只有做相应的替换就可以了。
查看image是否上传成功
localhost:IKS spark$ ibmcloud cr image-list Listing images... Repository Tag Digest Namespace Created Size Security status us.icr.io/spark_test/busybox 1.28.4 74f634b1bc1b spark_test 2 years ago 723 kB Unsupported OS us.icr.io/spark_test/mysql 5.6 574b626bb891 spark_test 1 week ago 103 MB 4 Issues us.icr.io/spark_test/wordpress 4.8-apache b40c224a95cd spark_test 3 years ago 170 MB 366 Issues
- 登录 ibm cloud
ibmcloud login -a cloud.ibm.com -r us-south -g friday-you --apikey xxxxx
- 使用IBM console 创建 cluster
cluster 创建好后，进入cluster overview页面获取 access 信息。 - 设置cluster 环境
localhost:IKS spark$ ibmcloud ks cluster config --cluster bvn8bkad0kms4uq0eij0 localhost:IKS spark$ kubectl config current-context spark-iks-us-south-1-bx2.4x16/bvn8bkad0kms4uq0eij0
- 查看cluster nodes，确认环境已经可用
localhost:IKS spark$ kubectl get nodes NAME STATUS ROLES AGE VERSION 10.240.0.4 Ready3. 创建Secret6h43m v1.18.13+IKS
https://kubernetes.io/zh/docs/concepts/configuration/secret/
#config/mysql-pass.yaml apiVersion: v1 kind: Secret metadata: name: mysql-pass type: kubernetes.io/basic-auth stringdata: password: t0p-Secret #这个mysql db root password localhost:IKS spark$ kubectl apply -f config/mysql-pass.yaml secret/mysql-pass created localhost:IKS spark$ kubectl get secret mysql-pass NAME TYPE DATA AGE mysql-pass kubernetes.io/basic-auth 1 65s localhost:IKS spark$ kubectl describe secret mysql-pass Name: mysql-pass Namespace: default Labels:4. （可选） 创建自己的 storageclassesAnnotations: Type: kubernetes.io/basic-auth Data ==== password: 10 bytes localhost:IKS spark$
free plan 默认没有定义好的storage classes，需要自己定义。
localhost:IKS spark$ kubectl apply -f config/storage-classes.yaml storageclass.storage.k8s.io/my-ibmc-vpc-block-retain-5iops-tier created localhost:IKS spark$ kubectl get storageclasses NAME PROVISIonER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE ibmc-vpc-block-10iops-tier (default) vpc.block.csi.ibm.io Delete Immediate false 41m ibmc-vpc-block-5iops-tier vpc.block.csi.ibm.io Delete Immediate false 41m ibmc-vpc-block-custom vpc.block.csi.ibm.io Delete Immediate false 41m ibmc-vpc-block-general-purpose vpc.block.csi.ibm.io Delete Immediate false 41m ibmc-vpc-block-metro-10iops-tier vpc.block.csi.ibm.io Delete WaitForFirstConsumer false 41m ibmc-vpc-block-metro-5iops-tier vpc.block.csi.ibm.io Delete WaitForFirstConsumer false 41m ibmc-vpc-block-metro-custom vpc.block.csi.ibm.io Delete WaitForFirstConsumer false 41m ibmc-vpc-block-metro-general-purpose vpc.block.csi.ibm.io Delete WaitForFirstConsumer false 41m ibmc-vpc-block-metro-retain-10iops-tier vpc.block.csi.ibm.io Retain WaitForFirstConsumer false 41m ibmc-vpc-block-metro-retain-5iops-tier vpc.block.csi.ibm.io Retain WaitForFirstConsumer false 41m ibmc-vpc-block-metro-retain-custom vpc.block.csi.ibm.io Retain WaitForFirstConsumer false 41m ibmc-vpc-block-metro-retain-general-purpose vpc.block.csi.ibm.io Retain WaitForFirstConsumer false 41m ibmc-vpc-block-retain-10iops-tier vpc.block.csi.ibm.io Retain Immediate false 41m ibmc-vpc-block-retain-5iops-tier vpc.block.csi.ibm.io Retain Immediate false 41m ibmc-vpc-block-retain-custom vpc.block.csi.ibm.io Retain Immediate false 41m ibmc-vpc-block-retain-general-purpose vpc.block.csi.ibm.io Retain Immediate false 41m my-ibmc-vpc-block-retain-5iops-tier vpc.block.csi.ibm.io Retain Immediate false 12s5. Create dynamic PVC
动态pvc，通过指定storageClassName来调用storage plugin，由vpc gen2的storage 组件自动分配符合pvc要求的pv资源，并绑定pv到pvc下。
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: my-pvc
spec:
accessModes:
- ReadWriteonce #读写一次，只能mount到一个pod, 所有的block storage只支持 `ReadWriteOnce` 这一种模式, NFS 支持读写多次（不同的pod mount到一个NFS ）
resources:
requests:
storage: 10Gi # 10G大小
storageClassName: ibmc-vpc-block-retain-5iops-tier # 由于我们指定了pvc 的storage classes， 所以pv会自动创建。 如果需要手动创建pv，这个字段设置为空字符串。
localhost:IKS spark$ kubectl apply -f config/my_pvc.yaml
persistentvolumeclaim/my-pvc created
localhost:IKS spark$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
my-pvc Bound pvc-c09494f1-bfbc-45d7-a20b-1469819644df 10Gi RWO ibmc-vpc-block-retain-5iops-tier 100s
localhost:IKS spark$
localhost:IKS spark$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
my-pvc Pending ibmc-vpc-block-retain-5iops-tier 8s
localhost:IKS spark$ kubectl describe pvc my-pvc
Name: my-pvc
Namespace: default
StorageClass: ibmc-vpc-block-retain-5iops-tier
Status: Bound
Volume: pvc-c09494f1-bfbc-45d7-a20b-1469819644df
Labels:
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
volume.beta.kubernetes.io/storage-provisioner: vpc.block.csi.ibm.io
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 10Gi
Access Modes: RWO
VolumeMode: Filesystem
Mounted By:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Provisioning 37s vpc.block.csi.ibm.io_ibm-vpc-block-csi-controller-0_538d6bfe-8958-4c9a-af17-0bcc35bf0e0b External provisioner is provisioning volume for claim "default/my-pvc"
Normal ExternalProvisioning 24s (x2 over 37s) persistentvolume-controller waiting for a volume to be created, either by external provisioner "vpc.block.csi.ibm.io" or manually created by system administrator
Normal ProvisioningSucceeded 21s vpc.block.csi.ibm.io_ibm-vpc-block-csi-controller-0_538d6bfe-8958-4c9a-af17-0bcc35bf0e0b Successfully provisioned volume pvc-c09494f1-bfbc-45d7-a20b-1469819644df
检查PV 已经被绑定
localhost:IKS spark$ kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-c09494f1-bfbc-45d7-a20b-1469819644df 10Gi RWO Retain Bound default/my-pvc ibmc-vpc-block-retain-5iops-tier 40s localhost:IKS spark$6. 创建mysql
apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
name: wordpress-mysql
labels:
app: wordpress
spec:
selector:
matchLabels:
app: wordpress
tier: mysql
strategy:
type: Recreate
template:
metadata:
labels:
app: wordpress
tier: mysql
spec:
containers:
- image: us.icr.io/spark_test/mysql:5.6
name: mysql
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
secretKeyRef: # 引用secret password
name: mysql-pass
key: password
ports:
- containerPort: 3306
name: mysql
volumeMounts:
- name: mysql-persistent-storage
mountPath: /var/lib/mysql
volumes:
- name: mysql-persistent-storage
persistentVolumeClaim:
claimName: my-pvc # 通过指定pvc， mount到指定的pv
7. 创建mysql service
由于mysql是内部使用并且只有一个instance，这里使用headless的方式创建mysql
headless 不会为服务分配cluster-ip，同时dns会为每个pod instance分配对应的dns域名
apiVersion: v1
kind: Service
metadata:
name: wordpress-mysql
labels:
app: wordpress
spec:
ports:
- port: 3306
selector:
app: wordpress
tier: mysql
clusterIP: None # 设置cluster ip 为None，表明为headless service
localhost:single_instance_wordpress spark$ kubectl exec -it my-deployment-6d78f74d89-kw2dt -- nslookup wordpress-mysql Server: 172.21.0.10 Address 1: 172.21.0.10 kube-dns.kube-system.svc.cluster.local Name: 1wordpress-mysq Address 1: 172.17.59.87 172-17-59-87.wordpress-mysql.default.svc.cluster.local # ^- headless 不会为服务分配cluster-ip，同时dns会为每个pod instance分配对应的dns域名8. 同理创建 wordpress pvc 和 deployment
kubectl apply -f config/wordpress-pvc.yaml kubectl apply -f config/wordpress-deployment.yaml9. 通过loadbalance方式暴露 wordpress service
kubectl expose deploy wordpress --port=80 --target-port=80 --type=LoadBalancer --name wordpress-svc localhost:single_instance_wordpress spark$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 172.21.0.1443/TCP 9h mysql-svc LoadBalancer 172.21.151.23 f9d4dc4c-us-south.lb.appdomain.cloud 3306:30863/TCP 3h4m wordpress-mysql ClusterIP None 3306/TCP 7h25m wordpress-svc LoadBalancer 172.21.12.204 09cdb3f4-us-south.lb.appdomain.cloud 80:31380/TCP 5h18m
通过分配的域名 f9d4dc4c-us-south.lb.appdomain.cloud 可以直接在浏览器里访问wordpress
如果访问不了，需要在vpc下的default security group 下允许端口到 30000到 32767的tcp流量通过。
本地通过mysql client 连接暴露的mysql svc， 确认wordpress 已经连接到mysql svc，并产生数据表项
kubectl expose deploy wordpress-mysql --port=3306 --target-port=3306 --type=LoadBalancer --name mysql-svc
localhost:IKS spark$ mysql -h f9d4dc4c-us-south.lb.appdomain.cloud -u root --password=t0p-Secret mysql> SHOW DATAbaseS; +---------------------+ | Database | +---------------------+ | information_schema | | #mysql50#lost+found | | mysql | | performance_schema | | wordpress | +---------------------+ 5 rows in set (0.27 sec) mysql> use wordpress; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> tables; ... #省略 mysql> select * from wp_users; .... #省略 1 row in set (0.32 sec) mysql> exit;11 查看所有的资源
localhost:single_instance_wordpress spark$ kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 172.21.0.1一键部署443/TCP 9h mysql-svc LoadBalancer 172.21.151.23 f9d4dc4c-us-south.lb.appdomain.cloud 3306:30863/TCP 3h13m wordpress-mysql ClusterIP None 3306/TCP 7h34m wordpress-svc LoadBalancer 172.21.12.204 09cdb3f4-us-south.lb.appdomain.cloud 80:31380/TCP 5h27m localhost:single_instance_wordpress spark$ localhost:single_instance_wordpress spark$ kubectl get pvc NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE my-pvc Bound pvc-c09494f1-bfbc-45d7-a20b-1469819644df 10Gi RWO ibmc-vpc-block-retain-5iops-tier 8h wordpress-pvc Bound pvc-519a59f8-286a-400b-9d4a-76ba3ed93bc8 10Gi RWO ibmc-vpc-block-general-purpose 5h34m localhost:single_instance_wordpress spark$ localhost:single_instance_wordpress spark$ kubectl get pv NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE pvc-519a59f8-286a-400b-9d4a-76ba3ed93bc8 10Gi RWO Delete Bound default/wordpress-pvc ibmc-vpc-block-general-purpose 5h34m pvc-c09494f1-bfbc-45d7-a20b-1469819644df 10Gi RWO Retain Bound default/my-pvc ibmc-vpc-block-retain-5iops-tier 8h localhost:single_instance_wordpress spark$ localhost:single_instance_wordpress spark$ kubectl get pods NAME READY STATUS RESTARTS AGE my-deployment-6d78f74d89-kw2dt 1/1 Running 5 5h20m wordpress-77cb7cb6cc-zqbq5 1/1 Running 0 5h8m wordpress-mysql-84b6cc4c6c-n4hg6 1/1 Running 0 7h42m localhost:single_instance_wordpress spark$ localhost:single_instance_wordpress spark$ kubectl get deployments NAME READY UP-TO-DATE AVAILABLE AGE my-deployment 1/1 1 1 5h20m wordpress 1/1 1 1 5h9m wordpress-mysql 1/1 1 1 7h43m localhost:single_instance_wordpress spark$
可以指定文件夹来一键部署以上所有资源
kubectl apply -f ./config/
