helm入门_Go语言

helm入门简介

Helm是一个由CNCF孵化和管理的项目，用于对需要在Kubernetes 上部署的复杂应用进行定义、安装和更新。Helm以Chart的方式对应用软件进行描述，可以方便地创建、版本化、共享和发布复杂的应用软件。

helm3架构

helm3安装

#github地址
#https://github.com/helm/helm
#本文示例使用的是v3.7.0版本
wget https://get.helm.sh/helm-v3.7.0-linux-amd64.tar.gz
#解压->helm放入PATH一个路径下

helm中三大概念

Chart：一个Helm包，其中包含运行一个应用所需要的工具和资源定义，还可能包含Kubernetes集群中的服务定义，类似于Homebrew 中的formula、APT中的dpkg或者Yum中的RPM文件。
Release：在Kubernetes集群上运行的一个Chart实例。在同一个集群上，一个Chart可以被安装多次。例如有一个MySQL Chart，如果想在服务器上运行两个MySQL数据库，就可以基于这个Chart安装两次。每次安装都会生成新的Release，会有独立的Release名称。
Repository：用于存放和共享Chart仓库。简单来说，Helm整个系统的主要任务就是，在仓库中查找需要的 Chart，然后将Chart以Release的形式安装到Kubernetes集群中。

Helm Chart的使用

下面将使用一个例子展示helm chart的使用。

创建

$ helm create nginx

该命令会创建一个nginx文件目录，tree查看目录结构

$ tree
.
├── charts  #包含chart依赖的其他chart
├── Chart.yaml #包含了chart信息的YAML文件
├── templates #模板目录， 当和values 结合时，可生成有效的Kubernetes manifest文件
│   ├── deployment.yaml
│   ├── _helpers.tpl
│   ├── hpa.yaml
│   ├── ingress.yaml
│   ├── NOTES.txt
│   ├── serviceaccount.yaml
│   ├── service.yaml
│   └── tests #测试
│       └── test-connection.yaml
└── values.yaml #chart 默认的配置值

Chart.yaml

$ cat Chart.yaml 
apiVersion: v2 #在heml3中apiVersion必须是v2
name: nginx #chart名字
description: A Helm chart for Kubernetes #chart描述
type: application #chart类型 application（默认）、library
version: 0.1.0 #chart的版本
appVersion: "1.16.0" #应用的版本

values.yaml

$ cat values.yaml 
# Default values for nginx.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

replicaCount: 1

image:
  repository: nginx
  pullPolicy: IfNotPresent
  # Overrides the image tag whose default is the chart appVersion.
  tag: ""

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

serviceAccount:
  # Specifies whether a service account should be created
  create: true
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: ""

podAnnotations: {}

podSecurityContext: {}
  # fsGroup: 2000

securityContext: {}
  # capabilities:
  #   drop:
  #   - ALL
  # readOnlyRootFilesystem: true
  # runAsNonRoot: true
  # runAsUser: 1000

service:
  type: ClusterIP
  port: 80

ingress:
  enabled: false
  className: ""
  annotations: {}
    # kubernetes.io/ingress.class: nginx
    # kubernetes.io/tls-acme: "true"
  hosts:
    - host: chart-example.local
      paths:
        - path: /
          pathType: ImplementationSpecific
  tls: []
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

resources: {}
  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  # limits:
  #   cpu: 100m
  #   memory: 128Mi
  # requests:
  #   cpu: 100m
  #   memory: 128Mi

autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 100
  targetCPUUtilizationPercentage: 80
  # targetMemoryUtilizationPercentage: 80

nodeSelector: {}

tolerations: []

affinity: {}

templates目录下存放了应用编排文件。

（_）开头的文件用来存储局部和辅助对象，供其他chart模板使用。模板命令都是嵌入在{{ 和 }}之间的。

cat _helpers.tpl 
{{}}
{{- define "nginx.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}

{{}}
{{- define "nginx.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}

{{}}
{{- define "nginx.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{}}
{{- define "nginx.labels" -}}
helm.sh/chart: {{ include "nginx.chart" . }}
{{ include "nginx.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{}}
{{- define "nginx.selectorLabels" -}}
app.kubernetes.io/name: {{ include "nginx.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{}}
{{- define "nginx.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "nginx.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}

yaml格式的编排文件中将某些字段设置为“模板命令”，这些“模板命令”会在Helm部署应用时进行参数注入和模板的动态渲染。

$ cat deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "nginx.fullname" . }}
  labels:
    {{- include "nginx.labels" . | nindent 4 }}
spec:
  {{- if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
  {{- end }}
  selector:
    matchLabels:
      {{- include "nginx.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      {{- with .Values.podAnnotations }}
      annotations:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      labels:
        {{- include "nginx.selectorLabels" . | nindent 8 }}
    spec:
      {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      serviceAccountName: {{ include "nginx.serviceAccountName" . }}
      securityContext:
        {{- toYaml .Values.podSecurityContext | nindent 8 }}
      containers:
        - name: {{ .Chart.Name }}
          securityContext:
            {{- toYaml .Values.securityContext | nindent 12 }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /
              port: http
          readinessProbe:
            httpGet:
              path: /
              port: http
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.affinity }}
      affinity:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}

可是使用helm template [NAME] [CHART] [flags]命令在本地渲染Helm Chart并打印最终的应用模板内容，values.yaml和（_）开头的文件中的值会被注入到yaml中，参数--set则会覆盖value.yaml文件中的值。

$ helm template nginx  ./nginx --set image.repository=docker.io/library/nginx,service.type=NodePort
...
...
---
# Source: nginx/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    helm.sh/chart: nginx-0.1.0
    app.kubernetes.io/name: nginx
    app.kubernetes.io/instance: nginx
    app.kubernetes.io/version: "1.16.0"
    app.kubernetes.io/managed-by: Helm
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx
      app.kubernetes.io/instance: nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nginx
        app.kubernetes.io/instance: nginx
    spec:
      serviceAccountName: nginx
      securityContext:
        {}
      containers:
        - name: nginx
          securityContext:
            {}
          image: "docker.io/library/nginx:1.16.0"
          imagePullPolicy: IfNotPresent
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
          livenessProbe:
            httpGet:
              path: /
              port: http
          readinessProbe:
            httpGet:
              path: /
              port: http
          resources:
            {}
---
...
...

NOTES.txt在helm install 或 helm upgrade命令的最后，Helm会打印出对用户有用的信息，该文件是纯文本，但会像模板一样处理，所有正常的模板函数和对象都是可用的。

$ cat NOTES.txt 
1. Get the application URL by running these commands:
{{- if .Values.ingress.enabled }}
{{- range $host := .Values.ingress.hosts }}
  {{- range .paths }}
  http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
  {{- end }}
{{- end }}
{{- else if contains "NodePort" .Values.service.type }}
  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "nginx.fullname" . }})
  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT
{{- else if contains "LoadBalancer" .Values.service.type }}
     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
           You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "nginx.fullname" . }}'
  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "nginx.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
  echo http://$SERVICE_IP:{{ .Values.service.port }}
{{- else if contains "ClusterIP" .Values.service.type }}
  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "nginx.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
  export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
{{- end }}

部署

helm install [NAME] [CHART] [flags]命令安装部署，同样values.yaml和（_）开头的文件中的值会被注入到yaml中，参数--set则会覆盖value.yaml文件中的值。

$ helm -n default install nginx ./nginx --set image.repository=docker.io/library/nginx,service.type=NodePort
NAME: nginx
LAST DEPLOYED: Sat Oct  9 16:26:50 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services nginx)
  export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT

查看部署的release

$ helm -n default ls
NAME 	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART      	APP VERSION
nginx	default  	1       	2021-10-09 16:26:50.820014356 +0800 CST	deployed	nginx-0.1.0	1.16.0

release版本信息存储

默认存储在release相应namespace下的secret（可配置configmap、sql），每一个REVISION都会生成一个secret。

$ kubectl -n default get secret -l "owner=helm"
NAME                          TYPE                 DATA   AGE
sh.helm.release.v1.nginx.v1   helm.sh/release.v1   1      1m

查看存储内容

kubectl -n default get secret sh.helm.release.v1.nginx.v1 -ojsonpath={.data.release} |base64 -d |base64 -d |gzip -d

测试release是否符合预期（需等待所有部署的资源对象创建成功），原理：通过在集群中部署templates/tests/下的yaml，根据容器退出状态判断（exit 0：通过）是否测试通过。测试的yaml必须含有注解：helm.sh/hook: test。

$ helm test nginx
NAME: nginx
LAST DEPLOYED: Sat Oct  9 16:26:50 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE:     nginx-test-connection
Last Started:   Sat Oct  9 16:47:02 2021
Last Completed: Sat Oct  9 16:47:20 2021
Phase:          Succeeded
NOTES:
1. Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services nginx)
  export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT

升级、回滚、卸载

helm upgrade [RELEASE] [CHART] [flags]命令进行升级。

升级之前查看nginx release当前版本：REVISION=1

helm -n default ls
NAME 	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART      	APP VERSION
nginx	default  	1       	2021-10-09 16:26:50.820014356 +0800 CST	deployed	nginx-0.1.0	1.16.0

使用升级命令，将deployment中的镜像tag升级到1.17.0，可使用--dry-run试运行测试是否可以升级成功，并打印升级清单。

$ helm -n default upgrade nginx ./nginx/ --set image.repository=docker.io/library/nginx,service.type=NodePort,image.tag=1.17.0
Release "nginx" has been upgraded. Happy Helming!
NAME: nginx
LAST DEPLOYED: Sun Oct 10 13:41:17 2021
NAMESPACE: default
STATUS: deployed
REVISION: 2
NOTES:
1. Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].nodePort}" services nginx)
  export NODE_IP=$(kubectl get nodes --namespace default -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT

再次查看nginx release当前版本：REVISION=2

$ helm -n default ls
NAME 	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART      	APP VERSION
nginx	default  	2       	2021-10-10 13:41:17.486033411 +0800 CST	deployed	nginx-0.1.0	1.16.0

helm rollback [REVISION] [flags]命令进行回滚。

回滚之前可使用helm history RELEASE_NAME [flags]查看nginx release历史版本。

$ helm -n default history nginx
REVISION	UPDATED                 	STATUS    	CHART      	APP VERSION	DESCRIPTION     
1       	Sat Oct  9 16:26:50 2021	superseded	nginx-0.1.0	1.16.0     	Install complete
2       	Sun Oct 10 13:41:17 2021	deployed  	nginx-0.1.0	1.16.0     	Upgrade complete

将nginx release回滚到REVISION=1，回滚完成后可查看deployment image tag：1.16.0。

$ helm -n default rollback nginx 1
Rollback was a success! Happy Helming!

helm uninstall RELEASE_NAME [...] [flags]命令卸载release，卸载完成后release、release secret、k8s manifest都被删除。
```
$ helm -n default uninstall nginx
release "nginx" uninstalled
```

仓库使用 Harbor

使用helm安装harbor

$ helm repo add harbor https://helm.goharbor.io
"harbor" has been added to your repositories
#查看本地repo列表
$ helm repo ls
NAME  	URL                     
harbor	https://helm.goharbor.io
#查看harbor仓库helm chart版本
$ helm search repo harbor
NAME         	CHART VERSION	APP VERSION	DESCRIPTION                                       
harbor/harbor	1.7.3        	2.3.3      	An open source trusted cloud native registry th...
#部署harbor release
#persistence.enabled=false 关闭持久化存储
#expose.type=nodePort service类型设置为nodeport方便暴露服务
#expose.tls.enabled=false 关闭tls
#--create-namespace 没有harbor命名空间，则会创建
#externalURL中的ip为本机网卡ip
$ helm -n harbor install harbor harbor/harbor --set persistence.enabled=false,expose.type=nodePort,expose.tls.enabled=false,externalURL=http://10.23.18.211:30002 --create-namespace
NAME: harbor
LAST DEPLOYED: Mon Oct 11 18:38:47 2021
NAMESPACE: harbor
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Please wait for several minutes for Harbor deployment to complete.
Then you should be able to visit the Harbor portal at http://10.23.18.211:30002
For more details, please visit https://github.com/goharbor/harbor
#等待所有pod running
$ kubectl -n harbor get pod
NAME                                    READY   STATUS    RESTARTS   AGE
harbor-chartmuseum-6c9f9c84f8-ztpgp     1/1     Running   0          3m20s
harbor-core-c6dc8c895-vn5kx             1/1     Running   0          3m19s
harbor-database-0                       1/1     Running   0          3m20s
harbor-jobservice-6c4d647bcb-nd64t      1/1     Running   0          3m20s
harbor-nginx-7bc597c58b-67x6x           1/1     Running   0          3m19s
harbor-notary-server-5f598b9555-4qjpm   1/1     Running   0          3m20s
harbor-notary-signer-65b84d688d-g92tv   1/1     Running   0          3m19s
harbor-portal-945d945f-ks6tx            1/1     Running   0          3m20s
harbor-redis-0                          1/1     Running   0          3m20s
harbor-registry-6d976587dc-dbrnt        2/2     Running   0          3m20s
harbor-trivy-0                          1/1     Running   0          3m20s

浏览器访问http://10.23.18.211:30002即可访问harbor页面，admin/Harbor12345为默认用户密码。

推送nginx chart到仓库

登录harbor页面，项目–>新建项目，创建一个名为helm的helm chart仓库。

添加helm仓库。

$ helm repo add my-harbor http://10.23.18.211:30002/chartrepo/helm --username=admin --password=Harbor12345
"my-harbor" has been added to your repositories
#查看当前repo
$ helm repo ls
NAME     	URL                                     
harbor   	https://helm.goharbor.io                
my-harbor	http://10.23.18.211:30002/chartrepo/helm

由于原生helm没有推送命令，所以需要安装一个推送插件。

$ helm plugin install https://github.com/chartmuseum/helm-push.git
Downloading and installing helm-push v0.10.0 ...
https://github.com/chartmuseum/helm-push/releases/download/v0.10.0/helm-push_0.10.0_linux_amd64.tar.gz
Installed plugin: cm-push
#查看已安装的插件
$ helm plugin ls
NAME   	VERSION	DESCRIPTION                      
cm-push	0.10.0 	Push chart package to ChartMuseum

（备选方案）如果服务器不能访问外网。可以从能够访问外网的机器clone插件代码和插件压缩包，然后放到此服务器上。

#在能够访问外网的机器上
$ git clone https://github.com/chartmuseum/helm-push.git
$curl https://github.com/chartmuseum/helm-push/releases/download/v0.10.0/helm-push_0.10.0_linux_amd64.tar.gz
#将插件代码上传到服务器/root/.cache/helm/plugins目录下，创建release目录
$ mkdiir /root/.cache/helm/plugins/helm-push/release
#将插件压缩包上传到服务器/root/.cache/helm/plugins/helm-push/releases下
$ cd /root/.cache/helm/plugins
#vim 编辑helm-push/scripts/install_plugin.sh，根据需要注释掉 `# Download with curl if possible.`下边代码，以注释掉curl为例：
$ vim helm-push/scripts/install_plugin.sh
$ cat helm-push/scripts/install_plugin.sh
...
...
# Download with curl if possible.
if [ -x "$(which curl 2>/dev/null)" ]; then
  #  curl -sSL "${url}" -o "releases/v${version}.tar.gz"
     echo "jump curl"
else
    wget -q "${url}" -O "releases/v${version}.tar.gz"
fi
...
...
#安装推送插件
$ helm plugin install ./helm-push/
Downloading and installing helm-push v0.10.0 ...
https://github.com/chartmuseum/helm-push/releases/download/v0.10.0/helm-push_0.10.0_linux_amd64.tar.gz
jump curl
Installed plugin: cm-push
[root@master01 plugins]# helm plugin ls
NAME   	VERSION	DESCRIPTION                      
cm-push	0.10.0 	Push chart package to ChartMuseum

推送nginx chart。

#cd到nginx目录，推送
$ helm cm-push ./nginx my-harbor
Pushing nginx-0.1.0.tgz to my-harbor...
Done.
#更新本地缓存的my-harbor仓库信息，可以确认nginx chart上传到harbor上，也可以在harbor页面上确认
$ helm repo  update my-harbor 
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "my-harbor" chart repository
Update Complete. ⎈Happy Helming!⎈
$ helm search repo my-harbor
NAME           	CHART VERSION	APP VERSION	DESCRIPTION                
my-harbor/nginx	0.1.0        	1.16.0     	A Helm chart for Kubernetes

拉取、部署仓库中的chart

helm pull [chart URL | repo/chartname] [...] [flags]拉取仓库中chart，pull只会拉取到本地当前目录下的.tgz压缩包，可使用tar xzvf xxx.tgz解压。

#更新本地helm repo 缓存
$ helm repo update
#pull拉取nginx chart
$ helm pull my-harbor/nginx

helm install [NAME] [CHART] [flags]部署仓库中的chart，和上边部署篇幅类似，把使用本地文件改成使用仓库中的chart即可。

$ helm -n test install nginx my-harbor/nginx --set image.repository=docker.io/library/nginx,service.type=NodePort --create-namespace
NAME: nginx
LAST DEPLOYED: Mon Oct 11 19:47:36 2021
NAMESPACE: test
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export NODE_PORT=$(kubectl get --namespace test -o jsonpath="{.spec.ports[0].nodePort}" services nginx)
  export NODE_IP=$(kubectl get nodes --namespace test -o jsonpath="{.items[0].status.addresses[0].address}")
  echo http://$NODE_IP:$NODE_PORT

helm入门

Go语言相关栏目本月热门文章