flink部署模式（五）- native k8s 两种部署模式

首先需要k8s环境，并且k8s version >1.9。flink版本flink1.15.0。

k8s部署（minkube简单快速部署k8s）：https://minikube.sigs.k8s.io/docs/start/
如果使用 MiniKube，请确保在部署 Flink 集群之前先执行 minikube ssh ‘sudo ip link set docker0 promisc on’，否则 Flink 组件不能自动地将自己映射到 Kubernetes Service 中。

# create namespace
kubectl create ns flink-native
# 设置命名空间首选项
kubectl config set-context --current --namespace=flink-native
# create serviceaccount 
kubectl create serviceaccount flink
# 赋权
kubectl create clusterrolebinding flink-role-binding-flink --clusterrole=cluster-admin --serviceaccount=flink-native:flink --namespace=flink-native

./bin/kubernetes-session.sh 
  -Dkubernetes.cluster-id=flink-native-session 
  -Dkubernetes.namespace=flink-native
  -Dkubernetes.service-account=flink 
  -Dkubernetes.rest-service.exposed.type=ClusterIP 
  -Dtaskmanager.memory.process.size=4096m 
  -Dkubernetes.taskmanager.cpu=2 
  -Dtaskmanager.numberOfTaskSlots=5 
  -Dresourcemanager.taskmanager-timeout=30000 
  -Drest.flamegraph.enabled=true 
  -Dkubernetes.container-start-command-template="%java% %classpath% %jvmmem% %jvmopts% %logging% %class% %args%"
  

查看namespace

kubectl get ns

查看服务

kubectl get pod,svc,deployment -n flink-native

查看日志

kubectl logs deployment/flink-native-session -n flink-native

删除服务

kubectl delete deployment flink-native-session -n flink-native

将本机默认路由上的8082端口转发到service flink-native-session-rest中的8081端口上

kubectl -n flink-native port-forward --address 0.0.0.0 service/flink-native-session-rest 8082:8081

application 模式首先需要构建应用镜像

Dockerfile

tee Dockerfile < 
构建镜像
 
 
 
docker build -t ponylee/flink:1.15.0-java8 .
 
 
NOTE： 需要镜像上传到docker镜像私服（或者在每个node节点都构建相同的镜像），并保证所有k8s node节点都有权限链接到此私服。
 
登录本地镜像仓库
 
 
 
docker login 192.168.0.8 -u username -p xxx
 
 
重新打标签
 
 
 
docker tag ponylee/flink:1.15.0-java8 192.168.0.8/bdp/flink:1.15.0-java8
 
 
推送镜像
 
 
 
docker push 192.168.0.8/bdp/flink:1.15.0-java8
 
 
测试镜像可用性
 
 
 
docker run --name flink -d 192.168.0.8/bdp/flink:1.15.0-java8 jobmanager
 docker exec -it flink bash
 
 
启动集群 
./bin/flink run-application 
  --target kubernetes-application 
  -Dkubernetes.cluster-id=flink-native-application 
  -Dkubernetes.namespace=flink-native
  -Dkubernetes.service-account=flink 
  -Dkubernetes.rest-service.exposed.type=ClusterIP 
  -Dtaskmanager.memory.process.size=2048m 
  -Dkubernetes.taskmanager.cpu=2 
  -Dtaskmanager.numberOfTaskSlots=5 
  -Dresourcemanager.taskmanager-timeout=30000 
  -Drest.flamegraph.enabled=true 
  -Dkubernetes.container.image=192.168.0.8/bdp/flink:1.15.0-java8 
  -Dkubernetes.container-start-command-template="%java% %classpath% %jvmmem% %jvmopts% %logging% %class% %args%" 
  local:///opt/flink/usrlib/flink-demo-1.0-SNAPSHOT-pony-shade.jar 
 
NOTE：
 
 
 
local:///opt/flink/usrlib/flink-demo-1.0-SNAPSHOT-pony-shade.jar 为容器内部路径
 
 
查看服务
 
 
 
kubectl get pod,svc,deployment -n flink-native
 
 
查看日志
 
 
 
kubectl logs deployment/flink-native-application
 kubectl logs flink-native-application-fcc68fb99-r4j4k（pod） -n flink-native
 
 
删除服务
 
 
 
kubectl delete pod flink-native-application-fcc68fb99-r4j4k -n default
 kubectl delete deployment/flink-native-application -n flink-native
 kubectl delete service flink-native-application flink-native-application-rest -n flink-native
 
 
将本机默认路由上的8082端口转发到service flink-native-application-rest中的8081端口上
 
 
 
kubectl -n flink-native port-forward --address 0.0.0.0 service/flink-native-application-rest 8082:8081