无论是会话模式,还是单作业模式,Flink无法运行:
会话模式无法开启会话
单作业模式的job在yarn的web上查看作业是failed的
报错:
The Flink Yarn cluster has failed
org.apache.flink.client.deployment.ClusterDeploymentException: Couldn’t deploy Yarn session cluster
猜测是hadoop检查虚拟内存不够,因此加了配置不让它检查,然后竟然成功运行了
[dwc@hadoop102 hadoop-2.7.5]$ cd etc/hadoop/
[dwc@hadoop102 hadoop]$ vim yarn-site.xml
3 尝试提交单作业yarn.nodemanager.pmem-check-enabled false yarn.nodemanager.vmem-check-enabled false
./bin/flink run -t yarn-per-job -c com.dwc.CDCJob /opt/Environment_DataLake/flink-1.13.3/job/DataToLake-1.0-SNAPSHOT.jar
yarn的web中可以看到job成功running
4 会话模式./yarn-session.sh -nm test
进入输出的网址可以看到申请的资源



