栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 软件开发 > 后端开发 > Java

Flink1.15源码阅读flink-clients之GenericCLI、flinkYarnSessionCLI和DefaultCLI

Java 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

Flink1.15源码阅读flink-clients之GenericCLI、flinkYarnSessionCLI和DefaultCLI

文章目录
  • CliFrontend#main
  • 将三种命令行对象按顺序添加到List
  • CliFrontend#run
    • 获取活跃的命令行
      • 1. GenericCLI#isActive
      • 2. flinkYarnSessionCLI#isActive
      • 3. DefaultCLI#isActive

CliFrontend#main

进入到main方法,从命令行入手

// 3. load the custom command lines
        final List customCommandLines =
                loadCustomCommandLines(configuration, configurationDirectory);
将三种命令行对象按顺序添加到List

将三种命令行对象按顺序添加到List并返回命令行集合对象

 public static List loadCustomCommandLines(
            Configuration configuration, String configurationDirectory) {
        // 声明空集合
        List customCommandLines = new ArrayList<>();
        // 将GenericCLI添加到集合中
        customCommandLines.add(new GenericCLI(configuration, configurationDirectory));

        //	Command line interface of the YARN session, with a special initialization here
        //	to prefix all options with y/yarn.
        final String flinkYarnSessionCLI = "org.apache.flink.yarn.cli.FlinkYarnSessionCli";
        try {
        // 将flinkYarnSessionCLI添加到集合中
            customCommandLines.add(
                    loadCustomCommandLine(
                            flinkYarnSessionCLI,
                            configuration,
                            configurationDirectory,
                            "y",
                            "yarn"));
        } catch (NoClassDefFoundError | Exception e) {
            final String errorYarnSessionCLI = "org.apache.flink.yarn.cli.FallbackYarnSessionCli";
            try {
                LOG.info("Loading FallbackYarnSessionCli");
                customCommandLines.add(loadCustomCommandLine(errorYarnSessionCLI, configuration));
            } catch (Exception exception) {
                LOG.warn("Could not load CLI class {}.", flinkYarnSessionCLI, e);
            }
        }

        //	Tips: DefaultCLI must be added at last, because getActiveCustomCommandLine(..) will get
        // the
        //	      active CustomCommandLine in order and DefaultCLI isActive always return true.
        // 最后将默认客户端添加到集合中
        customCommandLines.add(new DefaultCLI());

		// 返回值集合
        return customCommandLines;
    }
CliFrontend#run

先看下该方法的全部代码

protected void run(String[] args) throws Exception {
        LOG.info("Running 'run-application' command.");

        final Options commandOptions = CliFrontendParser.getRunCommandOptions();
        final CommandLine commandLine = getCommandLine(commandOptions, args, true);

        if (commandLine.hasOption(HELP_OPTION.getOpt())) {
            CliFrontendParser.printHelpForRunApplication(customCommandLines);
            return;
        }
// 这里获取活跃的命令行,着重看这里
        final CustomCommandLine activeCommandLine = validateAndGetActiveCommandLine(checkNotNull(commandLine));

        final ApplicationDeployer deployer =
                new ApplicationClusterDeployer(clusterClientServiceLoader);

        final ProgramOptions programOptions;
        final Configuration effectiveConfiguration;

        // No need to set a jarFile path for Pyflink job.
        if (ProgramOptionsUtils.isPythonEntryPoint(commandLine)) {
            programOptions = ProgramOptionsUtils.createPythonProgramOptions(commandLine);
            effectiveConfiguration =
                    getEffectiveConfiguration(
                            activeCommandLine,
                            commandLine,
                            programOptions,
                            Collections.emptyList());
        } else {
            programOptions = new ProgramOptions(commandLine);
            programOptions.validate();
            final URI uri = PackagedProgramUtils.resolveURI(programOptions.getJarFilePath());
            effectiveConfiguration =
                    getEffectiveConfiguration(
                            activeCommandLine,
                            commandLine,
                            programOptions,
                            Collections.singletonList(uri.toString()));
        }

        final ApplicationConfiguration applicationConfiguration =
                new ApplicationConfiguration(
                        programOptions.getProgramArgs(), programOptions.getEntryPointClassName());
        deployer.run(effectiveConfiguration, applicationConfiguration);
    }
获取活跃的命令行
// 摘自上runApplication 方法
final CustomCommandLine activeCommandLine = validateAndGetActiveCommandLine(checkNotNull(commandLine));

点击进去validateAndGetActiveCommandLine方法

    public CustomCommandLine validateAndGetActiveCommandLine(CommandLine commandLine) {
        LOG.debug("Custom commandlines: {}", customCommandLines);
        for (CustomCommandLine cli : customCommandLines) {
            LOG.debug(
                    "Checking custom commandline {}, isActive: {}", cli, cli.isActive(commandLine));
            if (cli.isActive(commandLine)) {
                return cli;
            }
        }
        throw new IllegalStateException("No valid command-line found.");
    }

从上面注释中,很容易看出该方法是干什么的,获取活跃的命令行客户端;
for循环中顺序的取出之前存到集合中的三个客户端,根据cli.isActive(commandLine)这个方法检查,第一个活跃的客户端并立即返回,因为默认的客户端active总是true,从下面代码可以看出

1. GenericCLI#isActive
@Override
    public boolean isActive(CommandLine commandLine) {
        return configuration.getOptional(DeploymentOptions.TARGET).isPresent()
                || commandLine.hasOption(executorOption.getOpt())
                || commandLine.hasOption(targetOption.getOpt());
    }
  • configuration.getOptional(DeploymentOptions.TARGET).isPresent() TARGET是否存在

  • executorOption.getOpt()

    该配置是Generic CLI

  • targetOption.getOpt()

上面三个条件满足一个即可。

2. flinkYarnSessionCLI#isActive

3. DefaultCLI#isActive

DefaultCLI#isActive总是true,所以默认客户端总是兜底的

    @Override
    public boolean isActive(CommandLine commandLine) {
        // always active because we can try to read a JobManager address from the config
        return true;
    }
转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/1031919.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号