栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

Flink可视化控制台依赖配置

Flink可视化控制台依赖配置

WebUI可视化界面

访问:ip:8081
方式一:服务端部署Flink集群(生产环境)
方式二:本地依赖添加(测试开发)

 
        
            org.apache.flink
            flink-runtime-web_${scala.version}
            ${flink.version}
        

比较齐全的pom.xml文件



    4.0.0

    net.xdclass
    xdclass-flink
    1.0-SNAPSHOT

    
        UTF-8
        UTF-8
        1.8
        1.8
        1.8
        2.12
        1.13.1
    


    
        
            org.projectlombok
            lombok
            1.18.16
        

        
        
            org.apache.flink
            flink-clients_${scala.version}
            ${flink.version}
        

        
        
            org.apache.flink
            flink-scala_${scala.version}
            ${flink.version}
        
        
        
            org.apache.flink
            flink-java
            ${flink.version}
        

        
        
            org.apache.flink
            flink-streaming-scala_${scala.version}
            ${flink.version}
        
        
        
            org.apache.flink
            flink-streaming-java_${scala.version}
            ${flink.version}
        
        
        
            org.apache.flink
            flink-runtime-web_${scala.version}
            ${flink.version}
        

        
            org.apache.flink
            flink-statebackend-rocksdb_${scala.version}
            1.13.1
        

        
        
            mysql
            mysql-connector-java
            8.0.25
        

        
        
            org.apache.flink
            flink-cep_${scala.version}
            ${flink.version}
        

        
        
            org.apache.bahir
            flink-connector-redis_2.11
            1.0
        

        
        
            org.apache.flink
            flink-connector-kafka_${scala.version}
            ${flink.version}
        


        
        
            org.slf4j
            slf4j-log4j12
            1.7.7
            runtime
        
        
            log4j
            log4j
            1.2.17
            runtime
        

        
        
            com.alibaba
            fastjson
            1.2.44
        
    


    
        xdclass-flink
        

            
            
                org.apache.maven.plugins
                maven-compiler-plugin
                3.6.1
                
                    ${java.version}
                    ${java.version}
                    ${file.encoding}
                
            

            
                org.apache.maven.plugins
                maven-shade-plugin
                2.3
                
                    
                        package
                        
                            shade
                        
                    
                
            
        
    

日志配置 log4j.properties
### 配置appender名称
log4j.rootLogger = debugFile, errorFile
​
### debug级别以上的日志到:./logs/debug.log
log4j.appender.debugFile = org.apache.log4j.DailyRollingFileAppender
log4j.appender.debugFile.File = ./logs/flink.log
log4j.appender.debugFile.Append = true
#Threshold属性指定输出等级
log4j.appender.debugFile.Threshold = info
log4j.appender.debugFile.layout = org.apache.log4j.PatternLayout
log4j.appender.debugFile.layout.ConversionPattern = %-d{yyyy-MM-dd HH:mm:ss}  [ %t:%r ] - [ %p ]  %n%m%n
​
​
### error级别以上的日志 ./logs/error.log
log4j.appender.errorFile = org.apache.log4j.DailyRollingFileAppender
log4j.appender.errorFile.File = ./logs/error.log
log4j.appender.errorFile.Append = true
log4j.appender.errorFile.Threshold = error
log4j.appender.errorFile.layout = org.apache.log4j.PatternLayout
log4j.appender.errorFile.layout.ConversionPattern = %-d{yyyy-MM-dd HH:mm:ss}  [ %t:%r ] - [ %p ]  %n%m%n
代码开发
public static void main(String[] args) throws Exception {
​
​
        final StreamExecutionEnvironment env = StreamExecutionEnvironment.createLocalEnvironmentWithWebUI(new Configuration());
        //env.setParallelism(1);
​
        DataStream stringDataStream = env.socketTextStream("127.0.0.1",8888);
​
        
        DataStream flatMapDataStream = stringDataStream.flatMap(new FlatMapFunction() {
            @Override
            public void flatMap(String value, Collector out) throws Exception {
​
                String[] arr = value.split(",");
                for (String word : arr) {
                    out.collect(word);
                }
            }
        });
        flatMapDataStream.print("结果");
​
        //DataStream需要调用execute,可以取个名称
        env.execute("data stream job");
​
    }
nc命令介绍
Linux nc命令用于设置网络路由的
nc -lk 8888
开启 监听模式,用于指定nc将处于监听模式, 等待客户端来链接指定的端口
win | linux 需要安装
win 百度搜索博文参考不同系统安装
下载地址 https://eternallybored.org/misc/netcat/
linux 安装命令:
yum install -y netcat
yum install -y nc
转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/673431.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号