栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 软件开发 > 后端开发 > Java

Flink将数据写入到mysql

Java 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

Flink将数据写入到mysql

import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.connector.jdbc.JdbcConnectionOptions;
import org.apache.flink.connector.jdbc.JdbcExecutionOptions;
import org.apache.flink.connector.jdbc.JdbcSink;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;


public class _18SinkToMySQLTest {
    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        env.setParallelism(1);

        DataStreamSource streamSource = env.readTextFile("input/clicks.csv");
        // 将字符串封装为Event对象
        SingleOutputStreamOperator mapSource = streamSource.map(new MapFunction() {
            @Override
            public Event map(String line) throws Exception {
                String[] split = line.split(", ");
                Event event = new Event(split[0], split[1], Long.valueOf(split[2]));
                return event;
            }
        });

        mapSource.addSink(
                //参数1:sql
                JdbcSink.sink("INSERT INTO clicks(user, url) VALUES (?,?) ",
                        //参数2:statement占位符
                        (statement, data) -> {
                            statement.setString(1, data.user);
                            statement.setString(2, data.url);
                        },
                        //参数3:设置
                        JdbcExecutionOptions.builder()
                                .withBatchIntervalMs(200)   //批次提交周期:每200ms
                                .withBatchSize(1000)        //批次提交周期:每1000条
                                .withMaxRetries(5)          //批次提交周期,每5次
                                .build(),
                        new JdbcConnectionOptions.JdbcConnectionOptionsBuilder()
                        .withUrl("jdbc:mysql://hadoop102:3306/test")    //mysql地址
                        .withDriverName("com.mysql.jdbc.Driver")
                        .withUsername("root")
                        .withPassword("000000")
                        .build()
                        )
        );

        env.execute();
    }
}

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/941238.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号