栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 软件开发 > 后端开发 > Java

Flink将数据写入到redis

Java 更新时间: 发布时间: IT归档 最新发布 模块sitemap 名妆网 法律咨询 聚返吧 英语巴士网 伯小乐 网商动力

Flink将数据写入到redis

import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.connectors.redis.RedisSink;
import org.apache.flink.streaming.connectors.redis.common.config.FlinkJedisPoolConfig;
import org.apache.flink.streaming.connectors.redis.common.mapper.RedisCommand;
import org.apache.flink.streaming.connectors.redis.common.mapper.RedisCommandDescription;
import org.apache.flink.streaming.connectors.redis.common.mapper.RedisMapper;


public class _17SinkToRedisTest {
    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        env.setParallelism(1);

        DataStreamSource streamSource = env.readTextFile("input/clicks.csv");
        // 将字符串封装为Event对象
        SingleOutputStreamOperator mapSource = streamSource.map(new MapFunction() {
            @Override
            public Event map(String line) throws Exception {
                String[] split = line.split(", ");
                Event event = new Event(split[0], split[1], Long.valueOf(split[2]));
                return event;
            }
        });

        // 创建redis链接配置
        FlinkJedisPoolConfig conf = new FlinkJedisPoolConfig.Builder().setHost("hadoop102").build();

        mapSource.addSink(new RedisSink(conf,
                new MyRedisMapper()));

        env.execute();
    }

    public static class MyRedisMapper implements RedisMapper {
        @Override
        public RedisCommandDescription getCommandDescription() {
            // 参数1:redis数据类型   参数2:redis中存储的key
            return new RedisCommandDescription(RedisCommand.HSET, "clicks");
        }

        
        @Override
        public String getKeyFromData(Event event) {
            return event.user;
        }

        
        @Override
        public String getValueFromData(Event event) {
            return event.url;
        }
    }
}

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/941227.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号