栏目分类:
子分类:
返回
名师互学网用户登录
快速导航关闭
当前搜索
当前分类
子分类
实用工具
热门搜索
名师互学网 > IT > 前沿技术 > 大数据 > 大数据系统

Apache seaTunnel的本地启动

Apache seaTunnel的本地启动

准备spark环境:

linux启动Spark本地模式(Local模式)_怪只怪满眼尽是人间烟火-CSDN博客首先安装java环境下载spark:wget https://archive.apache.org/dist/spark/spark-2.3.0/spark-2.3.0-bin-hadoop2.7.tgz创建解析文件:mkdir /opt/inputcd /opt/input/touch wenjian.txt文件中输入测试内容:hello azhello spark金庸防火墙:CentOS7systemctl stop firewalldCentOShttps://qushen.blog.csdn.net/article/details/122997521

下载 seatunnel:

wget https://github.com/InterestingLab/seatunnel/releases/download/v1.5.7/seatunnel-1.5.7.zip -O seatunnel-1.5.7.zip

解压,并采用软连接重命名:

unzip seatunnel-1.5.7.zip
ln -s seatunnel-1.5.7.zip seatunnel

编辑 config/seatunnel-env.sh, 指定必须环境配置如SPARK_HOME

编辑 config/application.conf, 它决定了seatunnel启动后,数据输入,处理,输出的方式和逻辑。

spark {
  # seatunnel defined streaming batch duration in seconds
  spark.streaming.batchDuration = 5

  spark.app.name = "seatunnel"
  spark.ui.port = 13000
}

input {
  socketStream {}
}

filter {
  split {
    fields = ["msg", "name"]
    delimiter = ","
  }
}

output {
  stdout {}
}

重启启动一个窗口,启动netcat server用于发送数据,输入:

nc -l -p 9999

启动seatunnel

./bin/start-seatunnel.sh --master local[4] --deploy-mode client --config ./config/application.conf

 然后,切换到netcat server那个窗口,发送数据:

Hello World, Gary

seatunnel日志打印出:

转载请注明:文章转载自 www.mshxw.com
本文地址:https://www.mshxw.com/it/741956.html
我们一直用心在做
关于我们 文章归档 网站地图 联系我们

版权所有 (c)2021-2022 MSHXW.COM

ICP备案号:晋ICP备2021003244-6号