准备spark环境:
linux启动Spark本地模式(Local模式)_怪只怪满眼尽是人间烟火-CSDN博客首先安装java环境下载spark:wget https://archive.apache.org/dist/spark/spark-2.3.0/spark-2.3.0-bin-hadoop2.7.tgz创建解析文件:mkdir /opt/inputcd /opt/input/touch wenjian.txt文件中输入测试内容:hello azhello spark金庸防火墙:CentOS7systemctl stop firewalldCentOShttps://qushen.blog.csdn.net/article/details/122997521
下载 seatunnel:
wget https://github.com/InterestingLab/seatunnel/releases/download/v1.5.7/seatunnel-1.5.7.zip -O seatunnel-1.5.7.zip
解压,并采用软连接重命名:
unzip seatunnel-1.5.7.zip ln -s seatunnel-1.5.7.zip seatunnel
编辑 config/seatunnel-env.sh, 指定必须环境配置如SPARK_HOME
编辑 config/application.conf, 它决定了seatunnel启动后,数据输入,处理,输出的方式和逻辑。
spark {
# seatunnel defined streaming batch duration in seconds
spark.streaming.batchDuration = 5
spark.app.name = "seatunnel"
spark.ui.port = 13000
}
input {
socketStream {}
}
filter {
split {
fields = ["msg", "name"]
delimiter = ","
}
}
output {
stdout {}
}
重启启动一个窗口,启动netcat server用于发送数据,输入:
nc -l -p 9999
启动seatunnel
./bin/start-seatunnel.sh --master local[4] --deploy-mode client --config ./config/application.conf
然后,切换到netcat server那个窗口,发送数据:
Hello World, Gary
seatunnel日志打印出:



