flume故障转移测试

flume配置文件我习惯放在flume下的job目录里

cd job

vim flume-netcat-flume.conf

# Name the components on this agent 
a1.sources = r1 
a1.channels = c1 
a1.sinkgroups = g1 
a1.sinks = k1 k2 
 
# Describe/configure the source 
a1.sources.r1.type = netcat 
a1.sources.r1.bind = localhost 
a1.sources.r1.port = 44444 
a1.sinkgroups.g1.processor.type = failover 
a1.sinkgroups.g1.processor.priority.k1 = 5 
a1.sinkgroups.g1.processor.priority.k2 = 10 
a1.sinkgroups.g1.processor.maxpenalty = 10000 
 
# Describe the sink 
a1.sinks.k1.type = avro 
a1.sinks.k1.hostname = master 
a1.sinks.k1.port = 4141 
a1.sinks.k2.type = avro 
a1.sinks.k2.hostname = master 
a1.sinks.k2.port = 4142 
 
# Describe the channel 
a1.channels.c1.type = memory 
a1.channels.c1.capacity = 1000 
a1.channels.c1.transactionCapacity = 100 
 
# Bind the source and sink to the channel 
a1.sources.r1.channels = c1 
a1.sinkgroups.g1.sinks = k1 k2 
a1.sinks.k1.channel = c1 
a1.sinks.k2.channel = c1

复制第二个窗口，同样进入job下

vim flume-flume-console1.conf

# Name the components on this agent
 a2.sources = r1 
 a2.sinks = k1 
 a2.channels = c1 
 
# Describe/configure the source 
a2.sources.r1.type = avro 
a2.sources.r1.bind = master 
a2.sources.r1.port = 4141 
 
# Describe the sink 
a2.sinks.k1.type = logger 
 
# Describe the channel 
a2.channels.c1.type = memory 
a2.channels.c1.capacity = 1000 
a2.channels.c1.transactionCapacity = 100 
 
# Bind the source and sink to the channel 
a2.sources.r1.channels = c1 
a2.sinks.k1.channel = c1 

复制出第三个窗口，进入job目录

vim flume-flume-console2.conf

# Name the components on this agent 
a3.sources = r1 
a3.sinks = k1 
a3.channels = c2 
 
# Describe/configure the source 
a3.sources.r1.type = avro 
a3.sources.r1.bind = master 
a3.sources.r1.port = 4142 
 
# Describe the sink 
a3.sinks.k1.type = logger 
 
# Describe the channel 
a3.channels.c2.type = memory 
a3.channels.c2.capacity = 1000 
a3.channels.c2.transactionCapacity = 100 
 
# Bind the source and sink to the channel 
a3.sources.r1.channels = c2 
a3.sinks.k1.channel = c2 

复制虚拟机窗口
依次顺气执行下面的命令

bin/flume-ng agent --conf conf/ --name a3 --conf-file job/flume-flume-console2.conf -Dflume.root.logger=INFO,console

bin/flume-ng agent --conf conf/ --name a2 --conf-file job/flume-flume-console1.conf -Dflume.root.logger=INFO,console

bin/flume-ng agent --conf conf/ --name a1 --conf-file job/flume-netcat-flume.conf

最后在一个新的窗口输入

nc localhost 44444

随便输入信息

数据会传到第三个窗口，也就是a3启动的窗口

将a3 使用CTRL+C停掉

稍等片刻，数据就会传输到a2所在的窗口中