Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of log data. It has a simple and flexible architecture based on streaming data flows. It is robust and fault tolerant with tunable reliability mechanisms and many failover and recovery mechanisms. It uses a simple extensible data model that allows for online analytic application.
The Apache Flume team is pleased to announce the release of Flume 1.9.0.
Flume is a distributed, reliable, and available service for efficiently collecting, aggregating, and moving large amounts of streaming event data.
Version 1.9.0 is the eleventh Flume release as an Apache top-level project. Flume 1.9.0 is stable, production-ready software, and is backwards-compatible with previous versions of the Flume 1.x codeline.
Several months of active development went into this release: about 70 patches were committed since 1.8.0, representing many features, enhancements, and bug fixes. While the full change log can be found on the 1.9.0 release page (link below), here are a few new feature highlights:
System Requirements Java Runtime Environment - Java 1.8 or later Memory - Sufficient memory for configurations used by sources, channels or sinks Disk Space - Sufficient disk space for configurations used by channels or sinks Directory Permissions - Read/Write permissions for directories used by agent
tar -zxf apache-flume-1.9.0-bin.tar.gz -C /home/hadoop/opt/app
查看当前的目录结构：
测试Flume：
在安装目录下的conf目录下新建空文件netcat2logger.conf
cd /home/hadoop/opt/opp/apache-flume-1.9.0-bin/conf
vi netcat2logger.conf
加入一下内容：
# example.conf: A single-node Flume configuration
# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 6666
# Describe the sink
a1.sinks.k1.type = logger
# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
# list the sources, sinks and channels for the agent
.sources = .sinks = .channels =
# set channel for source
.sources..channels = ...
# set channel for sink
.sinks..channel =
