Hadoop实战-Flume之Hello world(九)

环境介绍: 主服务器ip:192.168.80.128

1、准备apache-flume-1.7.0-bin.tar文件

2、上传到master(192.168.80.128)服务器上

3、解压apache-flume-1.7.0-bin.tar

 tar -zxvf apache-flume-1.7.0-bin.tar

4、进入到Flume的配置文件目录

cd /apache-flume-1.7.0-bin/conf

5、修改apache-flume-1.7.0-bin/conf的里面内容

a1.sources = r1 

a1.sinks = k1 

a1.channels = c1

a1.sources.r1.type = netcat 

a1.sources.r1.bind = localhost 

a1.sources.r1.port = 44444

a1.sinks.k1.type = logger

a1.channels.c1.type = memory

a1.channels.c1.capacity = 1000 

a1.channels.c1.transactionCapacity = 100

a1.sources.r1.channels = c1 

a1.sinks.k1.channel = c1

5、启动Flume程序

bin/flume-ng agent --conf conf --conf-file /home/hadoop/apache-flume-1.7.0-bin/conf/flume-conf.properties --name a1 -Dflume.root.logger=INFO,console

ps:如果后台启动,在步骤5的基础上再加入

-n agent1 > log.log 2>&1 &

6、在另外的窗口输入 telnet localhost 44444

时间: 2024-10-25 04:33:26

Hadoop实战-Flume之Hello world(九)的相关文章

Hadoop实战-Flume之自定义Sink(十九)

import java.io.File; import java.io.FileNotFoundException; import java.io.FileOutputStream; import java.io.IOException; import org.apache.flume.Channel; import org.apache.flume.Context; import org.apache.flume.Event; import org.apache.flume.EventDeli

Hadoop实战-Flume之Source replicating(十四)

a1.sources = r1 a1.sinks = k1 k2 a1.channels = c1 c2 # Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind = localhost a1.sources.r1.port = 44444 # Describe the sink a1.sinks.k1.type =file_roll a1.sinks.k1.sink.directory=/hom

Hadoop实战-Flume之Sink Load-balancing(十七)

a1.sources = r1 a1.sinks = k1 k2 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind = localhost a1.sources.r1.port = 44444 # Describe the sink a1.sinks.k1.type =file_roll a1.sinks.k1.sink.directory=/home/c

Hadoop实战-Flume之Hdfs Sink(十)

a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind = localhost a1.sources.r1.port = 44444 # Describe the sink #a1.sinks.k1.type = logger a1.sinks.k1.type = hdfs a1.sinks.k1.hd

Hadoop实战-Flume之自定义Source(十八)

import java.nio.charset.Charset; import java.util.HashMap; import java.util.Random; import org.apache.flume.Context; import org.apache.flume.EventDeliveryException; import org.apache.flume.PollableSource; import org.apache.flume.conf.Configurable; im

Hadoop实战-Flume之Source regex_filter(十三)

a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind = localhost a1.sources.r1.port = 44444 a1.sources.r1.interceptors =i5 #a1.sources.r1.interceptors.i4.type = timestamp #a1.so

Hadoop实战-Flume之Source interceptor(十一)(2017-05-16 22:40)

a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind = localhost a1.sources.r1.port = 44444 a1.sources.r1.interceptors = i1 i2 i3 a1.sources.r1.interceptors.i1.type = timestamp

Hadoop实战-Flume之Source regex_extractor(十二)

a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind = localhost a1.sources.r1.port = 44444 a1.sources.r1.interceptors = i1 i2 i3 i4 a1.sources.r1.interceptors.i4.type = timesta

Hadoop应用开发实战(flume应用开发、搜索引擎算法、Pipes、集群、PageRank算法)

Hadoop是2013年最热门的技术之一,通过北风网robby老师<深入浅出Hadoop实战开发>.<Hadoop应用开发实战>两套课程的学习,普通Java开发人员可以在最快的时间内提升工资超过15000.成为一位完全精通Hadoop应用开发的高端人才. Hadoop是什么,为什么要学习Hadoop? Hadoop是一个分布式系统基础架构,由Apache基金会开发.用户可以在不了解分布式底层细节的情况下,开发分布式程序.充分利用集群的威力高速运算和存储.Hadoop实现了一个分布式