8.Processor

1.概述

Sink Group允许用户将多个Sink组合成一个实体。

Flume Sink Processor 可以通过切换组内Sink用来实现负载均衡的效果,或在一个Sink故障时切换到另一个Sink。

sinks – 用空格分隔的Sink集合

processor.type default 类型名称,必须是 default、failover 或 load_balance

2.Default Sink Processor

Default Sink Processor 只接受一个 Sink。

不要求用户为单一Sink创建processor

3.Failover Sink Processor

Failover Sink Processor 维护一个sink们的优先表。确保只要一个是可用的就事件就可以被处理。

失败处理原理是,为失效的sink指定一个冷却时间,在冷却时间到达后再重新使用。

sink们可以被配置一个优先级,数字越大优先级越高。

如果sink发送事件失败,则下一个最高优先级的sink将会尝试接着发送事件。

如果没有指定优先级,则优先级顺序取决于sink们的配置顺序,先配置的默认优先级高于后配置的。

在配置的过程中,设置一个group processor ,并且为每个sink都指定一个优先级。

优先级必须是唯一的。

另外可以设置maxpenalty属性指定限定失败时间。

sinks – Space-separated list of sinks that are participating in the group

processor.type default The component type name, needs to be failover

processor.priority.<sinkName> – Priority value. <sinkName> must be one of the sink instances associated with the current sink group A higher priority value Sink gets activated earlier. A larger absolute value indicates higher priority

processor.maxpenalty 30000 The maximum backoff period for the failed Sink (in millis)

Example for agent named a1:

------

a1.sinkgroups = g1

a1.sinkgroups.g1.sinks = k1 k2

a1.sinkgroups.g1.processor.type = failover

a1.sinkgroups.g1.processor.priority.k1 = 5

a1.sinkgroups.g1.processor.priority.k2 = 10

a1.sinkgroups.g1.processor.maxpenalty = 10000

------

4.Load balancing Sink Processor

Load balancing Sink processor 提供了在多个sink之间实现负载均衡的能力。

它维护了一个活动sink的索引列表。

它支持轮询 或 随机方式的负载均衡,默认值是轮询方式,可以通过配置指定。

也可以通过实现AbstractSinkSelector接口实现自定义的选择机制。

!processor.sinks – Space-separated list of sinks that are participating in the group

!processor.type default The component type name, needs to be load_balance

processor.backoff false Should failed sinks be backed off exponentially.

processor.selector round_robin Selection mechanism. Must be either round_robin, random or FQCN of custom class that inherits from AbstractSinkSelector

processor.selector.maxTimeOut 30000 Used by backoff selectors to limit exponential backoff (in milliseconds)

------

a1.sinkgroups = g1

a1.sinkgroups.g1.sinks = k1 k2

a1.sinkgroups.g1.processor.type = load_balance

a1.sinkgroups.g1.processor.backoff = true

a1.sinkgroups.g1.processor.selector = random

时间: 2024-12-15 06:56:14

8.Processor的相关文章

Processor Speculative &amp; pipeline

Branch-Prediction in a Speculative Dataflow Processor http://cseweb.ucsd.edu/~tullsen/mteac5/kuszmaul.pdf

Jafka来源分析——Processor

Jafka Acceptor接受client而建立后的连接请求,Acceptor会将Socket连接交给Processor进行处理.Processor通过下面的处理步骤进行client请求的处理: 1. 读取client请求. 2. 依据client请求类型的不同,调用对应的处理函数进行处理. Processor读取client请求是一个比較有意思的事情,须要考虑两个方面的事情:第一,请求规则(Processor须要依照一定的规则进行请求的解析).第二,怎样确定一次请求的读取已经结束(由于是非堵

Inter-partition communication in multi-core processor

A multi-core processor includes logical partitions that have respective processor cores, memory areas, and Ethernet controllers. At least one of the Ethernet controllers is disabled for external communication and is assigned as an inter-partition Eth

Flexible implementation of a system management mode (SMM) in a processor

A?system?management?mode?(SMM) of operating a processor includes only a basic set of hardwired hooks or mechanisms in the processor for supporting SMM. Most of SMM functionality, such as the processing actions performed when entering and exiting SMM,

Method and apparatus for transitioning between instruction sets in a processor

A data processor (104) is described. The data processor (104) is capable of decoding and executing a first instruction (212) of a first instruction set and a second instruction (213-219) in a second instruction set wherein the first instruction (212)

HP Microserver Gen8 Processor FAQ

http://homeservershow.com/forums/index.php?/topic/6596-hp-microserver-gen8-processor-faq/ This guide is a work in progress.  After reading some other people's questions about processors for the Microserver Gen8, including "What's the best choice for

Maintaining processor resources during architectural events

In one embodiment of the present invention, a method includes switching between a first address space and a second address space, determining if the second address space exists in a list of address spaces; and maintaining entries of the first address

PatentTips - Zero voltage processor sleep state

BACKGROUND Embodiments of the invention relate to the field of electronic systems and power management. More particularly, embodiments of the invention relate to a method and apparatus for a zero voltage processor sleep state. As the trend toward adv

PatentTips - Increasing turbo mode residency of a processor

BACKGROUND Many modern operating systems (OS's) use the Advanced Configuration and Power Interface (ACPI) standard, e.g., Rev. 3.0b, published Oct. 10, 2006, for optimizing system power. An ACPI implementation allows a processor core to be in differe