LMAX Disruptor – High Performance, Low Latency and Simple Too 转载

原文地址:http://www.symphonious.net/2011/07/11/lmax-disruptor-high-performance-low-latency-and-simple-too/

The LMAX disruptor is an ultra-high performance, low-latency message exchange between threads. It‘s a bit like a queue on steroids (but quite a lot of steroids) and is one of the key innovations used to make the LMAX exchange run so fast. There is a rapidly growing set of information about what the disruptor is, why it‘s important and how it works – a good place to start is the list of articles and for the on-going stuff, follow LMAX Blogs. For really detailed stuff, there‘s also the white paper (PDF).

While the disruptor pattern is ultimately very simple to work with, setting up multiple consumers with the dependencies between them can require a bit too much boilerplate code for my liking. To make it quick and easy for 99% of cases, I‘ve whipped up a simple DSL for the disruptor pattern. For example, to wire up a "diamond pattern" of consumers:

(Image blatantly stolen from Trisha Gee‘s excellent series explaining the disruptor pattern)

In this scenario, consumers C1 and C2 can process entries as soon as the producer (P1) puts them on the ring buffer (in parallel). However, consumer C3 has to wait for bothC1 and C2 to complete before it processes the entries. In real life this might be because we need to both journal the data to disk (C1) and validate the data (C2) before we do the actual business logic (C3).

With the raw disruptor syntax, these consumers would be created with the following code:

Executor executor = Executors.newCachedThreadPool();
BatchHandler handler1 = new MyBatchHandler1();
BatchHandler handler2 = new MyBatchHandler2();
BatchHandler handler3 = new MyBatchHandler3()
RingBuffer ringBuffer = new RingBuffer(ENTRY_FACTORY, RING_BUFFER_SIZE);
ConsumerBarrier consumerBarrier1 = ringBuffer.createConsumerBarrier();
BatchConsumer consumer1 = new BatchConsumer(consumerBarrier1, handler1);
BatchConsumer consumer2 = new BatchConsumer(consumerBarrier1, handler2);
ConsumerBarrier consumerBarrier2 =
ringBuffer.createConsumerBarrier(consumer1, consumer2);
BatchConsumer consumer3 = new BatchConsumer(consumerBarrier2, handler3);
executor.execute(consumer1);
executor.execute(consumer2);
executor.execute(consumer3);
ProducerBarrier producerBarrier =
ringBuffer.createProducerBarrier(consumer3);

We have to create our actual handlers (the two instances of MyBatchHandler), plus consumer barriers, BatchConsumer instances and actually execute the consumers on their own threads. The DSL can handle pretty much all of that setup work for us with the end result being:

Executor executor = Executors.newCachedThreadPool();
BatchHandler handler1 = new MyBatchHandler1();
BatchHandler handler2 = new MyBatchHandler2();
BatchHandler handler3 = new MyBatchHandler3();
DisruptorWizard dw = new DisruptorWizard(ENTRY_FACTORY, RING_BUFFER_SIZE, executor);
dw.consumeWith(handler1, handler2).then(handler3);
ProducerBarrier producerBarrier = dw.createProducerBarrier();

We can even build parallel chains of consumers in a diamond pattern: 

(Thanks to Trish for using her fancy graphics tablet to create a decent version of this image instead of my original finger painting on an iPad…)

dw.consumeWith(handler1a, handler2a);
dw.after(handler1a).consumeWith(handler1b);
dw.after(handler2a).consumeWith(handler2b);
dw.after(handler1b, handler2b).consumeWith(handler3);
ProducerBarrier producerBarrier = dw.createProducerBarrier();

The DSL is quite new so any feedback on it would be greatly appreciated and of course feel free to fork it on GitHub and improve it.

时间: 2024-11-05 11:54:29

LMAX Disruptor – High Performance, Low Latency and Simple Too 转载的相关文章

The LMAX disruptor Architecture--转载

原文地址: LMAX is a new retail financial trading platform. As a result it has to process many trades with low latency. The system is built on the JVM platform and centers on a Business Logic Processor that can handle 6 million orders per second on a sing

Optimizing web servers for high throughput and low latency

转自:https://blogs.dropbox.com/tech/2017/09/optimizing-web-servers-for-high-throughput-and-low-latency/ This is an expanded version of my talk at NginxConf 2017 on September 6, 2017. As an SRE on the Dropbox Traffic Team, I'm responsible for our Edge n

Log4j2 - java.lang.NoSuchMethodError: com.lmax.disruptor.dsl.Disruptor

问题 项目使用了log4j2,由于使用了全局异步打印日志的方式,还需要引入disruptor的依赖,最后使用的log4j2和disruptor的版本依赖如下: <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.11.1</version> </dependenc

lmax disruptor

http://www.cnblogs.com/haiq/p/4112689.html http://ifeve.com/disruptor-dsl/ http://www.iteye.com/topic/1129178 https://github.com/LMAX-Exchange/disruptor/ http://lmax-exchange.github.io/disruptor/ http://www.jdon.com/42452

XenServer 下一个版本Dundee Beta版发布

下一代的XenServer,研发代号为Dundee已于9月22日发布了Beta版本,在这一个版本XenServer相比之前在内核以及功能上做了相当程度的提升. 目前发布的主要功能如下: Intel GVT-g virtual GPU for Windows  (Intel vGPU solution) Software thin provisioning for SAN/Block storage Support for SMB/CFIS storage Support for offline

Windows-universal-samples学习笔记系列五:Custom user interactions

Custom user interactions Basic input Complex inking Inking Low latency input Simple inking Touch keyboard Basic input sample This sample shows how to handle input in Universal Windows Apps. Specifically, this sample shows how to: Listen for events on

System and method for dynamically adjusting to CPU performance changes

FIELD OF THE INVENTION The present invention is related to computing systems, and more particularly to a system and method for adjusting to changes in processor performance. BACKGROUND INFORMATION Designers of mobile computing platforms are faced wit

Zab: A simple totally ordered broadcast protocol(译)

摘要 这是一个关于ZooKeeper正在使用的全序广播协议(Zab)的简短概述.它在概念上很容易理解,也很容易实现,并且提供很高的性能.在这篇文章里,我们会呈现ZooKeeper在Zab上的需求,也会展示这个协议该如何使用,然后我们总体概述一下这个协议是如何工作的. 1. 简介 在雅虎(Yahoo!),我们开发了一款叫做ZooKeeper[9]的高性能高可用的协作服务,它允许大规模的应用群执行协作任务,比如Leader选举.状态传播和会合(rendezvous).该服务实现了一个层级的数据结点空

Java Performance Optimization Tools and Techniques for Turbocharged Apps--reference

Java Performance Optimization by: Pierre-Hugues Charbonneau reference:http://refcardz.dzone.com/refcardz/java-performance-optimization Java is among the most widely used programming languages in the software development world today. Java applications