How to use Asynchronous Servlets to improve perfor

原文:https://plumbr.eu/blog/java/how-to-use-asynchronous-servlets-to-improve-performance

下文中第一个例子是每个servlet中sleep一个0到2000毫秒的随机数。Jmeter测试性能时候是并行去请求,可以认为这些servlet线程并行的sleep了不到2000ms。所以第二个异步的例子里,只sleep 2000ms了一次。

这里的异步其实就是servlet把请求交给一个线程池去处理,然后servlet线程结束。线程池慢慢地去处理这些任务。

This post is going to describe a performance optimization technique applicable to a common problem related to modern webapps. Applications nowadays are no longer just passively waiting for browsers to initiate requests, they want to start  communication themselves. A typical example would involve chat applications, auction houses etc – the common denominator being the fact that most of the time the connections with the browser are idle and wait for a certain event being triggered.

This type of applications has developed a problem class of their own, especially when facing heavy load. The symptoms include starving threads, suffering user interaction, staleness issues etc.

Based on recent experience with this type of apps under load, I thought it would be a good time to demonstrate a simple solution. After Servlet API 3.0 implementations became mainstream, the solution has become truly simple, standardized and elegant.

But before we jump into demonstrating the solution, we should understand the problem in greater detail. What could be easier for our readers than to explain the problem with the help of some source code:

@WebServlet(urlPatterns = "/BlockingServlet")
public class BlockingServlet extends HttpServlet {
  private static final long serialVersionUID = 1L;

  protected void doGet(HttpServletRequest request,
                       HttpServletResponse response) throws ServletException, IOException {
    try {
      long start = System.currentTimeMillis();
      Thread.sleep(new Random().nextInt(2000));
      String name = Thread.currentThread().getName();
      long duration = System.currentTimeMillis() - start;
      response.getWriter().printf("Thread %s completed the task in %d ms.", name, duration);
    } catch (Exception e) {
      throw new RuntimeException(e.getMessage(), e);
    }
  }

The servlet above is an example of how an application described above could look like:

  • Every 2 seconds some event happens. E.g. stock quote arrives, chat is updated and so on.
  • End-user request arrives, announcing interest to monitor certain events
  • The thread is blocked until the next event arrives
  • Upon receiving the event, the response is compiled and sent back to the client

Let me explain this waiting aspect. We have some external event that happens every 2 seconds. When new request from end-user arrives, it has to wait some time between 0 and 2000ms until next event. In order to keep it simple, we have emulated this waiting part with a call to Thread.sleep() for random number of ms between 0 and 2000. So every request waits on average for 1 second.

Now – you might think this is a perfectly normal servlet. In many cases, you would be completely correct – there is nothing wrong with the code until the application faces significant load.

In order to simulate this load I created a fairly simple test with some help from JMeter, where I launched 2,000 threads, each running through 10 iterations of bombarding the application with requests to the /BlockedServlet. Running the test with the deployed servlet on an out-of-the-box Tomcat 7.0.42 I got the following results:

  • Average response time: 9,324 ms
  • Minimum response time: 5 ms
  • Maximum response time: 11,651 ms
  • Throughput: 193 requests/second

The default Tomcat configuration has got 200 worker threads which, coupled with the fact that the simulated work is replaced with the sleep cycle of average duration of 1000ms, explains nicely the minimum and maximum response times – in each second the 200 threads should be able to complete 200 sleep cycles, 1 second on average each. Adding context switch costs on top of this, the achieved throughput of 193 requests/second is pretty close to our expectations.

The throughput itself would not look too bad for 99.9% of the applications out there. However, looking at the maximum and especially average response times the problem starts to look more serious. Getting the worst case response in 11 seconds instead of the expected 2 seconds is a sure way to annoy your users.

Let us now take a look at an alternative implementation, taking advantage of the Servlet API 3.0 asynchronous support:

@WebServlet(asyncSupported = true, value = "/AsyncServlet")
public class AsyncServlet extends HttpServlet {
  private static final long serialVersionUID = 1L;

  protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    Work.add(request.startAsync());
  }
}
public class Work implements ServletContextListener {
  private static final BlockingQueue queue = new LinkedBlockingQueue();

  private volatile Thread thread;

  public static void add(AsyncContext c) {
    queue.add(c);
  }

  @Override
  public void contextInitialized(ServletContextEvent servletContextEvent) {
    thread = new Thread(new Runnable() {
      @Override
      public void run() {
        while (true) {
          try {
            Thread.sleep(2000);
            AsyncContext context;
            while ((context = queue.poll()) != null) {
              try {
                ServletResponse response = context.getResponse();
                response.setContentType("text/plain");
                PrintWriter out = response.getWriter();
                out.printf("Thread %s completed the task", Thread.currentThread().getName());
                out.flush();
              } catch (Exception e) {
                throw new RuntimeException(e.getMessage(), e);
              } finally {
                context.complete();
              }
            }
          } catch (InterruptedException e) {
            return;
          }
        }
      }
    });
    thread.start();
  }

  @Override
  public void contextDestroyed(ServletContextEvent servletContextEvent) {
    thread.interrupt();
  }
}

This bit of code is a little more complex, so maybe before we start digging into solution details, I can outline that this solution performed ~75x better latency- and ~10x better throughput-wise. Equipped with the knowledge of such results, you should be more motivated to understand what is actually going on in the second example.

The servlet itself looks truly simple. Two facts are worth outlining though, the first of which declares the servlet to support asynchronous method invocations:

@WebServlet(asyncSupported = true, value = "/AsyncServlet")

The second important aspect is hidden in the following line

Work.add(request.startAsync());

in which the whole request processing is delegated to the Work class. The context of the request is stored using an AsyncContext instance holding the request and response that were provided by the container.

Now, the second and more complex class – the Work implemented as ServletContextListener – starts looking simpler. Incoming requests are just queued in the implementation to wait for the notification – this could be an updated bid on the monitored auction or the next message in the group chat that all the requests are waiting for.

Now, the notification arrives every 2 seconds and we have simplified this as just waiting with Thread.sleep(). When it arrives all the blocked tasks in the queue are processed by a single worker thread responsible for compiling and sending the responses. Instead of blocking hundreds of threads to wait behind the external notification, we achieved this in a lot simpler and cleaner way – batching the interest groups together and processing the requests in a single thread.

And the results speak for themselves – the very same test on the very same Tomcat 7.0.42 with default configuration resulted in the following:

  • Average response time: 265 ms
  • Minimum response time: 6 ms
  • Maximum response time: 2,058 ms
  • Throughput: 1,965 requests/second

The specific case here is small and synthetic, but similar improvements are achievable in the real-world applications.

Now, before you run to rewrite all your servlets to the asynchronous servlets – hold your horses for a minute. The solution works perfectly on a subset of usage cases, such as group chat notifications and auction house price alerts. You will most likely not benefit in the cases where the requests are waiting behind unique database queries being completed. So, as always, I must reiterate my favorite performance-related recommendation – measure everything. Do not guess anything.

Not sure if threads behave or are causing problems? Let Plumbr monitor your Java app and tell you if you need to change your code.

But on the occasions when the problem does fit the solution shape, I can only praise it. Besides the now-obvious improvements on throughput and latency, we have elegantly avoided possible thread starvation issues under heavy load.

Another important aspect – the approach to asynchronous request processing is finally standardized. Independent of your favorite Servlet API 3.0 – compliant application server such as Tomcat 7, JBoss 6 or Jetty 8 – you can be sure the approach works. No more wrestling with different Comet implementations or platform-dependent solutions, such as the Weblogic FutureResponseServlet.

时间: 2024-10-13 11:37:52

How to use Asynchronous Servlets to improve perfor的相关文章

Java性能提示(全)

http://www.onjava.com/pub/a/onjava/2001/05/30/optimization.htmlComparing the performance of LinkedLists and ArrayLists (and Vectors) (Page last updated May 2001, Added 2001-06-18, Author Jack Shirazi, Publisher OnJava). Tips: ArrayList is faster than

HttpWebRequest - Asynchronous Programming Model/Task.Factory.FromAsyc

Posted by Shiv Kumar on 23rd February, 2011 The Asynchronous Programming Model (or APM) has been around since .NET 1.1 and is still (as of .NET 4.0) the best/recommended solution for asynchronous I/O. Many people go down the route of using a multi-th

Improve Scalability With New Thread Pool APIs

Pooled Threads Improve Scalability With New Thread Pool APIs Robert Saccone Portions of this article are based on a prerelease version of Windows Server 2008. Details contained herein are subject to change. Code download available at: VistaThreadPool

转-Asynchronous bulk transfer using libusb

https://falsinsoft.blogspot.jp/2015/02/asynchronous-bulk-transfer-using-libusb.html The 'linusb' is one of the most used open source library for work with usb devices. It allow to make the basic usb data transfers operation in a little bit easier way

Asynchronous Pluggable Protocols 初探

Asynchronous Pluggable Protocols,异步可插入协议,允许开发者创建可插协议处理器,MIME过滤器,以及命名空间处理器工作在微软IE4.0浏览器以及更高版本或者URL moniker中.这涉及到Urlmon.dll动态链接库所公开(输出)的可插协议诸多功能,本文不进行深入的原理讲解,只对它其中之一的应用进行解析,那就是如何将一个应用程序注册为URL协议. 应用场景: tencent协议: 当我们打开"tencent://message/?uin=要链接的QQ号 &qu

ajax(Asynchronous JavaScript + XML) 技术学习

参考文档:https://developer.mozilla.org/en-US/docs/AJAX 本文进行了大致翻译. Ajax 本身本不是一门技术,而是在2005年由Jesse James Garrett首创的描述为一个"新"途径来应用许多已存在的技术,包括:HTML 或者 XHTML, Cascading Style Sheets, JavaScript, The Document Object Model, XML, XSLT, 和最重要的 XMLHttpRequest ob

Asynchronous function in a while-loop

I have a question about how to perform an asynchronous task in a while-loop until some condition is met. This is more of a theoretical question but I can see how this could be an issue in some scenarios. I'll try demonstrate the problem at an example

Async/Await - Best Practices in Asynchronous Programming

https://msdn.microsoft.com/en-us/magazine/jj991977.aspx Figure 1 Summary of Asynchronous Programming Guidelines Name Description Exceptions Avoid async void Prefer async Task methods over async void methods Event handlers Async all the way Don’t mix

(译)Asynchronous programming and Threading in C# (.NET 4.5)

原文地址:http://www.codeproject.com/Articles/996857/Asynchronous-programming-and-Threading-in-Csharp-N 介绍: Asynchronous programming and threading is very important feature for concurrent or parallel programming. Asynchronous programming may or may not us