How to deal with high threading usage due to the many AJAX requests in WebSphere
I was told during stress testing that I need to reduce the use of the app within the app.
All of our other apps follow 1 request per page model while I build a single page app that fires 8 AJAX requests on load.
I am working on Websphere 8.5.5.1 (Servlet 3.0).
I am making 8 AJAX requests from the browser one right after the other up to the same Servlet, so they are in Parallel.
The servlet basically checks the request parameters and calls the corresponding ESB service over http and then returns the response.
Please correct me if anything I say here is rubbish as I am not 100% on any of them ...
My understanding is that when a request hits Websphere it gets a thread from the thread pool and uses that thread to execute the Servlet and keeps it on the same thread until a response is returned. I also think the thread will block while waiting for a response from the ESB. Is it correct?
I am considering the following 3 options:
1) The best thing we could do with Async servlets in Servlet 3.0 is to queue the request and put the thread back into the pool. Then one or more threads will pass through the queue serving the requests. So we just swap one thread pool for another, but we can limit / control the size of the second.
If we had Servlet 3.1 (JSR 340), we could use a ReadListener, and I guess we could avoid blocking threads while waiting for an ESB response?
2) Queue AJAX requests on Frontend, so instead of dropping 8 AJAX requests, I fire 3 and when they finish firing next and so on, so you never have more than three requests in parallel.
3) Make 1 AJAX request to the servlet and let it call all 8 ESB services and merge the results.
Can you advise me if any of the above solutions are valid, and if I even fully understand them and suggest other possible solutions?
Thank.
source to share
Tries to answer point by point:
- "Websphere obtains a thread from the thread pool and uses that thread to execute the Servlet and holds on to the same thread until a response is returned." Your understanding is wrong. This would be correct if WS was using old IO, but WS version 6.1 uses Native IO (NIO) and Asynchronous IO (AIO) , which allows multiple threads to serve thousands of concurrent connections. See Link. This way you shouldn't worry about making parallel connections with the AJAX client, and parallelism is fine .
- Having said that, the application should try and avoid performing any blocking operations that block any worker threads that might otherwise be handling multiple concurrent connections. Think about how you can do each async task, which would mean you can do something without blocking yourself. In your case, you are expecting (and blocking) responses from the ESB. Connection to ESB service ~ 8 will be blocked. Check how you are calling the ESB and if you can use NIO for these calls. Understand that WS is a client for calls in the ESB and it must use non-blocking IO for the operations it does in the ESB.
- Alternatively: is there a way for the ESB to also behave async and free up the WS request stream? The ESB can call back the URL of the WS service when it's done with the request, and then you can create an AJAX response. This way you won't be supporting WS streams.
In your options above, it is clear that # 1 and # 3 will not help you. # 2 will of course limit concurrent ESB operations to three. But I would recommend looking to see if ESB calls can be asynchronous and non-blocking. It would be perfect.
Update
To make asynchronous and non-blocking calls to an external service from your application server, consider using a library such as async-http-client . Netty is used by default, but other NIO structures can also be connected. The README.md of this project has good examples of how to make an HTTP call and write a response to complete an HTTP request. You can use onCompletion
to reply to your customer response.
Final statement:
- Be a non-blocking server when listening to your clients. Whenever possible, make sure the container / server request threads are not blocked.
- Be a non-blocking client when accessing an external service. This will allow you to make a large number of requests with multiple threads.
source to share