High CPU usage with multithreaded HTTP requests with HTTPUrlConnection

I am making multithreaded HTTP requests with a small Java library called Scribe. Scribe uses HTTPUrlConnection

to execute the request and read the InputStream

String. The buffer size is 65kb, although the response size is usually 2MB, but I doubt it contributes.

When I start 10 threads, each doing HTTP requests, my CPU usage is exceeded on all 4 cores. From profiling I can see that about 60% of the CPU is spent calling HttpUrlConnection.getResponseCode();

(presumably waiting for a response), and about 20% of my time is being spent calling InputStream.read()

in StreamUtils .

In jVisualVM, I can see that threads are "running" all the time. I figured out that the base class would have a way to block the thread on I / O to free up CPU resources, but I think my understanding here is a little shaky.

What's a more efficient way to make concurrent HTTP requests?

+3


source to share





All Articles