Scale long processing of Azure Service Bus messages
What is the best way to scale up a worker role that handles many long Azure Service Bus messages using Message Message QueueClient Message.
If using QueueClient.OnMessageOptions.MaxConcurrentCalls = 6 and QueueClient.OnMessage does that mean I can handle a maximum of 6 messages at a time?
Is it really bad form to have long processing in the OnMessage callback to create a new task to finish processing it?
Should I use QueueClient.OnMessageAsync instead?
Thanks for any help.
source to share
By "long run" do you mean IO binding or CPU bound?
Assuming IO-bound, I would not create a new task in the OnMessage callback. This creates a flow control overhead that can slow down processing downscale.
Consider using OnMessageAsync if you are using IO bound operations and make sure you expect asynchronous implementations of any of these operations. This uses your existing streams much more efficiently.
If your operations are CPU bound, then creating a task can do more for you. The mechanics of this are discussed in a series of excellent posts by Stephen Cleary:
http://blog.stephencleary.com/2013/10/taskrun-etiquette-and-proper-usage.html
The MaxConcurrentCalls property controls the number of concurrent requests to Service Bus. Increasing this number has a limited impact if you are IO bound and limited by the available bandwidth. I would recommend doing a little performance testing with Azure Client Performance Counters to get the best value for your environment.
source to share