Optimal sleep time in multiple manufacturer / single consumer models
I am writing an application that has a multi-vendor model, one consumer model (multiple threads send messages to a single file write thread).
Each producer thread contains two queues, one for writing and one for reading by the consumer. Each consumer thread loop iterates through each producer and locks the producer mutex, swaps queues, unlocks, and writes from a queue that the producer no longer uses.
In a consumer thread loop, it sleeps for a certain amount of time after all producer threads have been processed. One thing I noticed right away was that the average time for a producer to write something in the queue and come back increased dramatically (by 5 times) when I went from 1 producer thread to 2. When more threads are added, this is the average time decreases until it is out - there is not much difference between the time spent with 10 manufacturers versus 15 manufacturers. This appears to be because with more producers to handle, there are fewer conflicts for the producer's thread mutex.
Unfortunately, 5 is a fairly common scenario for an application and I would like to optimize sleep times to get reasonable performance no matter how many manufacturers there are. I noticed that by increasing the sleep time, I can get better performance for low performance numbers, but worse performance for large manufacturers.
Has anyone else encountered this, and if so, what was your solution? I tried to scale sleep time with the number of threads, but it seems somewhat machine specific and quite tentative and buggy.
source to share
You can choose sleep times based on the number of producers, or even customize sleep times based on some dyanmic scheme. If the consumer wakes up and is not working, double the sleep time and halve it. But limit your sleep time to the minimum and maximum.
Either way, you are reworking a more fundamental problem. Sleep and interrogation is easy to get right and sometimes the only access available, but it has many flaws and is not the "right" way.
You can go in the right direction by adding a semaphore that grows when the producer adds an item to the queue and decreases when the consumer processes an item in the queue. The consumer will only wake up when there are items to process and will do so immediately.
Query polling can still be a problem. You can add a new queue that refers to any queue that has items. But rather the question arises as to why you don't have a single queue that the consumer processes, rather than a queue per producer. Everyone else is equal, which sounds like the best approach.
source to share
Instead of sleeping, I would recommend that your consumer block be in the state signaled by the manufacturers. On a posix compatible system, you can make it work with pthread_cond. Create an array pthread_cond_t
, one for each manufacturer, then create an additional one to be used between them. Manufacturers signal their individual condition variable first and then the general one. The user waits for the shared state and then iterates over the elements of the array, doing pthread_cond_timed_wait()
for each element of the array (use pthread_get_expiration_np()
to get the absolute time for "now"). If wait returns 0, then this producer has written the data. The consumer must re-initialize the condition variables before waiting again.
By using blocking waits, you minimize the time during which the consumer unnecessarily blocks producers. You can also make this work with semaphores as mentioned in the previous answer. Semaphores have simplified semantics over conditions in my opinion, but you have to be careful to reduce the total semaphore once per producer, which was processed on each pass through the consumer loop. Condition variables have the advantage that you can basically use them as boolean semaphores if you reinitialize them after they have been passed.
source to share
It seems to me that you accidentally introduce some buffering when the consumer thread is busy somewhere else, either sleeping or doing actual work. (a queue acting as a buffer) Perhaps doing simple producer-side buffering will reduce your contention.
It seems like your system is very sensitive to conflict between producer and consumer, but I'm confused as to why such a simple swap operation takes enough CPU time to show up in your startup statistics.
Can you show the code?
edit: maybe you take your lock and swap the queues even when there is no work to do?
source to share