ConcurrentQueue contains a reference to an object or value? out of memory exception

Are the objects pushed to the ConcurrentQueue copied to the queue or just their references?

I don't understand any scenario.

Explanation:

I have defined ConcurrentQueue like this:

// BufferElement is a class I created
private ConcurrentQueue<BufferElement> _bufferQueue;

      

I have a function called many times and this purpsoe is to insert an item into the queue:

private void EnqueueElementToBuffer(string data, int moreData)
{
    // the bufferElement constructor is setting data and moreData to it fields.
    BufferElement bufferElement = new BufferElement(data, moreData);
    bufferQueue.Enqueue(bufferElement);
}

      

When I run this, I get an Out of Memory exception after a while. I thought it might be because the garbage collector is not collecting bufferElement

, because it still references bufferQueue

, so I changed the function to this:

private void EnqueueElementToBuffer(string data, int moreData)
{
    // _bufferElement is now a filed of the class
    _bufferElement.Data = data;
    _bufferElement.MoreData = moreData;
    bufferQueue.Enqueue(_bufferElement);
}

      

And I didn't get an exception and was not going to judge from memory in Windows Task Manager.

Now I thought the problem was solved because when I put objects in the queue, only the object reference is bound to the queue, but I was afraid that all the items in the queue refer to the same object, so I checked another function which I have in another thread what it does:

    // while bufferQueue is not empty do the following
    BufferElement bufferElement = null;
    bufferQueue.TryDequeue(out bufferElement);

      

And I checked the content of several elements and saw that their content was different! so if the objects pushed into the queue are copied by value, why did I get an Out of Memory exception first?

+3


source to share


2 answers


When called, Enqueue

only a copy of the link is saved in ConcurrentQueue<T>

. But this reference is tightly held, which means it keeps the actual object in memory. The item will not be eligible for collection until the link is removed fromConcurrentQueue<T>

The reason you didn't notice OutOfMemoryException

when switching to using the field is because you fundamentally changed the semantics

  • Original code: pressed N references to N items in the queue, hence it is holding N items in memory
  • Modified code: Pushed N references to 1 item into the queue, so it holds 1 item in memory

This greatly reduced the amount of memory held in the object's memory ConcurrentQueue<T>

and prevented the exception from being thrown.



It looks like the problem is that you are just performing operations much faster than processing them. As long as this is true, you will eventually run out of memory in your application. Your code should be set up so that it doesn't end up in this state.

Note. This answer is written if BufferElement

is class

, not a struct

.

Note 2. As Servy points out in the comments, you might want to consider switching to BlockingCollection<T>

as it has several throttling options that might help you.

+3


source


To answer the first question, ConcurrentQueue<T>

contains a link to content. This is true even for value types T

, since they are queued after being queued.

The reason you got the OOM to start with is probably because you either never deactivate the elements (and hence yes, the queue will keep references to them and keep them alive until you delete them ), or you're not fooling them fast enough compared to how quickly you queue them up.



In your "fix", you simply create only BufferElement

once and overwrite it constantly. You are injecting the same class instance over and over, which is almost certainly not the behavior you want.

+1


source







All Articles