In the return function, can you ensure that the finalizer is called on the same thread?
I have a tricky problem that comes up in some of my codes. I have a cache manager that either returns items from the cache or calls a delegate to create them (expensive).
I find that I am having trouble completing part of my method running on a different thread than the rest.
Here's the cut version
public IEnumerable<Tuple<string, T>> CacheGetBatchT<T>(IEnumerable<string> ids, BatchFuncT<T> factory_fn) where T : class
{
Dictionary<string, LockPoolItem> missing = new Dictionary<string, LockPoolItem>();
try
{
foreach (string id in ids.Distinct())
{
LockPoolItem lk = AcquireLock(id);
T item;
item = (T)resCache.GetData(id); // try and get from cache
if (item != null)
{
ReleaseLock(lk);
yield return new Tuple<string, T>(id, item);
}
else
missing.Add(id, lk);
}
foreach (Tuple<string, T> i in factory_fn(missing.Keys.ToList()))
{
resCache.Add(i.Item1, i.Item2);
yield return i;
}
yield break; // why is this needed?
}
finally
{
foreach (string s in missing.Keys)
{
ReleaseLock(l);
}
}
}
Acquiring and releasing a lock fills the dictionary with LockPoolItem objects that were locked with Monitor.Enter / Monitor.Exit [I've also tried mutexes]. The problem occurs when ReleaseLock is called on a different thread from the one on which AcquireLock was called.
The problem comes when you call this from another function that uses threads, sometimes the finalize block being called gets called due to the removal of the IEnumerator executing on the returned iteration.
The following example is a simple example.
BlockingCollection<Tuple<Guid, int>> c = new BlockingCollection<Tuple<Guid,int>>();
using (IEnumerator<Tuple<Guid, int>> iter = global.NarrowItemResultRepository.Narrow_GetCount_Batch(userData.NarrowItems, dicId2Nar.Values).GetEnumerator()) {
Task.Factory.StartNew(() => {
while (iter.MoveNext()) {
c.Add(iter.Current);
}
c.CompleteAdding();
});
}
This doesn't seem to be happening when I add an output break - however I am having a hard time debugging it as it happens very rarely. It does happen, however - I've tried registering thread IDs and finalizing if you call different threads ...
I'm pretty sure this is wrong behavior: I don't understand why the dispose method (like exit using) would be called on a different thread.
Any ideas how to protect against this?
source to share
There seems to be a race here.
It looks like your calling code creates an enumerator, then starts a task on the thread pool to enumerate it, and then removes the enumerator. My initial thoughts:
-
If the enumerator is set before the enumeration starts, nothing will happen. From a short test, this does not prevent the enum after being posted.
-
If the enumeration is located during enumeration, the finally block (on the calling thread) will be called and the enumeration will stop.
-
If the enumeration is completed by a task action, the finally block will be called (on the thread pool thread).
To try and demonstrate, consider this method:
private static IEnumerable<int> Items()
{
try
{
Console.WriteLine("Before 0");
yield return 0;
Console.WriteLine("Before 1");
yield return 1;
Console.WriteLine("After 1");
}
finally
{
Console.WriteLine("Finally");
}
}
If you choose before enumeration, nothing will be written to the console. This is what I suspect you will be doing most of the time, since the current thread reaches the end of the block using
before the task starts:
var enumerator = Items().GetEnumerator();
enumerator.Dispose();
If the enumeration is completed before Dispose
, the final call MoveNext
will call the block finally
.
var enumerator = Items().GetEnumerator();
enumerator.MoveNext();
enumerator.MoveNext();
enumerator.MoveNext();
Result:
"Before 0"
"Before 1"
"After 1"
"Finally"
If you choose when listing, the call Dispose
will call the block finally
:
var enumerator = Items().GetEnumerator();
enumerator.MoveNext();
enumerator.Dispose();
Result:
"Before 0"
"Finally"
I would suggest you create, list, and delete an enumerator in the same thread.
source to share
Thanks to all the answers, I figured out what was happening and why. The solution to my problem was pretty simple. I just had to make sure everything was called in the same thread.
BlockingCollection<Tuple<Guid, int>> c = new BlockingCollection<Tuple<Guid,int>>();
Task.Factory.StartNew(() => {
using (IEnumerator<Tuple<Guid, int>> iter = global.NarrowItemResultRepository.Narrow_GetCount_Batch(userData.NarrowItems, dicId2Nar.Values).GetEnumerator()) {
while (iter.MoveNext()) {
c.Add(iter.Current);
}
c.CompleteAdding();
}
});
source to share
The term "finalizer" refers to a concept completely unrelated to the "last" block; nothing contains the context of finalizers, but I think you are really interested in "finally" blocks.
A finally
block surrounded yield return
will be executed by any thread calls Dispose
in the iterator enumerator. Enumerators usually have the right to assume that all operations performed on them, including Dispose
, will be performed by the same thread that created them, and, as a rule, they are not required to behave in anything that even remotely resembles a reasonable way in cases where they are not. The system does not prevent code from using counters across multiple threads, but in the event that a program uses counters across multiple threads, an enumerator that makes no promises that it will work in this regard means that any consequences that follow from doing so are not an enumerator error, but rather an error of the program that used it illegally.
Typically, it is sufficient for classes to enable sufficient invalid threading protection to ensure that improper multithreading does not lead to security vulnerabilities, but does not worry about preventing any other kind of harm or confusion.
source to share