Deadlock at Microsoft.Win32.Win32Native.CreateFile

We have a WCF service that reads and writes documents to network file storage (shared folder). Sometimes we see a situation where threads start getting stuck in Win32Native code. Once the thread is blocked, all subsequent calls are blocked as well. Interestingly, the problem persists after restarting the Windows service. The very first thread that tries to work with document storage units and the problem recurs. Only after restarting the server does the service continue to run without problems for some time (sometimes several days, sometimes several weeks).

This only happens in our production environment. I currently have no information about the state of the server at the time the problem occurred, about any backup processes or Windows event logs from that time. So any ideas on where to look and what might be causing this problem would be very helpful.

An example of code where threads are blocked is creating an empty file in the document store:

public void CreateFile( string filePath )
{
    using ( FileStream stream = File.Create( filePath ) )
    {
    }
}

      

        
        
        
      

    

Blocked threads stack trace:

   at Microsoft.Win32.Win32Native.CreateFile(String lpFileName, Int32 dwDesiredAccess, FileShare dwShareMode, SECURITY_ATTRIBUTES securityAttrs, FileMode dwCreationDisposition, Int32 dwFlagsAndAttributes, IntPtr hTemplateFile)
   at Microsoft.Win32.Win32Native.CreateFile(String lpFileName, Int32 dwDesiredAccess, FileShare dwShareMode, SECURITY_ATTRIBUTES securityAttrs, FileMode dwCreationDisposition, Int32 dwFlagsAndAttributes, IntPtr hTemplateFile)
   at Microsoft.Win32.Win32Native.SafeCreateFile(String lpFileName, Int32 dwDesiredAccess, FileShare dwShareMode, SECURITY_ATTRIBUTES securityAttrs, FileMode dwCreationDisposition, Int32 dwFlagsAndAttributes, IntPtr hTemplateFile)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize)
   at MyService.DocumentProcessingComponent.CreateFile(string filePath)
   . . . Custom Code . . .
   at MyService.Service.DoSomething(Message requestMessage)
   at SyncInvokeInvokeService(Object , Object[] , Object[] )
   at System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(Object instance, Object[] inputs, Object[]& outputs)
   at System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(MessageRpc& rpc)
   at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)
   at System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(RequestContext request, Boolean cleanThread, OperationContext currentOperationContext)
   at System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(RequestContext request, OperationContext currentOperationContext)
   at System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(IAsyncResult result)
   at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
   at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
   at System.ServiceModel.Channels.TransportDuplexSessionChannel.TryReceiveAsyncResult.OnReceive(IAsyncResult result)
   at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
   at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
   at System.ServiceModel.Channels.SynchronizedMessageSource.ReceiveAsyncResult.OnReceiveComplete(Object state)
   at System.ServiceModel.Channels.SessionConnectionReader.OnAsyncReadComplete(Object state)
   at System.ServiceModel.Channels.SocketConnection.OnReceiveAsync(Object sender, SocketAsyncEventArgs eventArgs)
   at System.Net.Sockets.SocketAsyncEventArgs.FinishOperationSuccess(SocketError socketError, Int32 bytesTransferred, SocketFlags flags)
   at System.Net.Sockets.SocketAsyncEventArgs.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
   at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)

      

        
        
        
      

    

Sometimes threads are blocked in another Win32Native method, as in the following example:

public ReadFileResult ReadFileBlock( string filePath, int blockIndex )
{
    if ( !File.Exists( filePath ) )
        return new ReadFileResult() { Error = "File not found" };

    byte[] content = null;
    int nextBlockIndex = 0;

    using ( FileStream fs = File.Open( filePath, FileMode.Open, FileAccess.Read, FileShare.Read ) )
    {
        int offset = _defaultBlockSize * blockIndex;

        if ( offset >= fs.Length )
            return new ReadFileResult() { Error = "Block index out of range" };

        fs.Seek( offset, SeekOrigin.Begin );
        int bytesToRead = ( int )Math.Min( _defaultBlockSize, fs.Length - fs.Position );
        content = new byte[ bytesToRead ];
        int bytesRead = fs.Read( content, 0, bytesToRead );

        if ( bytesRead < _defaultBlockSize || fs.Position == fs.Length )
            nextBlockIndex = -1;
        else
            nextBlockIndex = blockIndex + 1;
    }

    return new ReadFileResult() { Block = content, NextBlockIndex = nextBlockIndex };
}

      

        
        
        
      

    

Blocked threads stack trace:

   at Microsoft.Win32.Win32Native.GetFileAttributesEx(String name, Int32 fileInfoLevel, WIN32_FILE_ATTRIBUTE_DATA& lpFileInformation)
   at Microsoft.Win32.Win32Native.GetFileAttributesEx(String name, Int32 fileInfoLevel, WIN32_FILE_ATTRIBUTE_DATA& lpFileInformation)
   at System.IO.File.FillAttributeInfo(String path, WIN32_FILE_ATTRIBUTE_DATA& data, Boolean tryagain, Boolean returnErrorOnNotFound)
   at System.IO.File.InternalExists(String path)
   at System.IO.File.InternalExistsHelper(String path, Boolean checkHost)
   at MyService.DocumentProcessingComponent.ReadFileBlock(string filePath, int blockIndex)