Asynchronous Call Completion Procedure ReadDirectoryChangesW

I am trying to use ReadDirectoryChangesW

(async with completion routine) from python.

Following the notes MS Documentation I learned that an OVERLAPPED structure and callback should be provided.

The callback is described with this documentation as:

VOID CALLBACK FileIOCompletionRoutine(
  _In_     DWORD dwErrorCode,
  _In_     DWORD dwNumberOfBytesTransfered,
  _Inout_  LPOVERLAPPED lpOverlapped
);

      

        
        
        
      

    

So, I created an instance of ctypes

WINFUNCTYPE factory as:

RDCW_CALLBACK_F = ctypes.WINFUNCTYPE(None, ctypes.wintypes.DWORD, ctypes.wintypes.DWORD, ctypes.POINTER(OVERLAPPED))

      

        
        
        
      

    

And tried to access the call FileIOCompletionRoutine

:

ReadDirectoryChangesW = ctypes.windll.kernel32.ReadDirectoryChangesW

ReadDirectoryChangesW.restype = ctypes.wintypes.BOOL
ReadDirectoryChangesW.errcheck = _errcheck_bool
ReadDirectoryChangesW.argtypes = (
    ctypes.wintypes.HANDLE,  # hDirectory
    LPVOID,  # lpBuffer
    ctypes.wintypes.DWORD,  # nBufferLength
    ctypes.wintypes.BOOL,  # bWatchSubtree
    ctypes.wintypes.DWORD,  # dwNotifyFilter
    ctypes.POINTER(ctypes.wintypes.DWORD),  # lpBytesReturned
    ctypes.POINTER(OVERLAPPED),  # lpOverlapped
    RDCW_CALLBACK_F  # FileIOCompletionRoutine # lpCompletionRoutine
)

      

        
        
        
      

    

Note that if I try to use FileIOCompletionRoutine

as a synchronous call (passing None

to its last two parameters) it works.

The problem occurs when I try to complete the completion routine:

#Routine implementation
def dir_change_callback(dwErrorCode,dwNumberOfBytesTransfered,p):
    print("dir_change_callback!")

#Get WINFUNC from factory
call2pass = RDCW_CALLBACK_F(dir_change_callback)

try:
    event_buffer = ctypes.create_string_buffer(BUFFER_SIZE)
    nbytes = ctypes.wintypes.DWORD()
    overlapped_read_dir = OVERLAPPED()    
    ReadDirectoryChangesW(handle, ctypes.byref(event_buffer),
                          len(event_buffer), recursive,
                          WATCHDOG_FILE_NOTIFY_FLAGS,
                          ctypes.byref(nbytes), 
                          ctypes.byref(overlapped_read_dir), call2pass)

      

        
        
        
      

    

The call works, and if I don't make changes to the monitored directory while the program is running, it starts without fail. However, if a crash occurs, the program will crash:

hand = get_directory_handle(os.path.abspath("/test/"))
read_directory_changes(hand, False)
time.sleep(10)

      

        
        
        
      

    

Crash report

Nombre del evento de problema (Problem event name): APPCRASH
Nombre de la aplicación (Application Name):         python.exe
Nombre del módulo con errores (Module name):        ntdll.dll
Versión del módulo con errores (Module version):    6.1.7601.18247
Código de excepción (Exception code):               c0000005
Desplazamiento de excepción (Exception offset):     000337a2
Versión del sistema operativo (OS Version):         6.1.7601.2.1.0.256.48

      

        
        
        
      

    

I think the problem occurs when WinApi tries to invoke the callback code, so I suspect that I am not creating the WINFUNC

one passed to the call correctly .

On the other hand, I found the following warning on the python documentation site ctypes :

Note. Make sure you keep references to CFUNCTYPE () objects while they are used from C code. Ctypes doesnt, and if you don't want them, they can garbage collect, your program crashing on callbacks.

But I kept the global callable reference so if I'm not mistaken I don't think this is the problem.

Have you ever called kernel32

Windows calls passing callbacks implemented in python? Am I trying to get it right?

EDIT

I just figured out what caused the crash: event_buffer

it was not global and it was going before the callback started. Even though this is globally committed, no message "dir_change_callback!"

is printed on the process terminal. I tried to create a new file from the callback and write some kind of message to it, but the result was the same: It looks like the callback is not being executed . This is rather strange as the above crash must be related to the execution of the callback.

ctypes

the documentation says:

Also note that if the callback function is called on a created thread outside of Pythons control (for example, by external code that calls the callback), ctypes creates a new dummy Python thread on each invocation. This behavior is correct for most purposes, but it means that the values ​​stored in threading.local will not persist across different callbacks, even if those calls are made from the same C thread.

As a complete newbie ctypes

, my understanding is that this dummy thread is the thread of the process that made the system call, and it is possible that this process ends before the call is made. This question Using thread blocking in the ctypes callback seems to support this idea. I tried to reproduce this locking question in my code, but the locking remains unlocked just like the version trying to print: The callback doesn't seem to be executed:

lck = threading.Lock()
lck.acquire()

def dir_change_callback(dwErrorCode,dwNumberOfBytesTransfered,p):
    lck.release()