What is the ideal size for the FileSystemWatcher InternalBufferSize?
I have a problem with the FileSystemWatcher file system.
I have an application that needs to track a large, really remarkable number of files that have been created in a folder in a short amount of time.
When I start developing it, I realize that there are many files where it was not notified if my buffer was less than 64KB, which is what Microsoft recommends. I tried increasing the buffer size above this until I reached the value that worked for me, which is 2,621,440 bytes!
What would you recommend using a small size for this case, or what was the ideal buffer size?
My example code:
WATCHER = new FileSystemWatcher(SignerDocument.UnsignedPath, "*.pdf");
WATCHER.InternalBufferSize = 2621440; //Great and expensive buffer 2.5mb size!
WATCHER.IncludeSubdirectories = true;
WATCHER.EnableRaisingEvents = true;
WATCHER.Created += new FileSystemEventHandler(watcher_Created);
WATCHER.Renamed += new RenamedEventHandler(watcher_Renamed);
And what Microsoft says about it in .NET 2.0:
Increasing the buffer size is expensive because it comes from inexperienced memory that cannot be swapped out to disk, so keep the buffer as small as possible. To avoid buffer overflows, use the NotifyFilter and IncludeSubdirectories Properties to filter out unwanted notification changes. reference: FileSystemWatcher.InternalBufferSize Property
source to share
For such a huge workload, you might want to choose the "periodic sweep" method instead of instant notifications. For example, you can scan a directory every 5 seconds and process the added files. If you move the file to a different directory after processing it, your intermittent workload may even be minimal.
It's also a safer approach because even if your processing code crashes, you can always recover, unlike notifications, your checkpoint won't get lost.
source to share
You can set the buffer to 4KB or more, but it must not exceed 64KB. If you try to set the InternalBufferSize property to less than 4096 bytes, your value will be discarded and the InternalBufferSize property will be set to 4096 bytes. For best performance, use multiple 4K computers on Intel-based computers.
source to share