C # FileSystemWatcher: high CPU usage when no changes happen to directory

My case is as follows:

  • Files are created inside a directory by some process (about 10 files per minute with a maximum file size of 5MB each). Lets call this MYROOT folder.
  • So that files are moved and classified into subdirectories according to a certain logic based on the file name and external settings.
  • MYROOT will have 5 subdirectories with some subdirectories inside, let's call them C1, C2, C3, C4, C5. And some subdirectories inside them like A1, A2, A3, An ...
  • So, we will have the following classification: C: \ MYROOT \ C1 \ A1, C: \ MYROOT \ C1 \ A2, C: \ MYROOT \ C2 \ B1, etc.
  • All files are written to C: \ MYROOT and have the same file type and the same naming convention. Renaming, deletion, changes are not made to this folder.

FileSystemWatcher is installed as follows:

this._watcher = new FileSystemWatcher();
this._watcher.NotifyFilter = NotifyFilters.CreationTime;
this._watcher.Filter = string.Empty;
this._watcher.Path = @"C:\MYROOT";
this._watcher.IncludeSubdirectories = false;
this._watcher.Created += this._watcher_Created;

      

Event handler:

private void _watcher_Created(object sender, FileSystemEventArgs e)
{
    string filePath = string.Empty;

    if (e.ChangeType != WatcherChangeTypes.Created)
        return;

    lock (FileOrganizer.Manager.s_LockObject)
    {
        filePath = e.FullPath;

        //Further processing here like creating a custom object that 
        //contains the file name and some other info, add that object inside a queue
        //that is processed async with logic applied to wait the file to be written,
        //validate file contents, etc.. and finally move it where it should be moved.
        //The code here is very fast, exception free and in a fire-and-forget manner.
    }
}

      

When the MYROOT directory is empty and files are being created by another process, they are moved within folders as I described. Folders are created only once if they don't exist. The number of files that exist inside subfolders is increasing and we're talking ~ 200GB and counting.

Now listen: If no files are created (the "creator" process is not running) nothing should start the watcher, and I can see this from the logs that I include for debugging before posting this question. But my process containing the watcher reaches a constant 13 to 14% CPU on an octa-core CPU and grows as the size of the subdirectories grows. During processing, even if I create (copy-paste), 2000 files are 1% more at once. The funny part is that when I change the monitoring path to empty or the same but with less volume inside it, my process's CPU usage is 0%, and only when files are created does it reach the maximum value of 2% - 5%.

Again, as said, the observation becomes clear when the sub directories of the monitoring path contain data, this data affects the internals of the observer even if you have configured not to monitor the sub directories. And if that amount of data is large, high cpu resources are required for the file system watcher. It is the same even if no change occurs to trigger any observer events.

Is this normal or design behavior of FileSystemWatcher?

PS. When I control the MYROOT folder and move files outside in that folder, everything seems to be OK, but this is not an acceptable solution.

Thanx.

Marios

+3


source to share





All Articles