Understanding this multi-threaded python daemon code
So I am new to python and am working on a file system event handler. I came across the watchdog api and there I saw multi-threaded code that I can't figure out.
Here is the code that is posted on their website:
import sys import time import logging from watchdog.observers import Observer from watchdog.events import LoggingEventHandler if __name__ == "__main__": logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S') path = sys.argv if len(sys.argv) > 1 else '.' event_handler = LoggingEventHandler() observer = Observer() observer.schedule(event_handler, path, recursive=True) observer.start() try: while True: time.sleep(1) except KeyboardInterrupt: observer.stop() observer.join()
This code starts an endless loop and listens to some folder and logs what it sees to the console. My doubt is the path towards the bottom of the code.
So you start as an observer. Then ask it to go into an endless loop until some click is made. My guess is that somewhere in the observer.start () code they also set daemon = True. With some pressing, the program ends and stops the observer. In the watchdog api, the stop () definition says it stops the daemon thread.
1) Then it executes join (). But what is the need for this unification. I have already stopped the demon flow. Not join () means wait for all threads to stop and then and only then exit the program. Can I remove join () from the code. After removing it, my program still works correctly.
2) I also don't understand the need for sleep (1) inside a while loop. What happens if I just put the "pass" statement there. I assume the while loop will consume more resources ??? And the reason we set the sleep time as 1 second rather than 2-3 seconds is because in the worst case, it may take the user 2-3 seconds to close the program. But I could be wrong.
source to share
Remember that the daemon is running in the parent process, process. You need to keep the parent process alive while this thread is running, otherwise it will be killed as the program exits (and probably not in a graceful way). This
ensures that the process stays alive until all the threads have actually exited; just because you called
does not guarantee that the thread has actually finished executing.
is a request to stop a thread, it does not require blocking until the thread finishes (and it should not be so that the parent thread can call
for many child threads at the same time).
This is purely for lower CPU consumption. If you just had
there, the CPU will run that and loop as fast as possible, with pauses. The call
voluntarily gives the CPU to other processes as it knows it doesn't need to react quickly to any specific conditions. And you're essentially right, it is
, so your worst response time is roughly 1 second.
Here's an example of why it's important to have
. Let's say the following was running on the thread:
while not self.stop: # self.stop is set to True when stop() is called ... self.results.append(item) # do some stuff that involves appending results to a list with open('~/output.txt', 'w') as outfile: outfile.write('\n'.join(str(item) for item in item))
, the while loop will end and the result file will open and recording will begin. If
not invoked, the process may complete before the operation completes
, resulting in corrupted results.
ensures that the parent thread is waiting for this write to complete. It also ensures that the process is actually waiting for a whole iteration of that while loop; without,
you can not only skip writing the file, but also end in the middle of this block
If, however, the thread it was referencing
didn’t do anything long after completing
effectively returned instantly and therefore basically turned into a NOP.
With regard to a sleep call, certain events (for example
) can get out of even the call
in the parent process. Therefore, in this particular case, the length of the sleep does not really matter. Setting it to 1 second is just a convention to make it clear that you are basically doing a "performance processor" and not just sleeping.
source to share