Clever Geek Handbook

How can I create my own log instance for each thread?

For example, I have these files with classes that I want to use (just an example may not work)

# helper_one.py

import logging

logger = logging.getLogger('HelperOne')

class HelperOne:
    def __init__(self, data):
        self.data = data

    def run(self):
        logger.debug('HelperOne::Run function with data: ' + self.data)
        return True

...

# controller_one.py

import logging

from helper_one import HelperOne

logger = logging.getLogger('ControllerOne')

class ControllerOne:
    def __init__(self, data):
        self.data = data

    def run(self):
        logger.debug('ControllerOne::Run function with data: ' + self.data)

        ho = HelperOne(self.data + '_data')
        return ho.run()

And I have a file that creates Threads

import logging 

from controller_one import ControllerOne

# this function creates threads (its just an example)
def run_task(task_id, data):
    logging.basicConfig(
        filename=os.path.join('logs', str(task_id) + '.log'),
        level=logging.DEBUG,
        format='%(asctime)s\t%(name)s\t%(levelname)s\t%(message)s'
    )

    result = ControllerOne(data)

    logging.debug('run_task: ' + result)

If I create a log instance like in my example, the entire log is written to one file. How can I create its own logging instance for each thread, logging its own file?

Best regards, Alex.

+3


source to share


1 answer


The registrar name (the string passed to the function getLogger

) can be whatever you want. As the docs explain:

name

- a potentially hierarchical meaning separated by a period, for example foo.bar.baz

(although it could also be just simple foo

, for example) Registrars that are lower in the hierarchical list are children of the registrars higher in the list. For example, when registering a registrar with a name, foo

registrars with names foo.bar

, foo.bar.baz

and foo.bam

are descendants foo

. The log name hierarchy is similar to and identical to the Python package hierarchy if you organize the loggers on a per module basis using the recommended construction logging.getLogger(__name__)

. That's because in module __name__

is the name of modules in the Python package namespace.

So, using __name__

as a name is recommended, but not required. And in fact you are already explicitly breaking this:

logger = logging.getLogger('ControllerOne')

So, you can just use a separate name for each thread by putting the thread id or name in the log name. For example:



logger = logging.getLogger('ControllerOne.{}'.format(threading.get_ident())

Or, in your case, since you seem to have a unique one task_id

for each thread, you can simply use that instead of the thread id.

Of course, this means that you must actually use a registrar; you cannot just call logging.debug

. And you cannot rely on basicConfig

; you will need to explicitly configure the logger on each thread. Since each thread creates its own independent instance ControllerOne

, the obvious place to do this is in ControllerOne.__init__

; in this function you can call getLogger

with the name of the log that includes the thread or task id, create and set FileHandler

, which also includes the thread or the task id in the name and save it as self.logger

. Then when you need to record something, you just do self.logger.debug(…)

.

If this all sounds like gobbledegook to you because you don't know what it is FileHandler

, you probably need to read at least a basic tutorial in the HOWTO, but the advanced tutorial and cookbook are very helpful too.

+5


source






More articles:


All Articles