Writing to the same log file using Pyramid and rq
I am using Pyramid and rq and I would like to register rq work orders in the same file as the Pyramid application.
My setup looks like this:
development.ini
# regular pyramid setup here...
[loggers]
keys = root, main
[handlers]
keys = console, debuglog
[formatters]
keys = generic
[logger_root]
level = DEBUG
handlers = console, debuglog
[logger_main]
level = DEBUG
handlers =
qualname = main
[handler_debuglog]
class = handlers.RotatingFileHandler
args = ('%(here)s/__logs/pyramid_debug.log','a')
level = DEBUG
formatter = generic
[formatter_generic]
format = %(asctime)s %(levelname)-8s [%(name)s][%(threadName)s] %(message)s
main.py
from bunch import Bunch
from redis import Redis
from rq import Queue, Worker
from helpers.rqworker import run_worker
from helpers.log import write_log
write_log('info', 'APP STARTED.')
q = Queue(connection=Redis())
req_obj = Bunch()
req_obj.remote_addr = self.request.remote_addr
job = q.enqueue(run_worker, req_obj, {'some_other': 'data'})
write_log('info', 'APP ENDED.')
helpers /rqworker.py
from helpers.log import write_log
def run_worker(req, data):
write_log(req, 'debug', 'WORKER STARTED', separator=True)
helpers /log.py
import logging
log = logging.getLogger(__name__)
def write_log(request, loglevel, message=None, **kwargs):
msg = '[{0}] {1}'.format(request.remote_addr, message)
level = logging.getLevelName(loglevel.upper())
if 'separator' in kwargs:
log.log(level, '='*40)
if message:
log.log(level, msg)
if 'stack' in kwargs:
log.log(level, '{0} EXCEPTION {0}'.format('*'*15))
log.log(level, kwargs['stack'])
log.log(level, '{0} EXCEPTION END {0}'.format('*'*13))
When I run my application, I cannot see the logs from rqworker.py
(possibly because it is a separate thread), so I wonder how I can (and possibly) achieve writing to the same log file?
source to share
This is most likely not helpful to the OP, but the way to redirect the output of rq Worker to a custom file is to register it like this:
logger = logging.getLogger('rq.worker')
You can add a new FileHandler to this logger that outputs to your file and change the formatting too:
h = logging.FileHandler("/path/to/logfile.log")
formatter = logging.Formatter(...)
fh.setFormatter(formatter)
logger.addHandler(fh)
I have not been able to remove the coloring of the output, though, which causes the log to clutter up the ANSI color codes a bit when opened in a text editor.
Note that this removes the logging to the console. Perhaps you can keep this as well by adding another handler.
source to share