Django Logging Rotating files not working
So I'm having trouble logging into Django when it reaches size maxBytes
. Basically, when this happens, the file is not rotated and does not create a new file.
Someone told me that it might have something to do with server write permissions, but I'm not sure how to set this correctly so that django can create a new log file when the old one is full.
my settings:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.RequireDebugFalse'
}
},
'formatters': {
'verbose': {
'format': "[%(asctime)s] %(levelname)s [%(name)s:%(lineno)s] %(message)s",
'datefmt': "%d/%b/%Y %H:%M:%S"
},
'simple': {
'format': '[%(levelname)-7s] %(asctime)s - %(message)s'
},
},
'handlers': {
'mail_admins': {
'level': 'ERROR',
'filters': ['require_debug_false'],
'class': 'django.utils.log.AdminEmailHandler'
},
'console': {
'level': 'INFO',
'class': 'logging.StreamHandler'
},
'boom_file': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'maxBytes': 1024*1024*10, # 10 MB
'backupCount': 10,
'filename': '/var/log/boom.log',
'formatter': 'simple'
},
'binglaw_crawler_file': {
'level': 'DEBUG',
'class': 'logging.handlers.RotatingFileHandler',
'maxBytes': 1024*1024*10, # 10 MB
'backupCount': 10,
'filename': '/var/log/boom-binglaw-crawler.log',
'formatter': 'simple'
},
},
'loggers': {
'django.request': {
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
'boom': {
'handlers': ['console', 'boom_file'],
'propagate': True,
'level': 'DEBUG',
},
'boom.binglaw_crawler': {
'handlers': ['binglaw_crawler_file', ],
'propagate': False,
'level': 'DEBUG',
},
}
}
I noticed that my other celeryd log seems to o be rotinging just fine .. isn't that weird?
-rw-r--rw- 1 root root 10485721 Aug 18 12:12 boom-binglaw-crawler.log
-rw-r--r-- 1 root root 403506 Nov 8 23:42 boom-celeryd.log
-rw-r--r-- 1 root root 20201 Oct 2 12:47 boom-celeryd.log.1
-rw-r--rw- 1 root root 1049478 Oct 1 18:49 boom-celeryd.log.2
UPDATE:
I am getting this error when I try to run a control command that creates a log file
Traceback (most recent call last):
File "/usr/lib/python2.7/logging/handlers.py", line 77, in emit
self.doRollover()
File "/usr/lib/python2.7/logging/handlers.py", line 142, in doRollover
os.rename(self.baseFilename, dfn)
OSError: [Errno 13] Permission denied
source to share
When using "logging.handlers.RotatingFileHandler" in Django registration, I had this error:
Traceback (most recent call last):
File "C:\Python33\lib\logging\handlers.py", line 73, in emit
self.doRollover()
File "C:\Python33\lib\logging\handlers.py", line 176, in doRollover
self.rotate(self.baseFilename, dfn)
File "C:\Python33\lib\logging\handlers.py", line 116, in rotate
os.rename(source, dest)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'log.txt.1'
This error was happening to me because when I start Django there seem to be 2 different processes running. Both processes set up logging, which means that both get a handle to the same LOGGING files defined in the settings.py config file.
I added this line to my settings.py file before setting my LOGGING variable.
print("Initializing LOGGING in settings.py - if you see this more than once use 'runserver --noreload'")
If you run the application with the "manage.py runningserver - noreload " option , it can cure the file conflict problem.
source to share
One solution that currently works for me was to use 'delay': True
handlers for identifiers that write to files (in my case RotatingFileHandler
). This only creates files when the first one emit()
is called from the registrars, and I understand that the thread conflict between the main thread and the autoload thread (which is causing the error) does not seem to exist at this point.
source to share
I had the same problem using TimedRotatingFileHandler in combination with a local development server.
My solution that doesn't provide a flag --noreload
:
if os.environ.get('RUN_MAIN') == "true":
logging.config.dictConfig(LOGGING)
logger = logging.getLogger(__name__)
else:
logger = logging.getLogger('root')
It's not 100% clean as the second process gets a different logger, but for me I wanted to enable the auto reboot feature.
source to share