MongoDB, log cannot logrotate
MongoDB v2.6.3 version on my server and mongod is running:
ubuntu@koala:/var/log/mongodb$ ps -ef | grep mongo
root 7434 1 17 Jun16 ? 06:57:26 mongod -f /etc/mongodb-onepiece.conf --fork
I am using logrotate to rotate MongoDB log file daily. Weird problem came up with logrotate. I check the log file:
ubuntu@koala:/var/log/mongodb$ ls -lth | grep mongodb
-rw-r--r-- 1 ubuntu ubuntu 1.9G Jun 18 10:23 mongodb-onepiece.log.1
-rw-r--r-- 1 ubuntu ubuntu 0 Jun 17 07:35 mongodb-onepiece.log
-rw-r--r-- 1 ubuntu ubuntu 838M Jun 15 07:35 mongodb-onepiece.log.3.gz
-rw-r--r-- 1 ubuntu ubuntu 22 Jun 14 20:52 mongodb-onepiece.log.2.gz
-rw-r--r-- 1 ubuntu ubuntu 1.1G Jun 4 17:10 mongodb-onepiece.log.4.gz
-rw-r--r-- 1 ubuntu ubuntu 53M May 29 19:14 mongodb-onepiece.log.5.gz
The most recent log file is .log. 1 , not .log. When I use tail -fn to check the log.1 file , I see that the log is still appending to it, and it grows :
ubuntu@koala:/var/log/mongodb$ tail -fn 2 mongodb-onepiece.log.1
2015-06-18T10:36:50.163+0800 [initandlisten] connection accepted from 192.168.1.52:50278 #2507 (49 connections now open)
2015-06-18T10:36:50.163+0800 [conn2503] command koala.$cmd command: isMaster { ismaster: 1 } keyUpdates:0 numYields:0 reslen:178 0ms
This means MongoDB is registering with a file that is not supposed to. As you can see from the mongod config file , MongoDB should enter the log file:
ubuntu@koala:/var/log/mongodb$ vim /etc/mongodb-onepiece.conf
dbpath=/var/lib/mongodb-onepiece
logpath=/var/log/mongodb/mongodb-onepiece.log
logappend=true
bind_ip = 192.168.1.*
port = 47017
fork=true
journal=true
master = true
From the above, I am guessing that the problem was not with the logrotate configuration , but with MongoDB writing to the wrong file. Every day when logrotate starts, it only checks the .log file and finds it empty, after which it will stop rotating the log.
If I restart the mongod daemon, the logarithm is correct for a moment (writing to the right log file). On this day, the .log file is not empty and it will be successfully rotated to a .log file. 1. But the same problem will be repeated after taking the logarithm, i.e. MongoDB will write to a .log file . 1 . The cycle is here.
The logrotate config file is given here:
ubuntu@koala:/var/log/mongodb$ vim /etc/logrotate.d/mongodb
/var/log/mongodb/*.log {
daily
rotate 52
missingok
copytruncate
notifempty
compress
delaycompress
}
The same logrotate configuration works fine with other MongoDB logs on a different server running MongoDB v2.6.5 , and I suppose postrotate isn't the trick here (I've also tried postrotate, but with no luck).
How to solve this problem?
source to share
I'm not a mongo expert, but:
- You must follow the official documentation https://docs.mongodb.org/v2.6/tutorial/rotate-log-files/
-
If you are going to use the logrotate config file as you pointed out, you will need
postrotate
lint for your config (failed to do so, so mongodb keeps getting logged . 1 )postrotate kill -SIGUSR1 `cat /var/run/mongodb.pid` >/dev/null 2>&1 || true
source to share
You need to change your logrorate config as shown below. This worked for me
ubuntu@koala:/var/log/mongodb$ vim /etc/logrotate.d/mongodb
/var/log/mongodb/*.log {
daily
rotate 5
compress
dateext
missingok
notifempty
sharedscripts
postrotate
/usr/bin/killall -SIGUSR1 mongod
endscript
}
and then
sudo logrotate -v -f /etc/logrotate.d/mongod
and then restart the mongod service and another node app (if any) connected to it
sudo service mongod restart
sudo pm2 restat all // if your running pm2 on background otherwise not needed
source to share