Redirecting IO in Linux Bash shell scripts not recreating moved / deleted file?
I am new to shell programming on Linux and on my Linux instance, I redirect the stdout and stderr of the program to two files as follows and run it in the background
myprog > run.log 2>> err.log &
This works great and I get the desired behavior
There is now another background process that monitors run.log and err.log and moves them to different filenames if the log files exceed a certain threshold.
eg. mv err.log err[date-time].log
my expectation is that after this move the err.log file will be re-created by redirecting the output of myprog and a new file will be written to this new file. However, after my log file monitoring process moves the file, err.log or run.log is never generated again, although myprog continues to run without any problem.
Is this common behavior on Linux? If so, what should I do to get my expected behavior to work?
source to share
Yes it is. If this is not the first time you run the program again, it will continue writing to the old file, even if you can no longer access it. In fact, the space used by this deleted file will only be available after each process ends. If reopening is not possible (i.e. you cannot change the executable or restart it) then your solution like http://httpd.apache.org/docs/2.4/programs/rotatelogs.html is your best bet ... It can rotate logs based on files or time and even call a custom script after rotation.
Usage example:
myprog | rotatelogs logname.log 50M
Thus, the log will be rotated whenever the size reaches 50 megabytes.
[EDIT: Pointer to new version of rotatologists]
source to share
If I had to guess, this actually associates the process that is being registered with the file descriptor, not the file name. When you rename it, you only change the filename. So the process just logs to a file. Just a guess. If I were instructed to fix it, I would stop the logging process and restart it at that point to re-associate it with the file I want.
Just guess.
source to share
Software that supports log rotation actually has support for this rotation. If you take a look man logrotate
, you will notice that a typical configuration looks like this:
"/var/log/httpd/access.log" /var/log/httpd/error.log {
rotate 5
mail www@my.org
size 100k
sharedscripts
postrotate
/usr/bin/killall -HUP httpd
endscript
}
... that is, it sends a HUP signal to the program whose log has been rotated; this program has a signal handler that reopens its output files.
You can also do this in your shell scripts:
reopen_logs() {
exec >>run.log 2>>err.log
}
trap reopen_logs HUP
... then, after turning your logs, run kill -HUP pid_of_yourscript
; the next time the script itself executes the command (since signal handlers only execute between foreground executables), it will reopen its output to recreate the log file without restarting.
source to share