How to make logstash start once a day and automatically stop after completing a job
I would like to develop a tool that should process various log files and provide some "just once a day" output. I couldn't find a way online to do this.
logstash has no batch mode. You will need to write a shell script to kill it when it is finished.
you would like to use exec logstash plugin. by visiting the website https://www.elastic.co/guide/en/logstash/current/plugins-inputs-exec.html#plugins-inputs-exec-interval "
"interval" configuration parameters can help you, Interval to run the command. The value is in seconds.
Logstash can run in batch mode (v2.3.4). Just omit the word "agent" in the command line options and pipe some output into the stdin input filter like this:
cat /var/log/somelog | logstash -e 'input { stdin {} } output { elasticsearch {} }'
You will probably need to add an extra field to provide the original filename so that you can query for data later.
NOTE. The -help text for the logstash executable does not specify "agent" as optional, but the example from the official docs omits the word "agent".