How to save count of deduplicated messages from Logstash to ElasticSearch?
I can see from this question that document_id can be easily used in Logstash to replace duplicate entry in ElasticSearch. How to add / increase counter value, eg. repeating syslog messages? Instead of just replacing the entry, I want to increment the counter, so I know how many times it has been repeated.
source to share
Depending on what you are using to view the data, it might just be the same as viewing a _version
document field . ES will automatically update this value when something changes for the document. Kibana doesn't show the _version field ( https://github.com/elasticsearch/kibana/issues/1234 ), but it's there.
source to share
I think I have found what I need. upsert will be inserted if the record does not exist and updated if it does. And including the _ source field will allow the existing field to grow. This combination gives me the ability to start at 1 and increment by 1 if the entry already exists. Thanks @Alcanzar your answer got my brain moving in the right direction to find this.
source to share