Timelion series from total sum of array length
I have docs similar to below:
{
dateTime: /* My time field */,
message: {
users: ['1', '2']
},
messageType: 'test'
}
I would like to plot a timelion series chart that shows me the cumulative sum of an array count message.users
. My first guess was to create a script:
if(doc.containsKey('message.users')) {
return doc['message.users'].length;
} else {
return 0;
}
From what I could tell, doc.containsKey('message.users')
it was always false, which tells me that it may not have been indexed correctly. I've tried lots of Timelions, all to no avail:
.es(index=logstash-*,timefield='dateTime',q='messageType:UserList').label('Users Online')
I am indexing my document via C # NEST api like so:
elasticClient.Index(
new
{
DateTime = DateTime.Now,
Message = evt.EventArgs.Message,
},
idx => idx.Index($"logstash-{evt.MessageCode}"));
source to share
I suggest adding one more field called to your documents userCount
, so you don't have to mess with the script (+ it will be more efficient).
So your documents should look like this:
{
dateTime: /* My time field */,
message: {
users: ['1', '2']
},
userCount: 2, <--- add this field
messageType: 'test'
}
Solution 1:
You need to modify your code a little:
elasticClient.Index(
new
{
DateTime = DateTime.Now,
Message = evt.EventArgs.Message,
UserCount = evt.EventArgs.Message.Users.Length
},
idx => idx.Index($"logstash-{evt.MessageCode}"));
Solution 2:
If you are using ES 5, you can use the Ingest API to create a pipeline that will automatically add that userCount
field for you. You don't need to change anything in your code.
PUT _ingest/pipeline/user-count-pipeline
{
"description" : "Creates a new userCount field",
"processors" : [
{
"script": {
"lang": "painless",
"inline": "ctx.userCount = ctx.message?.users?.size() ?: 0"
}
}
]
}
Then in Timelion it will be very easy to draw what you need using metric='sum:userCount'
to sum the values userCount
and a function cusum()
to get the sum userCount
over time. The whole expression will look like this:
.es(index=logstash-*,timefield='dateTime',q='messageType:UserList',metric='sum:userCount').label('Users Online').cusum()
Using a few sample documents, the time series looks like this is what you are looking for.
source to share