Clever Geek Handbook

Elasticsearch multiple aggregates not working

struggling with elasticsearch assemblies - advice might be needed ...

elasticsearch version: Version: 1.4.1, Build: 89d3241 / 2014-11-26T15: 49: 29Z, JVM: 1.7.0_72

sample data:

{
  "_index": "logstash-2014.12.17",
  "_type": "netflow",
  "_id": "AUpaDdUVUcM5Us_C6x7Z",
  "_score": 1,
  "_source": {
    "message": "<27>Dec 17 22:01:02 es01 nfcapd[29441]: expip=10.245.132.16 fweventtime=2014-12-17 22:01:02.793 fwevent=DENIED srcip=78.110.142.76 dstip=179.24.227.252 srcport=62327 dstport=41863 proto=UDP input=3 output=4 inbytes=0 outbytes=0 postnatsrcip=78.110.142.76 postnatdstip=179.24.227.252 postnatsrcport=62327 postnatdstport=41863 ingressacl=0x45b0635e/0x9872d678/0x724bf9a4 egressacl=0x0/0x0/0x0",
    "@version": "1",
    "@timestamp": "2014-12-17T21:01:02.794Z",
    "type": "netflow",
    "host": "127.0.0.1",
    "timestamp": "Dec 17 22:01:02",
    "hostname": "es01",
    "expip": "10.245.132.16",
    "time": "2014-12-17 22:01:02.793",
    "fwevent": "DENIED",
    "srcip": "78.110.142.76",
    "dstip": "179.24.227.252",
    "srcport": "62327",
    "dstport": "41863",
    "proto": "UDP",
    "output": "4",
    "inbytes": "0",
    "outbytes": "0",
    "postnatsrcip": "78.110.142.76",
    "postnatdstip": "179.24.227.252",
    "postnatsrcport": "62327",
    "postnatdstport": "41863",
    "ingressacl1": "0x45b0635e",
    "ingressacl2": "0x9872d678",
    "ingressacl3": "0x724bf9a4",
    "egressacl1": "0x0",
    "egressacl2": "0x0",
    "egressacl3": "0x0",
    "srcgeo": {
      "country_code3": "CHE",
      "latitude": 47,
      "longitude": 8,
      "location": [
        8,
        47
      ]
    }
  }
}

Request example:

GET _search
{
  "size": 1,
  "query": {
    "filtered": {
      "filter": {
        "range": {
          "@timestamp": {
            "gt": "2014-12-17T21:00:00"
          }
        }
      }
    }
  },
  "aggs": {
    "proto": {
      "terms": {
        "field": "proto"
      },
      "aggs": {
        "traffic_sum": {
          "sum": {
            "field": "outbytes"
          }
        }
      }
    }
  }
}

results in an error:

{
  "error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; 
            shardFailures {[jJZG3gX7QlujjG4ZXttyRA][logstash-2014.12.17][0]:
            ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}{[8Rz-FI7JSvebgBdGG9zOkA][logstash-2014.12.17][1]:
            RemoteTransportException[[bigdata02][inet[/<snip>:9301]][indices:data/read/search[phase/query]]];
              nested: ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; }{[8Rz-FI7JSvebgBdGG9zOkA][logstash-2014.12.17][2]:
            RemoteTransportException[[bigdata02][inet[/<snip>:9301]][indices:data/read/search[phase/query]]];
              nested: ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; }{[jJZG3gX7QlujjG4ZXttyRA][logstash-2014.12.17][3]:
            ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}{[jJZG3gX7QlujjG4ZXttyRA][logstash-2014.12.17][4]:
            ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}]",
"status": 500
}

* works fine with only one aggregation - fails if I insert a second aggregation * any idea?

+3


source to share


1 answer


this is the important part:

ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData 
cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}]"

You are trying to make an amount using a string field.

this field is the problem:



"outbytes": "0",

Or

  • delete the existing data and create a numeric field type by posting the document containing "outbytes": 0

    (note the lack of quotes).
  • Delete existing data and create an explicit mapping with field outbytes

    set to number.
  • Keep your data, but update the aggregation to invoke a script that converts the string to numeric conversion.

My recommendation would be to upgrade to option 2.

+2


source






More articles:


All Articles