Multiple levels of multiline in logstash

I want to use a multi-line filter and then another multi-line at a deeper level.

To be more precise, I want to have a stack trace of java exceptions, for example:

2014-06-20 Some-arbitrary-log
java.lang.IndexOutOfBoundsException: Index: 8, Size: 1
    at java.util.ArrayList.rangeCheck(ArrayList.java:604)
    at java.util.ArrayList.get(ArrayList.java:382)

      

And then, after that, combine a couple of them together like this using another multi-line and throttle:

2014-06-19 Some-arbitrary-log
java.lang.IndexOutOfBoundsException: Index: 2, Size: 1
   at java.util.ArrayList.rangeCheck(ArrayList.java:604)
   at java.util.ArrayList.get(ArrayList.java:382)
2014-06-20 Some-arbitrary-log
java.lang.IndexOutOfBoundsException: Index: 8, Size: 1
    at java.util.ArrayList.rangeCheck(ArrayList.java:604)
    at java.util.ArrayList.get(ArrayList.java:382)

      

My filter looks like this:

filter {
    if [type] =~ /test.+
    {
        multiline {
            pattern => "(^.+Exception.*)|(^\tat.+)"
            negate => false
            what => "previous"
        }
        if ("multiline" in [tags]) {
            mutate {
                add_field => [ "ERROR_TYPE", "java_exception" ]
            }
        }

        if ([ERROR_TYPE] == "java_exception") {
            throttle{
                key => ".*"
                period => 10
                before_count => 2
                after_count => -1
                add_tag => "throttled"
            }

            if ("throttled" not in [tags]) {
                multiline {
                    pattern => ".*"
                    negate => false
                    what => "previous"
                }
            }
        }

    }
}

      

The first level is only one stack trace. As is the case, this works as intended:

    multiline {
        pattern => "(^.+Exception.*)|(^\tat.+)"
        negate => false
        what => "previous"
    }
    if ("multiline" in [tags]) {
        mutate {
            add_field => [ "ERROR_TYPE", "java_exception" ]
        }
    }

      

However, concatenating multiple stacks of stack does not work. The output I'm using is this:

output {
    if [ERROR_TYPE] == "java_exception"{
    stdout {codec => rubydebug }
    elasticsearch {
        cluster => "logstash"
    }
    }
}

      

However, there are no combined stack traces. And they all have the tags "throttled".

To check if there are any chokes I did:

output {
    if [ERROR_TYPE] == "java_exception" and "throttled" not in [tags]{
    stdout {codec => rubydebug }
    elasticsearch {
        cluster => "logstash"
    }
    }
}

      

And nothing happened. Why doesn't it get "before counting" throttling in the choke filter?

Anyone thoughts?

+3
elasticsearch logstash


source to share


No one has answered this question yet

Check out similar questions:

76
How to handle multiple heterogeneous logins with Logstash?
18
Logstash grok multi-line message
4
Logstash: parse multi-line search box for elastics
4
Parse multiline JSON with grok in logstash
1
Logstash multi-line system does not group messages
1
Log in to Logstash Beats - Multi-Line Multi-Channel Codec
0
Combining multiple message fields using multi-line codec in logstash?
0
Parsing a multi-line logstash trace stack
0
Multiline logstash "next" not grouping
0
How is multiline Logstash for date strings?



All Articles
Loading...
X
Show
Funny
Dev
Pics