Class: LogStash::Filters::KV

Inherits:
Base show all
Defined in:
lib/logstash/filters/kv.rb

Overview

This filter helps automatically parse messages which are of the ‘foo=bar’ variety.

For example, if you have a log message which contains ‘ip=1.2.3.4 error=REFUSED’, you can parse those automatically by doing:

filter {
  kv { }
}

The above will result in a message of “ip=1.2.3.4 error=REFUSED” having the fields:

  • ip: 1.2.3.4

  • error: REFUSED

This is great for postfix, iptables, and other types of logs that tend towards ‘key=value’ syntax.

Further, this can often be used to parse query parameters like ‘foo=bar&baz=fizz’ by setting the field_split to “&”

Constant Summary

Constants inherited from Base

Base::RESERVED

Constants included from Config::Mixin

Config::Mixin::CONFIGSORT

Instance Attribute Summary

Attributes included from Config::Mixin

#config, #original_params

Attributes inherited from Plugin

#logger, #params

Instance Method Summary collapse

Methods inherited from Base

#execute, #initialize, #threadsafe?

Methods included from Config::Mixin

#config_init, included

Methods inherited from Plugin

#eql?, #finished, #finished?, #hash, #initialize, #inspect, lookup, #reload, #running?, #shutdown, #teardown, #terminating?, #to_s

Constructor Details

This class inherits a constructor from LogStash::Filters::Base

Instance Method Details

#filter(event) ⇒ Object

def register



166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
 
# File 'lib/logstash/filters/kv.rb', line 166

def filter(event)
  return unless filter?(event)

  kv = Hash.new

  value = event[@source]

  case value
    when nil; # Nothing to do
    when String; kv = parse(value, event, kv)
    when Array; value.each { |v| kv = parse(v, event, kv) }
    else
      @logger.warn("kv filter has no support for this type of data",
                   :type => value.class, :value => value)
  end # case value

  # Add default key-values for missing keys
  kv = @default_keys.merge(kv)

  # If we have any keys, create/append the hash
  if kv.length > 0
    if @target.nil?
      # Default is to write to the root of the event.
      dest = event.to_hash
    else
      if !event[@target].is_a?(Hash)
        @logger.debug("Overwriting existing target field", :target => @target)
        dest = event[@target] = {}
      else
        dest = event[@target]
      end
    end

    dest.merge!(kv)
    filter_matched(event)
  end
end

#register ⇒ Object 



160
161
162
163
164
 
# File 'lib/logstash/filters/kv.rb', line 160

def register
  @trim_re = Regexp.new("[#{@trim}]") if !@trim.nil?
  @trimkey_re = Regexp.new("[#{@trimkey}]") if !@trimkey.nil?
  @scan_re = Regexp.new("((?:\\\\ |[^"+@field_split+@value_split+"])+)["+@value_split+"](?:\"([^\"]+)\"|'([^']+)'|((?:\\\\ |[^"+@field_split+"])+))")
end