Class: LogStash::Filters::Memoize

Inherits:
Base
  • Object
show all
Defined in:
lib/logstash/filters/memoize.rb

Overview

This filter provides en.wikipedia.org/wiki/Memoization[memoization] to wrapped filter. Internally, It based on en.wikipedia.org/wiki/Cache_replacement_policies#LRU[LRU] cache algorithm.

See below an example of how this filter might be used.

source,ruby

filter 

memoize {
  key => "%{host" <1>
  fields => ["host_owner", "host_location"] <2>
  filter_name => "elasticsearch" <3>
  filter_options => { <4>
    query => "host:%host"
    index => "known_host"
    fields => {
      "host_owner" => "host_owner"
      "host_location" => "host_location"
    }
  }
}

}

  • When an event with a new <1> key comes in, execute wrapped <3> <4> filter and caches the <2> fields value.

  • When an event with a same <1> key comes in, sets cached value to target <2> fields without wrapped <3> <4> filter execution.

Instance Method Summary collapse

Instance Method Details

#filter(event) ⇒ Object 



62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
 
# File 'lib/logstash/filters/memoize.rb', line 62

def filter(event)
  formattedKey = event.sprintf(@key);
  result = @cache[formattedKey]

  if !result.nil?
    @logger.debug("Cached value found.", :key => formattedKey, :value => result) if @logger.debug?
    @fields.each { |field| event.set(field, result[field]) }
  else
    @logger.debug("Cached value not found. Do filter.", :key => formattedKey, :filter => @filter) if @logger.debug?
    @filter.filter(event)
    @fields.each { |field| (result ||= {})[field] = event.get(field) }
    @cache[formattedKey] = result
  end

  # filter_matched should go in the last line of our successful code
  filter_matched(event)
end

#register ⇒ Object 



55
56
57
58
59
 
# File 'lib/logstash/filters/memoize.rb', line 55

def register
  @filter = LogStash::Plugin.lookup("filter", @filter_name).new(@filter_options)
  @filter.register
  @cache = ::LruRedux::TTL::ThreadSafeCache.new(@cache_size, @ttl)
end