fluent-plugin-grep

Fluentd plugin to grep messages.

Configuration

Example 1:

<match foo.bar.**>
  type grep
  input_key message
  regexp WARN
  exclude favicon
  add_tag_prefix greped
</match>

Assuming following inputs are coming:

foo.bar: {"foo":"bar","message":"2013/01/13T07:02:11.124202 INFO GET /ping"}
foo.bar: {"foo":"bar","message":"2013/01/13T07:02:13.232645 WARN POST /auth"}
foo.bar: {"foo":"bar","message":"2013/01/13T07:02:21.542145 WARN GET /favicon.ico"}
foo.bar: {"foo":"bar","message":"2013/01/13T07:02:43.632145 WARN POST /login"}

then output bocomes as belows (like, | grep WARN | grep -v favicon):

greped.foo.bar: {"foo":"bar","message":"2013/01/13T07:02:13.232645 WARN POST /auth"}
greped.foo.bar: {"foo":"bar","message":"2013/01/13T07:02:43.632145 WARN POST /login"}

Example 2:

Now, regexpN and excludeN options are available to specify grep conditions for multiple fields.

<match foo.bar.**>
  type grep
  regexp1 message WARN
  regexp2 foo ^awesome$
  exclude1 message favicon
  add_tag_prefix greped
</match>

Assuming following inputs are coming:

foo.bar: {"foo":"bar","message":"2013/01/13T07:02:11.124202 INFO GET /ping"}
foo.bar: {"foo":"awesome","message":"2013/01/13T07:02:13.232645 WARN POST /auth"}
foo.bar: {"foo":"bar","message":"2013/01/13T07:02:21.542145 WARN GET /favicon.ico"}
foo.bar: {"foo":"bar","message":"2013/01/13T07:02:43.632145 WARN POST /login"}

then output bocomes as belows:

greped.foo.bar: {"foo":"awesome","message":"2013/01/13T07:02:13.232645 WARN POST /auth"}

Parameters

input_key field_key

The target field key to grep out. Use with regexp or exclude.
regexp regexp

The filtering regular expression
exclude regexp

The excluding regular expression like grep -v
regexp[1-20] field_key regexp

The target field key and the filtering regular expression to grep out.
exclude[1-20] field_key regexp

The target field key and the excluding regular expression like grep -v
tag

The output tag name
add_tag_prefix

Add tag prefix for output message
remove_tag_prefix

Remove tag prefix for output message
add_tag_suffix

Add tag suffix for output message
remove_tag_suffix

Remove tag suffix for output message
replace_invalid_sequence

Replace invalid byte sequence in UTF-8 with '?' character if true. Default: true

ChangeLog

See CHANGELOG.md for details.

Contributing

Fork it
Create your feature branch (git checkout -b my-new-feature)
Commit your changes (git commit -am 'Add some feature')
Push to the branch (git push origin my-new-feature)
Create new Pull Request