fluent-plugin-grepcounter
Fluentd plugin to count the number of matched messages, and emit if exeeds the threshold
.
Configuration
Assume inputs from another plugin are as belows:
syslog.host1: {"message":"20.4.01/13T07:02:11.124202 INFO GET /ping" }
syslog.host1: {"message":"20.4.01/13T07:02:13.232645 WARN POST /auth" }
syslog.host1: {"message":"20.4.01/13T07:02:21.542145 WARN GET /favicon.ico" }
syslog.host1: {"message":"20.4.01/13T07:02:43.632145 WARN POST /login" }
An example of grepcounter configuration:
<match syslog.**>
type grepcounter
count_interval 60
input_key message
regexp WARN
exclude favicon.ico
threshold 1
add_tag_prefix warn.count
</match>
Then, output bocomes as belows (indented):
warn.count.syslog.host1: {
"count":2,
"message":["20.4.01/13T07:02:13.232645 WARN POST /auth","20.4.01/13T07:02:43.632145 WARN POST /login"],
"input_tag":"syslog.host1",
"input_tag_last":"host1",
}
Output message by joining with a delimiter
As default, the grepcounter
plugin outputs matched message
as an array as shown above.
You may want to output message
as a string, then use delimiter
option like:
<match syslog.**>
type grepcounter
count_interval 60
input_key message
regexp WARN
exclude favicon.ico
threshold 1
add_tag_prefix warn.count
delimiter \n
</match>
Then, output bocomes as belows (indented). You can see the message
field is joined with \n.
warn.count.syslog.host1: {
"count":2,
"message":"20.4.01/13T07:02:13.232645 WARN POST /auth\n20.4.01/13T07:02:43.632145 WARN POST /login",
"input_tag":"syslog.host1",
"input_tag_last":"host1",
}
Parameters
count_interval
The interval time to count in seconds. Default is 60.
input_key field_key
The target field key to grep out. Use with regexp or exclude.
regexp regexp
The filtering regular expression
exclude regexp
The excluding regular expression like grep -v
regexp[1-20] field_key regexp (experimental)
The target field key and the filtering regular expression to grep out. No
message
is outputted in this case.exclude[1-20] field_key regexp (experimental)
The target field key and the excluding regular expression like grep -v. No
message
is outputted in this case.threshold
The threshold number to emit. Emit if
count
value >= specified value.greater_equal
This is same with
threshold
option. Emit ifcount
value is greater than or equal to (>=) specified value.greater_than
Emit if
count
value is greater than (>) specified value.less_than
Emit if
count
value is less than (<) specified value.less_equal
Emit if
count
value is less than or equal to (<=) specified value.tag
The output tag. Required for aggregate
all
.add_tag_prefix
Add tag prefix for output message
remove_tag_prefix
Remove tag prefix for output message
add_tag_suffix
Add tag suffix for output message
remove_tag_suffix
Remove tag suffix for output message
delimiter
Output matched messages after
join
ed with the specified delimiter.replace_invalid_sequence
Replace invalid byte sequence in UTF-8 with '?' character if
true
store_file
Store internal count data into a file of the given path on shutdown, and load on statring.
ChangeLog
See CHANGELOG.md for details.
Contributing
- Fork it
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create new Pull Request
Copyright
Copyright (c) 2013 Naotoshi SEO. See LICENSE for details.