fluent-plugin-filter-split
Fluentd filter plugin to split specified field.
Compared to existing plugins:
- https://github.com/wshihadeh/fluent-plugin-record-splitter
- This is filter plugin. It splits the specified field into multiple records. The target field must be lines.
- https://github.com/unquietwiki/fluent-plugin-split_record
- This is filter plugin. It splits the specified field into key/value pairs. The target field must contain "key=value" pairs.
- https://github.com/SNakano/fluent-plugin-split-array
- This is filter plugin. It splits the specified field into multiple records. The target field must be always array.
- https://github.com/toyama0919/fluent-plugin-split
- This is output plugin. It splits the specified field into multiple records. The target field must contains specific separator.
- https://github.com/activeshadow/fluent-plugin-split-event
- This is filter plugin. It splits the specified field into multiple records. The target field must contains specific separator.
- https://github.com/bitpatty/fluent-plugin-filter-split-message
- 404. The only gem exists. https://rubygems.org/gems/fluent-plugin-filter-split-message
- This is filter plugin. It splits the specified field into multiple records. The target field must contains specific separator.
Thus, about above plugins, there are limitation of type of field or missing feature to control keep/remove other fields.
Installation
RubyGems
$ gem install fluent-plugin-filter-split
Bundler
Add following line to your Gemfile:
gem "fluent-plugin-filter-split"
And then execute:
$ bundle
Configuration
parameter | type | description | default |
---|---|---|---|
split_key | string (required) | Specify a target key to split | |
keep_other_key | bool (optional) | Specify a flag whether other key must be kept or not | false |
keep_keys | array (optional) | Specify keys to be kept in filtered record | [] |
remove_keys | array (optional) | Specify keys to be removed in filtered record | [] |
Example
<filter test>
@type filter_split
split_key target_field
keep_other_key true
</filter>
If following record is passed:
{'foo:'bar', 'target_field':[ {'k1':'v1'}, {'k2':'v2'} ] }
Then, you got:
{'foo:'bar', 'k1':'v1'}
{'foo:'bar', 'k2':'v2'}
If following record is passed:
{'foo:'bar', 'target_field':[ 'v1', 'v2' ] }
Then, you got:
{'foo:'bar', 'target_field':'v1'}
{'foo:'bar', 'target_field':'v2'}
Copyright
- Copyright(c) 2023- Kentaro Hayashi
- License
- Apache License, Version 2.0