fluent-plugin-file-alternative
File output plugin alternative implementation.
FileAlternativeOutput slices data by time (for specified units), and store these data as plain text on local file. You can specify to:
- format whole data as serialized JSON, single attribute or separated multi attributes
- include time as line header, or not
- include tag as line header, or not
- change field separator (default: TAB)
- add new line as termination, or not
And you can specify output file path as:
- Standard out_file way
- configure
path /path/to/dir/access
- and
time_slice_format %Y%m%d
- got
/path/to/dir/access.20120316.log
- configure
- Standard out_file way (alternative)
- configure
path /path/to/dir/access.*.log
- and
time_slice_format %Y%m%d
- got
/path/to/dir/access.20120316.log
- configure
- Alternative style
- configure
path /path/to/dir/access.%Y%m%d.log
only (don't include*
) - got
/path/to/dir/access.20120316.log
- configure
And, gzip compression is also supported.
Configuration
FileAlternativeOutput
Standard out_file way (hourly log, compression, time-tag-json):
<match out.**>
type file_alternative
path /var/log/service/access.*.log
time_slice_format %Y%m%d_%H
compress gzip
</match>
By this configuration, in gzip compressed file '/var/log/service/access.20120316_23.log.gz', you get:
2012-03-16T23:59:40 [TAB] out.service.xxx [TAB] {"field1":"value1","field2":"value2"}
2012-03-16T23:59:40 [TAB] out.service.xxx [TAB] {"field1":"value1","field2":"value2"}
2012-03-16T23:59:40 [TAB] out.service.xxx [TAB] {"field1":"value1","field2":"value2"}
2012-03-16T23:59:40 [TAB] out.service.xxx [TAB] {"field1":"value1","field2":"value2"}
If you don't want fluentd-time and tag in written file, and messages with single attribute (as raw full apache log with newline):
<match out.**>
type file_alternative
path /var/log/service/access.%Y%m%d_%H.log
compress gzip
output_include_time false
output_include_tag false
output_data_type attr:message
add_newline false
symlink_path /var/log/service/access_latest.log
dir_mode 0777
</match>
Then, you will get:
192.168.0.1 - - [16/Mar/2012:23:59:40 +0900] "GET /content/x HTTP/1.1" 200 -
192.168.0.1 - - [16/Mar/2012:23:59:40 +0900] "GET /content/x HTTP/1.1" 200 -
192.168.0.1 - - [16/Mar/2012:23:59:40 +0900] "GET /content/x HTTP/1.1" 200 -
TODO
- consider what to do next
- patches welcome!
Copyright
Copyright:: Copyright (c) 2012- TAGOMORI Satoshi (tagomoris) License:: Apache License, Version 2.0