fluent-plugin-vertica-query

Fluentd Input plugin to execute vertica query and fetch rows. It is useful for stationary interval metrics measurement.

Installation

install with gem or fluent-gem command as:

# for fluentd
$ gem install fluent-plugin-vertica-query

# for td-agent
$ sudo /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-vertica-query

Configuration

Config Sample

<source>
  type            vertica_query
  host            localhost           # Optional (default: localhost)
  port            5433                # Optional (default: 5433)
  username        nagios              # Optional (default: dbadmin)
  password        passw0rd            # Optional (default not set)
  interval        30s                 # Optional (default: 1m)
  database        db                  # Optional (default: not set as there is only one database per cluster)
  ssl             true                # Optional connection via ssl (default: false)
  role            pseudosuperuser     # Optional additional role(s) to set for the user
  search_path     test_date           # Optional (default: not set)
  tag             input.vertica       # Required
  query           SELECT * FROM V_CATALOG.RESOURCE_POOLS; # Required
  # record hostname into message.
  record_host     yes                 # Optional (default: no)
  # multi row results into nested record or separated message.
  nest_result     yes                 # Optional (default: no)
  nest_key        data                # Optional (default: result)
  # record the number of lines of a query result
  row_count       yes                 # Optional (default: no)
  row_count_key   row_count           # Optional (default: row_count)
</source>

<match input.vertica>
  type stdout
</match>

Output Sample

record_hostname: yes, nest_result: no

input.vertica: {"hostname":"myhost.example.com","Variable_name":"thread_cache_size","Value":"16"}
input.vertica: {"hostname":"myhost.example.com","Variable_name":"thread_stack","Value":"262144"}

record_hostname: yes, nest_result: yes, nest_key: data

input.vertica: {"hostname":"myhost.example.com","data":[{"Variable_name":"thread_cache_size","Value":"16"},{"Variable_name":"thread_stack","Value":"262144"}]}

record_hostname: yes, nest_result: yes, nest_key: data, row_count: yes, row_count_key: row_count

input.vertica: {"hostname":"myhost.example.com","row_count":2,"data":[{"Variable_name":"thread_cache_size","Value":"16"},{"Variable_name":"thread_stack","Value":"262144"}]}

Example Query

SELECT MAX(id) AS max_foo_id FROM foo_table;
SELECT * FROM nodes;
SELECT * FROM V_CATALOG.RESOURCE_POOLS;
SELECT user_id, memory_cap_kb FROM users WHERE resource_pool='general';