Treasure Data logging library for Rails

About

This gem is a logging library for Treasure Data. The events logged by this module will be uploaded into the cloud. There’re two ways to upload:

direct upload from applications: app -> cloud. This option is easier to configure and setup, but it requires extra memory in the application processes. The data is logged directly in the Treasure Data Cloud through the REST APIs by leveraging the Treasure Data Ruby Client (td-client-ruby) library.
indirect upload from td-agent: app -> td-agent -> cloud. This option requires extra steps to install the daemons into your cluster, but lowers the logging impact on your applications. The data is sent to a td-agent running either locally or remotely by leveraging the Fluentd Ruby Logger (fluent-logger-ruby) library.

The actual upload method can be selected in the config file as described below.

Installation

Add the following line to your Gemfile:

gem 'td-logger'

For Rails 2.x (not tested) without Bundler, edit environment.rb and add to the initalizer block:

config.gem "td-logger"

Configuration

Direct Upload

This configuration enables the application to upload the events directly from the application processes (app -> cloud) through the Treasure Data REST APIs. Please add this template config/treasure_data.yml file and set your apikey:

development:
  apikey: "YOUR_API_KEY"
  database: rails_development
  debug_mode: true  # enable debug mode

production:
  apikey: "YOUR_API_KEY"
  database: rails_production
  debug_mode: false

# disable logging
test:

If you would wish to keep your API key as an environment variable, you can use this setting:

apikey: <%= ENV['TD_API_KEY'] %>

Alternatively, one can initialize the Ruby Logger module for direct upload straight from within the application:

TreasureData::Logger.open('production',
                          :apikey => "YOUR_API_KEY",
                          :auto_create_table => true)

The hash key / value pairs specified in the second and following arguments of the ‘open’ function correspond to the configuration option of the underlying td-client-ruby Ruby Client library.

Refer to the Ruby SDK documentation for more information.

Please note that we recommend to use a write-only API key. To obtain one, please:

Login into the Treasure Data Console at console.treasuredata.com;
Visit your Profile page at console.treasuredata.com/users/current;
Insert your password under the ‘API Keys’ panel;
In the bottom part of the panel, under ‘Write-Only API keys’, either copy the API key or click on ‘Generate New’ and copy the new API key.

Indirect Upload

When the ‘agent’ and ‘tag’ options are specified, the events are logged into the local td-agent daemon. Later on, the daemon uploads the event periodically to the cloud (app -> td-agent -> cloud).

This requires additional setup steps but lowers the memory and performance impact to your application processes. For the installation process, please look at the section “How to Install Treasure Agent?” on “Quickstart Guide”.

NOTE: This configuration is not supported on PaaS platforms.

production:
  agent: "localhost:24224"
  tag: td.myapp
  debug_mode: false

Alternatively, one can initialize the Ruby Logger module for indirect upload within the application:

TreasureData::Logger.open_agent('td.myapp',
                                :host => 'localhost',
                                :port => 24224)

The arguments of the ‘open_agent’ function correspond to the configuration option of the underlying Fluentd Ruby Logger (fluent-logger-ruby) library.

Logging events

You can log anytime using ‘TD.event.post’ method:

TD.event.post('table_name', {:foo=>:bar})

Rails extension

In rails application, you can use ‘TD.event.attribute’ to set static attribute to all events:

class ApplicationController
  def authenticate
    # set 'uid' attribute to succeeding event logs
    TD.event.attribute[:uid] = 198
  end
end

class MyController < ApplicationController
  def mymethod
    authenticate()
    # this event includes 'uid' attribute
    TD.event.post('table_name', {:foo=>:bar})
  end
end

Test mode

This mode uses TestLogger of fluent-logger-ruby to store the events. If you want to enable this mode, then specify ‘test_mode’ in your configuration.

development:
  apikey: "YOUR_API_KEY"
  database: rails_development
  debug_mode: true
  test_mode: true

You can access the logger instance via TD.logger to check the status.

TD.logger.queue #=> [{},{},..]

Copyright

License: Apache License, Version 2.0