Grok

Grok aims to be a replacement for the now antiquated SEC (Simple Event Correlator).

Getting started

A simple Grok watcher needs very little in the way of configuration

require 'grok'

configure do |c|
  c.file = "/var/log/auth.log"
  c.interval = 2
  c.replay = 0
end

The above script won’t do very much, though.

Configuration

There’s only a few configuration parameters for Grok at this stage

file: The log file to watch
interval: How often to check the log file for changes (in seconds)
replay: The number of lines to read from the bottom of the file on startup
process: Spawn this process and feed the output into grok

Responding to log events

At it’s most basic, you can simply get Grok to print out each message as it receives them (pretty pointless)

on /(.*)/ do |line|
  puts line
end

Lets try something a bit more useful though. Lets say I want to know every time there’s an SSH authenitcation failure. For that, we can make use of the RegExp functionality in the event handlers

on /sshd\[\d+\]: Failed password for ([\d\w]+) from ([\d\.]+)/ do |username, ip|
  puts "SSH authentication failure for #{username} from #{ip}"
end

This is a bit better. It seems a bit unfair to block someone for a single typo though, so why don’t we give them three tries before blocking them?

on /sshd\[\d+\]: Failed password for ([\d\w]+) from ([\d\.]+)/, :times => 3 do |username, ip|
  puts "SSH authentication failure for #{username} from #{ip}"
end

Getting there. What if our user failed a couple of times over the past month? We don’t really want to him out for that, so we’ll put a time limit on the rule so only 3 incorrect login attempts within the past 2 minutes will trigger it.

on /sshd\[\d+\]: Failed password for ([\d\w]+) from ([\d\.]+)/, :times => 3, :within => '2m' do |username, ip|
  puts "SSH authentication failure for #{username} from #{ip}"
end

Your time string can be made of any combination of years (y), months (M), weeks (w), days (d), hours (h), minutes (m) and seconds (s). For example

'1d2h3s' => 1 day, 2 hours and 3 seconds
'2y3m' => 2 years and 3 minutes

Ignoring log events

If you want grok to ignore certain log events, you can simply pass a regexp to the ignore function.

ignore /regexp/

This is particularly useful if you have a catch-all event in your script somewhere

ignore /kernel: imklog [\d\.]+, log source = \/proc\/kmsg started/

on /(.*) do |line|
  #foo
end

Other events

Exit

You can also define event handlers to run when your script exits (for the purposes of printing a summary, or whatever you want).

exit do 
  puts "Done"
end

You can define as many of these handlers as you’d like and they’ll be run when the Ruby process has been sent a SIGINT.

Start

You can also define event handlers to run before Grok starts reading it’s input.

start do
  puts "Starting"
end

You can define as many of these event handlers as you’d like

SIGUSR1 and SIGUSR2

You can define event handlers to run whenever your grok process receives SIGUSR1 or SIGUSR2 as well.

on :usr1 do
  puts "SIGUSR1 received"
end

on :usr2 do
  puts "SIGUSR2 received"
end

You can define as many of these event handlers as you’d like.

Note on Patches/Pull Requests

Fork the project.
Make your feature addition or bug fix.
Add tests for it. This is important so I don’t break it in a future version unintentionally.
Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
Send me a pull request. Bonus points for topic branches.