Module: LogParser

Defined in:
lib/production_log/parser.rb

Overview

LogParser parses a Syslog log file looking for lines logged by the ‘rails’ program. A typical log line looks like this:

Mar  7 00:00:20 online1 rails[59600]: Person Load (0.001884)   SELECT * FROM people WHERE id = 10519 LIMIT 1

LogParser does not work with Rails’ default logger because there is no way to group all the log output of a single request. You must use SyslogLogger.

Defined Under Namespace

Classes: LogEntry

Class Method Summary collapse

Class Method Details

.detect_mode(stream) ⇒ Object 



175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
 
# File 'lib/production_log/parser.rb', line 175

def self.detect_mode(stream)
  lines_read = []
  while(!stream.eof? && lines_read.size < 5)
    lines_read << stream.readline
  end
  stream.rewind
  
  lines_read.each do |line|
    line =~ / ([^ ]+) ([^ ]+)\[(\d+)\]: (.*)/
    if $4        
      return syslog_mode!
    end
  end
  return vanilla_mode!
end

.extract_bucket_and_data(line) ⇒ Object 



163
164
165
166
167
168
169
170
171
172
173
 
# File 'lib/production_log/parser.rb', line 163

def self.extract_bucket_and_data(line)
  if LogParser.syslog_mode?
    line =~ / ([^ ]+) ([^ ]+)\[(\d+)\]: (.*)/
    return nil if $2.nil? or $2 == 'newsyslog'
    bucket = [$1, $2, $3].join '-'
    data = $4
    return [bucket, data]
  else
    return ["(none)", line]
  end    
end

.parse(stream) ⇒ Object

Parses IO stream stream, creating a LogEntry for each recognizable log entry.

Log entries are recognised as starting with Processing, continuing with the same process id through Completed.



198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
 
# File 'lib/production_log/parser.rb', line 198

def self.parse(stream) # :yields: log_entry
  buckets = Hash.new { |h,k| h[k] = [] }
  comp_count = Hash.new 0
  
  LogParser.detect_mode(stream)
  
  stream.each_line do |line|
    bucket, data = LogParser.extract_bucket_and_data(line)
    next if !bucket
    
    buckets[bucket] << data

    case data
    when /^Start rendering component / then
      comp_count[bucket] += 1
    when /^End of component rendering$/ then
      comp_count[bucket] -= 1
    when /^Completed/ then
      next unless comp_count[bucket] == 0
      entry = buckets.delete bucket
      yield LogEntry.new(entry)
    end
  end

  buckets.each do |bucket, data|
    yield LogEntry.new(data)
  end
end

.syslog_mode! ⇒ Object 



16
17
18
 
# File 'lib/production_log/parser.rb', line 16

def self.syslog_mode!
  @syslog_mode = true
end

.syslog_mode? ⇒ Boolean

Returns:

  • (Boolean) 


12
13
14
 
# File 'lib/production_log/parser.rb', line 12

def self.syslog_mode?
  @syslog_mode
end

.vanilla_mode! ⇒ Object 



19
20
21
 
# File 'lib/production_log/parser.rb', line 19

def self.vanilla_mode!
  @syslog_mode = false
end