Module: RubyRunMonitor__

Includes:
RubyRunBufferMgr__, RubyRunCommander__, RubyRunGlobals, RubyRunHTML__, RubyRunReport__, RubyRunTracer__, RubyRunUtils__
Included in:
RubyRunInstrumentor__
Defined in:
lib/rubyrun/rubyrun_monitor__.rb

Overview

—————————————————————#

                                                             #  
(C) Copyright Rubysophic Inc. 2007-2008                      #
All rights reserved.                                         #
                                                             #  
Use, duplication or disclosure of the code is not permitted  #
unless licensed.                                             #
                                                             #            
Last Updated: 7/09/08                                        #

—————————————————————#

                                                             #  
RubyRunMonitor__ is responsible for keeping track of         #
the response time of a Rails Request and its decomposition.  #
                                                             #  
For instance, a typical WEBrick/RAILS servlet dispatch       #
response time components generally break down look roughly   # 
like this:           						    #
                                                             #      
SVS   DI   ACT   DB   DB    DB          VI    VI             #    
-+-----+----+----+--+--+--+--+--+----+--+----+-+--+---+----+ #
                   DB'   DB'   DB'  ACT'    VI'  VI' DI' SVS'#            
Legend:                                                      #
 SVS = DispatchServlet.service                               #
 DI = DispatchServlet.handle_dispatch                        #
 ACT = controller.action                                     #    
 DB  = ActiveRecord::ConnectionAdapters::*Adapter.execute    #
 VI  = ActionView.Base:render_template                       #
 All apostrophe means "end of event"                         #
                                                             #
 Notes:                                                      #  
 1. mutex held from SVS to SVS'                              #
 2. Total dispatch time with mutex = SVS' - SVS              #
 3. Total DBIO time = Sum(DB' - DB)                          #    
 4. Action Time = ACT' - ACT                                 #
 5. View time = Sum(VI' - VI)                                #
 6. View rendering includes template + layout                #
                                                             #      
Another key function is to act as a command agent, responding#
to command such as displaying thread status, terminating     #
threads with stack tracke and showing object heap info.      #
                                                             #

—————————————————————#

Constant Summary

Constants included from RubyRunHTML__

RubyRunHTML__::METHOD_TRACE_EVEN_ROW, RubyRunHTML__::METHOD_TRACE_HEADER, RubyRunHTML__::METHOD_TRACE_ODD_ROW, RubyRunHTML__::OBJ_MAP_EVEN_ROW, RubyRunHTML__::OBJ_MAP_HTML, RubyRunHTML__::OBJ_MAP_ODD_ROW, RubyRunHTML__::REQ_PERF_BREAKDOWN_HTML, RubyRunHTML__::REQ_PERF_BREAKDOWN_TABLE_EVEN, RubyRunHTML__::REQ_PERF_BREAKDOWN_TABLE_ODD, RubyRunHTML__::THREAD_STATUS_EVEN_ROW, RubyRunHTML__::THREAD_STATUS_HTML, RubyRunHTML__::THREAD_STATUS_ODD_ROW, RubyRunHTML__::THROUGHPUT_BAR_TABLE, RubyRunHTML__::THROUGHPUT_HTML, RubyRunHTML__::THROUGHPUT_LABEL_TABLE, RubyRunHTML__::TOP_SLOWEST_REQUESTS_HTML, RubyRunHTML__::TOP_SLOWEST_REQUESTS_TABLE

Constants included from RubyRunGlobals

RubyRunGlobals::RUBYRUN_ACTIVERECORD, RubyRunGlobals::RUBYRUN_CMD_EXIT, RubyRunGlobals::RUBYRUN_CMD_HARD_KILL, RubyRunGlobals::RUBYRUN_CMD_OBJECT_MAP, RubyRunGlobals::RUBYRUN_CMD_SOFT_KILL, RubyRunGlobals::RUBYRUN_CMD_STATUS, RubyRunGlobals::RUBYRUN_DIR_HASH_FILE, RubyRunGlobals::RUBYRUN_DOC_DIR, RubyRunGlobals::RUBYRUN_ETC_DIR, RubyRunGlobals::RUBYRUN_EXCLUDE_HASH_FILE, RubyRunGlobals::RUBYRUN_FIREWALL_HASH, RubyRunGlobals::RUBYRUN_HIGHLIGHT_THRESHOLD, RubyRunGlobals::RUBYRUN_INCLUDE_HASH_FILE, RubyRunGlobals::RUBYRUN_INNER_DISPATCH_HASH, RubyRunGlobals::RUBYRUN_KILL_3_STRING, RubyRunGlobals::RUBYRUN_LOG, RubyRunGlobals::RUBYRUN_MONITOR_TIMER, RubyRunGlobals::RUBYRUN_OPTS_FILE, RubyRunGlobals::RUBYRUN_OUTER_DISPATCH_HASH, RubyRunGlobals::RUBYRUN_OUTPUT_PERF_SUMMARY, RubyRunGlobals::RUBYRUN_OUTPUT_TXN_LOG, RubyRunGlobals::RUBYRUN_PREFIX, RubyRunGlobals::RUBYRUN_PREFIX_LENGTH, RubyRunGlobals::RUBYRUN_PROP_DEFAULTS, RubyRunGlobals::RUBYRUN_REPORT, RubyRunGlobals::RUBYRUN_SIGNATURE, RubyRunGlobals::RUBYRUN_STARTUP_ID_TYPE_PORT, RubyRunGlobals::RUBYRUN_STARTUP_ID_TYPE_PROCESS, RubyRunGlobals::RUBYRUN_THREAD_END_HASH, RubyRunGlobals::RUBYRUN_VIEW_HASH, RubyRunGlobals::RUBYRUN_WORKING_DIR_NAME

Instance Method Summary collapse

Methods included from RubyRunCommander__

#dump_object_map, #dump_reports, #dump_thread_status, #exit_monitor?, #hard_kill?, #kill_threads, #object_map?, #remove_cmd_folder, #return_joined_thread, #soft_kill?, #thread_status?, #unsupport_function, #update_perf_metrics

Methods included from RubyRunReport__

#add_perf_summary_rss_item, #add_txn_log_csv_item, #create_csv_files, #create_rss_channels

Methods included from RubyRunBufferMgr__

#push_current_buffer, #return_and_switch_buffer

Methods included from RubyRunTracer__

#back_trace_all, #enter_trace, #write_trace

Methods included from RubyRunUtils__

#env_var_exists?, #fatal_exit, #get_caller_detail, #get_thread_id, #is_action?, #is_application_controller, #is_in?, #is_rails_controller?, #return_method_name

Instance Method Details

#create_metrics_hash(klass) ⇒ Object

Create a place holding global metrics hash for each RAILS application controller class to accumulate performance metrics by action. Key elements in the hash are, for instance: #{controller => #{action => [dispatch_t, action_t, dbio_t, view_t, uncap_t, dispatch_wait_t]}}



117
118
119
 
# File 'lib/rubyrun/rubyrun_monitor__.rb', line 117

def create_metrics_hash(klass)
  $rubyrun_metrics_hash[klass.to_s.downcase[0..-11]] ||= {}
end

#create_thread_local(tid, request, klass, mid) ⇒ Object

Simulate a thread local storage by using a private hash keyed on thread id. Key elements in the hash are, for instance: #{tid => :controller=>name, :action=> name,

:action_t=>t, :dbio=>t, :dispatch_t=>t, :view_t=>t, :uncaptured_t=>t,
:dispatch_wait_t=>t}

Thread local is used to store performance metrics of a RAILS request. When the same thread serves a different request, the current thread local data needs to be rolled up and re-initialized.



103
104
105
106
107
108
109
110
111
 
# File 'lib/rubyrun/rubyrun_monitor__.rb', line 103

def create_thread_local(tid, request, klass, mid)
  return unless ($rubyrun_thread_local[tid] || is_action?(klass, mid))
  $rubyrun_thread_local[tid] ||= {}  
  init_thread_local(tid, request, klass, mid) unless $rubyrun_thread_local[tid][:req]
  if $rubyrun_thread_local[tid][:req] != request.object_id
    roll_up_metrics(tid)
    init_thread_local(tid, request, klass, mid)
  end
end

#report_rails_timing(klass, mid, t2, t1, tid) ⇒ Object

Report timings to thread local after decomposing it into the right component. For each action, there are 5 components of Response Time (RT)

:dispatch_t, :#{action}_t, :dbio_t, :view_t, :uncaptured_t



124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
 
# File 'lib/rubyrun/rubyrun_monitor__.rb', line 124

def report_rails_timing(klass, mid, t2, t1, tid)
  t = t2 - t1
  if is_in_hash?($rubyrun_adapter_hash, klass, mid)  
    $rubyrun_thread_local[tid][:dbio_t] += t
  elsif is_in_hash?(RUBYRUN_VIEW_HASH, klass, mid)
    $rubyrun_thread_local[tid][:view_t] << t2 << t1
  elsif is_in_hash?($rubyrun_outer_dispatch_hash, klass, mid)
    $rubyrun_thread_local[tid][:outer_dispatch_t] << t      
  elsif is_in_hash?($rubyrun_inner_dispatch_hash, klass, mid)
    $rubyrun_thread_local[tid][:inner_dispatch_t] << t
  elsif is_rails_controller?(klass)
    $rubyrun_thread_local[tid][:action_t] = t
    $rubyrun_thread_local[tid][:scafold_style] = $rubyrun_thread_local[tid][:view_t].empty? ? true : false
  elsif is_in?(RUBYRUN_THREAD_END_HASH, klass, mid, 'strict')
    roll_up_metrics(tid, true)   
  end
end

#start_thread_monitor ⇒ Object

In response to the presence of a ‘cmd_status’, ‘cmd_soft_kill’, ‘cmd_hard_kill’ or ‘cmd_object_map’ in the work directory, the monitor thread will either display thread status, interrupt the threads in different manner, or show object instances in memory



71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
 
# File 'lib/rubyrun/rubyrun_monitor__.rb', line 71

def start_thread_monitor
  $rubyrun_logger.info "----- RubyRun Thread Monitor started -----"
  $rubyrun_monitor_thr = Thread.new {
    cycle = $rubyrun_report_timer / RUBYRUN_MONITOR_TIMER
    sleep_count = 0
    loop do
      sleep RUBYRUN_MONITOR_TIMER
      $rubyrun_monitor_thr.exit if exit_monitor?
      Thread.new {
        begin
          sleep_count += 1
          sleep_count == cycle ? (dump_reports(true); sleep_count = 0) : dump_reports
          dump_thread_status if thread_status?
          dump_object_map if object_map?
          kill_threads($rubyrun_monitor_thr) if soft_kill? || hard_kill?
        rescue Exception => e
          $rubyrun_logger.error e.to_s + "\n" + e.backtrace.join("\n")
          exit(-1)
        end
      }
    end
  }
end