Module: RubyRunMonitor__

Includes:
RubyRunBufferMgr__, RubyRunCommander__, RubyRunGlobals, RubyRunHTML__, RubyRunReport__, RubyRunTracer__, RubyRunUtils__
Included in:
RubyRunInstrumentor__
Defined in:
lib/rubyrun/rubyrun_monitor__.rb

Overview

—————————————————————#

                                                             #  
(C) Copyright Rubysophic Inc. 2007-2008                      #
All rights reserved.                                         #
                                                             #  
Duplication or disclosure of the code is not permitted       #
unless licensed.                                             #
                                                             #            
Last Updated: 4/21/10                                        #
                                                             #
In a JRuby-Rack environment Rubyrun is to be run as a Rails  #
preinitilaizer.rb. This allows rubyrun to sit inside any     #
JRuby container inside a J2EE servlet container such as      #
Tomcat and Glassfish.                                        #                                                             #

—————————————————————#

                                                             #  
RubyRunMonitor__ is responsible for keeping track of         #
the response time of a Rails Request and its decomposition.  #
                                                             #  
For instance, a typical WEBrick/RAILS servlet dispatch       #
response time components generally break down look roughly   # 
like this:           						    #
                                                             #      
SVS   DI   ACT   DB   DB    DB          VI    VI             #    
-+-----+----+----+--+--+--+--+--+----+--+----+-+--+---+----+ #
                   DB'   DB'   DB'  ACT'    VI'  VI' DI' SVS'#            
Legend:                                                      #
 SVS = DispatchServlet.service                               #
 DI = DispatchServlet.handle_dispatch                        #
 ACT = controller.action                                     #    
 DB  = ActiveRecord::ConnectionAdapters::*Adapter.execute    #
 VI  = ActionView.Base:render_template                       #
 All apostrophe means "end of event"                         #
                                                             #
 Notes:                                                      #  
 1. mutex held from SVS to SVS'                              #
 2. Total dispatch time with mutex = SVS' - SVS              #
 3. Total DBIO time = Sum(DB' - DB)                          #    
 4. Action Time = ACT' - ACT                                 #
 5. View time = Sum(VI' - VI)                                #
 6. View rendering includes template + layout                #
                                                             #      
Another key function is to act as a command agent, responding#
to command such as displaying thread status, terminating     #
threads with stack tracke and showing object heap info.      #
                                                             #

—————————————————————#

Constant Summary

Constants included from RubyRunHTML__

RubyRunHTML__::METHOD_TRACE_EVEN_ROW, RubyRunHTML__::METHOD_TRACE_HEADER, RubyRunHTML__::METHOD_TRACE_ODD_ROW, RubyRunHTML__::OBJ_MAP_EVEN_ROW, RubyRunHTML__::OBJ_MAP_HTML, RubyRunHTML__::OBJ_MAP_ODD_ROW, RubyRunHTML__::REQ_PERF_BREAKDOWN_HTML, RubyRunHTML__::REQ_PERF_BREAKDOWN_TABLE_EVEN, RubyRunHTML__::REQ_PERF_BREAKDOWN_TABLE_ODD, RubyRunHTML__::THREAD_STATUS_EVEN_ROW, RubyRunHTML__::THREAD_STATUS_HTML, RubyRunHTML__::THREAD_STATUS_ODD_ROW, RubyRunHTML__::THROUGHPUT_BAR_TABLE, RubyRunHTML__::THROUGHPUT_HTML, RubyRunHTML__::THROUGHPUT_LABEL_TABLE, RubyRunHTML__::TOP_SLOWEST_REQUESTS_HTML, RubyRunHTML__::TOP_SLOWEST_REQUESTS_TABLE

Constants included from RubyRunGlobals

RubyRunGlobals::RUBYRUN_ACTIVERECORD, RubyRunGlobals::RUBYRUN_CMD_EXIT, RubyRunGlobals::RUBYRUN_CMD_HARD_KILL, RubyRunGlobals::RUBYRUN_CMD_OBJECT_MAP, RubyRunGlobals::RUBYRUN_CMD_SOFT_KILL, RubyRunGlobals::RUBYRUN_CMD_STATUS, RubyRunGlobals::RUBYRUN_DIR_HASH_FILE, RubyRunGlobals::RUBYRUN_DOC_DIR, RubyRunGlobals::RUBYRUN_ETC_DIR, RubyRunGlobals::RUBYRUN_EXCLUDE_HASH_FILE, RubyRunGlobals::RUBYRUN_FIREWALL_HASH, RubyRunGlobals::RUBYRUN_HIGHLIGHT_THRESHOLD, RubyRunGlobals::RUBYRUN_INCLUDE_HASH_FILE, RubyRunGlobals::RUBYRUN_INNER_DISPATCH_HASH, RubyRunGlobals::RUBYRUN_KILL_3_STRING, RubyRunGlobals::RUBYRUN_LOG, RubyRunGlobals::RUBYRUN_MONITOR_TIMER, RubyRunGlobals::RUBYRUN_OPTS_FILE, RubyRunGlobals::RUBYRUN_OUTER_DISPATCH_HASH, RubyRunGlobals::RUBYRUN_OUTPUT_PERF_SUMMARY, RubyRunGlobals::RUBYRUN_OUTPUT_TXN_LOG, RubyRunGlobals::RUBYRUN_PREFIX, RubyRunGlobals::RUBYRUN_PREFIX_LENGTH, RubyRunGlobals::RUBYRUN_PROP_DEFAULTS, RubyRunGlobals::RUBYRUN_REPORT, RubyRunGlobals::RUBYRUN_SIGNATURE, RubyRunGlobals::RUBYRUN_STARTUP_ID_TYPE_PORT, RubyRunGlobals::RUBYRUN_STARTUP_ID_TYPE_PROCESS, RubyRunGlobals::RUBYRUN_THREAD_END_HASH, RubyRunGlobals::RUBYRUN_VIEW_HASH, RubyRunGlobals::RUBYRUN_WORKING_DIR_NAME

Instance Method Summary collapse

Methods included from RubyRunCommander__

#dump_object_map, #dump_reports, #dump_thread_status, #exit_monitor?, #hard_kill?, #kill_threads, #object_map?, #remove_cmd_folder, #return_joined_thread, #soft_kill?, #thread_status?, #unsupport_function, #update_perf_metrics

Methods included from RubyRunReport__

#add_perf_summary_rss_item, #add_txn_log_csv_item, #create_csv_files, #create_rss_channels

Methods included from RubyRunBufferMgr__

#push_current_buffer, #return_and_switch_buffer

Methods included from RubyRunTracer__

#back_trace_all, #enter_trace, #write_trace

Methods included from RubyRunUtils__

#env_var_exists?, #fatal_exit, #get_caller_detail, #get_thread_id, #is_action?, #is_application_controller, #is_in?, #is_rails_controller?, #return_class_name, #return_method_name, #return_mid

Instance Method Details

#create_metrics_hash(klass) ⇒ Object

Create a place holding global metrics hash for each RAILS application controller class to accumulate performance metrics by action. Key elements in the hash are, for instance: #{controller => #{action => [dispatch_t, action_t, dbio_t, view_t, uncap_t, dispatch_wait_t]}}



122
123
124
 
# File 'lib/rubyrun/rubyrun_monitor__.rb', line 122

def create_metrics_hash(klass)
  $rubyrun_metrics_hash[klass.to_s.downcase[0..-11]] ||= {}
end

#create_thread_local(tid, request, klass, mid) ⇒ Object

Simulate a thread local storage by using a private hash keyed on thread id. Key elements in the hash are, for instance: #{tid => :controller=>name, :action=> name,

:action_t=>t, :dbio=>t, :dispatch_t=>t, :view_t=>t, :uncaptured_t=>t,
:dispatch_wait_t=>t}

Thread local is used to store performance metrics of a RAILS request. When the same thread serves a different request, the current thread local data needs to be rolled up and re-initialized.



108
109
110
111
112
113
114
115
116
 
# File 'lib/rubyrun/rubyrun_monitor__.rb', line 108

def create_thread_local(tid, request, klass, mid)
  return unless ($rubyrun_thread_local[tid] || is_action?(klass, mid))
  $rubyrun_thread_local[tid] ||= {}  
  init_thread_local(tid, request, klass, mid) unless $rubyrun_thread_local[tid][:req]
  if $rubyrun_thread_local[tid][:req] != request.object_id
    roll_up_metrics(tid)
    init_thread_local(tid, request, klass, mid)
  end
end

#report_rails_timing(klass, mid, t2, t1, tid) ⇒ Object

Report timings to thread local after decomposing it into the right component. For each action, there are 5 components of Response Time (RT)

:dispatch_t, :#{action}_t, :dbio_t, :view_t, :uncaptured_t



129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
 
# File 'lib/rubyrun/rubyrun_monitor__.rb', line 129

def report_rails_timing(klass, mid, t2, t1, tid)
  t = t2 - t1
  if is_in_hash?($rubyrun_adapter_hash, klass, mid)  
    $rubyrun_thread_local[tid][:dbio_t] += t
  elsif is_in_hash?(RUBYRUN_VIEW_HASH, klass, mid)
    $rubyrun_thread_local[tid][:view_t] << t2 << t1
  elsif is_in_hash?($rubyrun_outer_dispatch_hash, klass, mid)
    $rubyrun_thread_local[tid][:outer_dispatch_t] << t      
  elsif is_in_hash?($rubyrun_inner_dispatch_hash, klass, mid)
    $rubyrun_thread_local[tid][:inner_dispatch_t] << t
  elsif is_rails_controller?(klass)
    $rubyrun_thread_local[tid][:action_t] = t
    $rubyrun_thread_local[tid][:scafold_style] = $rubyrun_thread_local[tid][:view_t].empty? ? true : false
  end
  # For Rails 2.3.5, class/method for thread end is the same as outer dispatch
  # So there has to be a separate logic to check if this is a thread end
  if is_in?(RUBYRUN_THREAD_END_HASH, klass, mid, 'strict')
    roll_up_metrics(tid, true)   
  end

  # For a J2EE environment, there are no explicit collection of outer dispatch
  # or thread end processing because RubyRun is booted late in preintializer
  # We need to roll the metrics at the end of a inner dispath
  if $j2ee && $rubyrun_thread_local[tid][:inner_dispatch_t].size > 0
    roll_up_metrics(tid, true)
  end
end

#start_thread_monitor ⇒ Object

In response to the presence of a ‘cmd_status’, ‘cmd_soft_kill’, ‘cmd_hard_kill’ or ‘cmd_object_map’ in the work directory, the monitor thread will either display thread status, interrupt the threads in different manner, or show object instances in memory



76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
 
# File 'lib/rubyrun/rubyrun_monitor__.rb', line 76

def start_thread_monitor
  $rubyrun_logger.info "----- RubyRun Thread Monitor started -----"
  $rubyrun_monitor_thr = Thread.new {
    cycle = $rubyrun_report_timer / RUBYRUN_MONITOR_TIMER
    sleep_count = 0
    loop do
      sleep RUBYRUN_MONITOR_TIMER
      $rubyrun_monitor_thr.exit if exit_monitor?
      Thread.new {
        begin
          sleep_count += 1
          sleep_count == cycle ? (dump_reports(true); sleep_count = 0) : dump_reports
          dump_thread_status if thread_status?
          dump_object_map if object_map?
          kill_threads($rubyrun_monitor_thr) if soft_kill? || hard_kill?
        rescue Exception => e
          $rubyrun_logger.error e.to_s + "\n" + e.backtrace.join("\n")
          exit(-1)
        end
      }
    end
  }
end