Class: Arachni::BrowserCluster::Worker
- Inherits:
-
Arachni::Browser
- Object
- Arachni::Browser
- Arachni::BrowserCluster::Worker
- Defined in:
- lib/arachni/browser_cluster/worker.rb
Overview
Overrides some Arachni::Browser methods to make multiple browsers play well with each other when they’re part of a Arachni::BrowserCluster.
Constant Summary collapse
- RESPAWN_WHEN_WINDOW_COUNT_REACHES =
Returns We can’t just close all windows because PhantomJS for some reason freezes after we do it a lot of times and we can’t just leave open windows accumulate, so we’ve got to take more drastic measures and kill it when the amount of open windows reaches or exceeds this setting.
5
Constants inherited from Arachni::Browser
Arachni::Browser::ASSET_EXTENSIONS, Arachni::Browser::ASSET_EXTRACTORS, Arachni::Browser::ELEMENT_APPEARANCE_TIMEOUT, Arachni::Browser::PHANTOMJS_SPAWN_TIMEOUT, Arachni::Browser::WATIR_COM_TIMEOUT
Instance Attribute Summary collapse
-
#job ⇒ Job
readonly
Currently assigned job.
- #job_timeout ⇒ Integer
- #master ⇒ BrowserCluster readonly
- #max_time_to_live ⇒ Integer
-
#time_to_live ⇒ Integer
readonly
Remaining time-to-live measured in jobs.
Attributes inherited from Arachni::Browser
#javascript, #last_dom_url, #last_url, #page_snapshots_with_sinks, #pid, #preloads, #transitions, #watir
Class Method Summary collapse
Instance Method Summary collapse
-
#distribute_event(page, element, event) ⇒ Object
Direct the distribution to the master and let it take it from there.
-
#initialize(options = {}) ⇒ Worker
constructor
A new instance of Worker.
- #inspect ⇒ Object
-
#run_job(job) ⇒ Array<Page>
Pages which resulted from firing events, clicking JavaScript links and capturing AJAX requests.
-
#shutdown(wait = true) ⇒ Object
Shuts down the worker.
-
#trigger_events ⇒ Object
We change the default scheduling to distribute elements and events to all available browsers ASAP, instead of building a list and then consuming it, since we’re don’t have to worry about messing up our page’s state in this setup.
Methods inherited from Arachni::Browser
add_asset_domain, asset_domains, #cache, #capture?, #capture_snapshot, #captured_pages, #clear_buffers, #cookies, #each_element_with_events, executable, #explore_and_flush, #fire_event, #flush_page_snapshots_with_sinks, #flush_pages, #goto, has_executable?, #load, #load_delay, #on_fire_event, #on_new_page, #on_new_page_with_sink, #on_response, #page_snapshots, #preload, #response, #selenium, #skip_path?, #snapshot_id, #source, #source_with_line_numbers, #start_capture, #stop_capture, #to_page, #trigger_event, #url, #wait_for_timers
Methods included from Support::Mixins::Observable
Methods included from Utilities
#available_port, #bytes_to_kilobytes, #bytes_to_megabytes, #caller_name, #caller_path, #cookie_decode, #cookie_encode, #cookies_from_document, #cookies_from_file, #cookies_from_response, #exception_jail, #exclude_path?, #follow_protocol?, #form_decode, #form_encode, #forms_from_document, #forms_from_response, #full_and_absolute_url?, #generate_token, #get_path, #hms_to_seconds, #html_decode, #html_encode, #include_path?, #links_from_document, #links_from_response, #normalize_url, #page_from_response, #page_from_url, #parse_set_cookie, #path_in_domain?, #path_too_deep?, #port_available?, #rand_port, #random_seed, #redundant_path?, #regexp_array_match, #remove_constants, #request_parse_body, #seconds_to_hms, #skip_page?, #skip_path?, #skip_resource?, #skip_response?, #to_absolute, #uri_decode, #uri_encode, #uri_parse, #uri_parse_query, #uri_parser, #uri_rewrite
Methods included from UI::Output
#debug?, #debug_off, #debug_on, #disable_only_positives, #included, #mute, #muted?, #only_positives, #only_positives?, #print_bad, #print_debug, #print_debug_backtrace, #print_debug_level_1, #print_debug_level_2, #print_debug_level_3, #print_error, #print_error_backtrace, #print_exception, #print_info, #print_line, #print_ok, #print_status, #print_verbose, #reroute_to_file, #reroute_to_file?, reset_output_options, #unmute, #verbose?, #verbose_on
Constructor Details
#initialize(options = {}) ⇒ Worker
Returns a new instance of Worker.
47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 |
# File 'lib/arachni/browser_cluster/worker.rb', line 47 def initialize( = {} ) @master = .delete( :master ) @max_time_to_live = .delete( :max_time_to_live ) || Options.browser_cluster.worker_time_to_live @time_to_live = @max_time_to_live @job_timeout = .delete( :job_timeout ) || Options.browser_cluster.job_timeout # Don't store pages if there's a master, we'll be sending them to him # as soon as they're logged. super .merge( store_pages: false ) @done_signal = Queue.new start_capture return if !@master start end |
Instance Attribute Details
#job ⇒ Job (readonly)
Returns Currently assigned job.
35 36 37 |
# File 'lib/arachni/browser_cluster/worker.rb', line 35 def job @job end |
#job_timeout ⇒ Integer
38 39 40 |
# File 'lib/arachni/browser_cluster/worker.rb', line 38 def job_timeout @job_timeout end |
#master ⇒ BrowserCluster (readonly)
31 32 33 |
# File 'lib/arachni/browser_cluster/worker.rb', line 31 def master @master end |
#max_time_to_live ⇒ Integer
41 42 43 |
# File 'lib/arachni/browser_cluster/worker.rb', line 41 def max_time_to_live @max_time_to_live end |
#time_to_live ⇒ Integer (readonly)
Returns Remaining time-to-live measured in jobs.
45 46 47 |
# File 'lib/arachni/browser_cluster/worker.rb', line 45 def time_to_live @time_to_live end |
Class Method Details
.name ⇒ Object
203 204 205 |
# File 'lib/arachni/browser_cluster/worker.rb', line 203 def self.name "BrowserCluster Worker##{object_id}" end |
Instance Method Details
#distribute_event(page, element, event) ⇒ Object
Direct the distribution to the master and let it take it from there.
158 159 160 161 162 163 164 165 166 167 168 169 170 |
# File 'lib/arachni/browser_cluster/worker.rb', line 158 def distribute_event( page, element, event ) master.queue @job.forward_as( @job.class::EventTrigger, resource: page, element: element, event: event ) true # Job may have been marked as done or the cluster may have been shut down. rescue BrowserCluster::Job::Error::AlreadyDone, BrowserCluster::Error::AlreadyShutdown false end |
#inspect ⇒ Object
194 195 196 197 198 199 200 201 |
# File 'lib/arachni/browser_cluster/worker.rb', line 194 def inspect s = "#<#{self.class} " s << "pid=#{@pid} " s << "job=#{@job.inspect} " s << "last-url=#{@last_url.inspect} " s << "transitions=#{@transitions.size}" s << '>' end |
#run_job(job) ⇒ Array<Page>
Returns Pages which resulted from firing events, clicking JavaScript links and capturing AJAX requests.
76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 |
# File 'lib/arachni/browser_cluster/worker.rb', line 76 def run_job( job ) @job = job print_debug "Started: #{@job}" # PhantomJS may have crashed (it happens sometimes) so make sure that # we've got a live one before running the job. # If we can't respawn, then bail out. return if browser_respawn_if_necessary.nil? begin with_timeout @job_timeout do exception_jail false do begin @job.configure_and_run( self ) rescue Selenium::WebDriver::Error::WebDriverError, Watir::Exception::Error => e print_debug "Error while processing job: #{@job}" print_debug print_debug_exception e browser_respawn end end end rescue TimeoutError => e print_bad "Job timed-out after #{@job_timeout} seconds: #{@job}" # Could have left us with a broken browser. browser_respawn end begin watir..clear # Working window was closed by JS (probably), start from scratch. rescue Selenium::WebDriver::Error::NoSuchWindowError browser_respawn end decrease_time_to_live browser_respawn_if_necessary print_debug "Finished: #{@job}" true rescue Selenium::WebDriver::Error::WebDriverError browser_respawn nil ensure @javascript.taint = nil clear_buffers # The jobs may have configured callbacks to capture pages etc., # remove them. clear_observers @job = nil end |
#shutdown(wait = true) ⇒ Object
If there is a running job it will wait for it to finish.
Shuts down the worker.
175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 |
# File 'lib/arachni/browser_cluster/worker.rb', line 175 def shutdown( wait = true ) return if @shutdown @shutdown = true # Keep checking to see if any of the 'done' criteria are true. kill_check = Thread.new do sleep 0.1 while browser_alive? && wait && @job @done_signal << nil end # If we've got a job running wait for it to finish before closing the # browser otherwise we'll get Selenium errors and zombie processes. @done_signal.pop kill_check.join @consumer.kill if @consumer super() end |
#trigger_events ⇒ Object
We change the default scheduling to distribute elements and events to all available browsers ASAP, instead of building a list and then consuming it, since we’re don’t have to worry about messing up our page’s state in this setup.
142 143 144 145 146 147 148 149 150 151 152 |
# File 'lib/arachni/browser_cluster/worker.rb', line 142 def trigger_events root_page = to_page each_element_with_events do |element, events| events.each do |name, _| distribute_event( root_page, element, name.to_sym ) end end true end |