Class: Arachni::RPC::Server::Instance

Inherits:
Object
  • Object
show all
Includes:
UI::Output, Utilities
Defined in:
lib/arachni/rpc/server/instance.rb

Overview

Note:

Ignore:

  • Inherited methods and attributes -- only public methods of this class are accessible over RPC.
  • block parameters, they are an RPC implementation detail for methods which perform asynchronous operations.
Note:

Methods which expect Symbol type parameters will also accept String types as well.

For example, the following:

instance.service.scan url: 'http://testfire.net'

Is the same as:

instance.service.scan 'url' => 'http://testfire.net'

Represents an Arachni instance (or multiple instances when running a multi-Instance scan) and serves as a central point of access and control.

Methods

Provides methods for:

(A nice simple example can be found in the RPC command-line client interface.)

Examples:

A minimalistic example -- assumes Arachni is installed and available.

require 'arachni'
require 'arachni/rpc/client'

instance = Arachni::RPC::Client::Instance.new( Arachni::Options.instance,
                                               'localhost:1111', 's3cr3t' )

instance.service.scan url: 'http://testfire.net',
                      audit:  {
                          elements: [:links, :forms]
                      },
                      # load all XSS checks
                      checks: 'xss*'

print 'Running.'
while instance.service.busy?
    print '.'
    sleep 1
end

# Grab the report
report = instance.service.report

# Kill the instance and its process, no zombies please...
instance.service.shutdown

puts
puts
puts 'Logged issues:'
report['issues'].each do |issue|
   puts "  * #{issue['name']} in '#{issue['vector']['type']}' input '#{issue['vector']['affected_input_name']}' at '#{issue['vector']['action']}'."
end

Author:

Instance Method Summary collapse

Methods included from Utilities

#available_port, available_port_mutex, #bytes_to_kilobytes, #bytes_to_megabytes, #caller_name, #caller_path, #cookie_decode, #cookie_encode, #cookies_from_file, #cookies_from_parser, #cookies_from_response, #exception_jail, #exclude_path?, #follow_protocol?, #form_decode, #form_encode, #forms_from_parser, #forms_from_response, #full_and_absolute_url?, #generate_token, #get_path, #hms_to_seconds, #html_decode, #html_encode, #include_path?, #links_from_parser, #links_from_response, #normalize_url, #page_from_response, #page_from_url, #parse_set_cookie, #path_in_domain?, #path_too_deep?, #port_available?, #rand_port, #random_seed, #redundant_path?, #regexp_array_match, #remove_constants, #request_parse_body, #seconds_to_hms, #skip_page?, #skip_path?, #skip_resource?, #skip_response?, #to_absolute, #uri_decode, #uri_encode, #uri_parse, #uri_parse_query, #uri_parser, #uri_rewrite

Methods included from UI::Output

#caller_location, #debug?, #debug_level, #debug_level_1?, #debug_level_2?, #debug_level_3?, #debug_level_4?, #debug_off, #debug_on, #disable_only_positives, #error_buffer, #error_log_fd, #error_logfile, #has_error_log?, #included, #log_error, #mute, #muted?, #only_positives, #only_positives?, #print_bad, #print_debug, #print_debug_backtrace, #print_debug_exception, #print_debug_level_1, #print_debug_level_2, #print_debug_level_3, #print_debug_level_4, #print_error, #print_error_backtrace, #print_exception, #print_info, #print_line, #print_ok, #print_status, #print_verbose, #reroute_to_file, #reroute_to_file?, reset_output_options, #set_error_logfile, #unmute, #verbose?, #verbose_off, #verbose_on

Constructor Details

#initialize(options, token) ⇒ Instance

Initializes the RPC interface and the framework.

Parameters:



118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
 
# File 'lib/arachni/rpc/server/instance.rb', line 118

def initialize( options, token )
    @options = options
    @token   = token

    @options.snapshot.save_path ||= @options.paths.snapshots

    @framework      = Server::Framework.new( Options.instance )
    @active_options = Server::ActiveOptions.new( @framework )

    @server = Base.new( @options, token )
    @server.logger.level = @options.datastore.log_level if @options.datastore.log_level

    @options.datastore.token = token

    if @options.output.reroute_to_logfile
        reroute_to_file "#{@options.paths.logs}/Instance - #{Process.pid}" <<
                            "-#{@options.rpc.server_port}.log"
    else
        reroute_to_file false
    end

    set_error_logfile "#{@options.paths.logs}/Instance - #{Process.pid}" <<
                          "-#{@options.rpc.server_port}.error.log"

    set_handlers( @server )

    # trap interrupts and exit cleanly when required
    %w(QUIT INT).each do |signal|
        next if !Signal.list.has_key?( signal )
        trap( signal ){ shutdown if !@options.datastore.do_not_trap }
    end

    @consumed_pids = []

    Reactor.global.run do
        run
    end
end

Instance Method Details

#abort_and_report(&block) ⇒ Hash

Note:

Don't forget to #shutdown the instance once you get the report.

Cleans up and returns the report.

Returns:

See Also:



295
296
297
 
# File 'lib/arachni/rpc/server/instance.rb', line 295

def abort_and_report( &block )
    @framework.clean_up { block.call report.to_h }
end

#abort_and_report_as(name, &block) ⇒ Object

Note:

Don't forget to #shutdown the instance once you get the report.

Cleans up and delegates to #report_as.

Parameters:

  • name (String) —

    Name of the report component to run, as presented by #list_reporters's :shortname key.

See Also:



316
317
318
 
# File 'lib/arachni/rpc/server/instance.rb', line 316

def abort_and_report_as( name, &block )
    @framework.clean_up { block.call report_as( name ) }
end

#alive? ⇒ true

Returns:

  • (true) 


200
201
202
 
# File 'lib/arachni/rpc/server/instance.rb', line 200

def alive?
    @server.alive?
end

#busy?(&block) ⇒ Bool

Returns true if the scan is initializing or running, false otherwise.

Returns:

  • (Bool) —

    true if the scan is initializing or running, false otherwise.



206
207
208
209
210
211
212
213
 
# File 'lib/arachni/rpc/server/instance.rb', line 206

def busy?( &block )
    if @scan_initializing
        block.call( true ) if block_given?
        return true
    end

    @framework.busy?( &block )
end

#errors(starting_line = 0, &block) ⇒ Array<String>

Parameters:

  • starting_line (Integer) (defaults to: 0) —

    Sets the starting line for the range of errors to return.

Returns:



217
218
219
 
# File 'lib/arachni/rpc/server/instance.rb', line 217

def errors( starting_line = 0, &block )
    @framework.errors( starting_line, &block )
end

#list_checks ⇒ Object 



233
234
235
 
# File 'lib/arachni/rpc/server/instance.rb', line 233

def list_checks
    @framework.list_checks
end

#list_platforms ⇒ Object 



228
229
230
 
# File 'lib/arachni/rpc/server/instance.rb', line 228

def list_platforms
    @framework.list_platforms
end

#list_plugins ⇒ Object 



238
239
240
 
# File 'lib/arachni/rpc/server/instance.rb', line 238

def list_plugins
    @framework.list_plugins
end

#list_reporters ⇒ Object 



243
244
245
 
# File 'lib/arachni/rpc/server/instance.rb', line 243

def list_reporters
    @framework.list_reporters
end

#pause(&block) ⇒ Object

Pauses the running scan on a best effort basis.



253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
 
# File 'lib/arachni/rpc/server/instance.rb', line 253

def pause( &block )
    if @rpc_pause_request
        block.call( true )
        return
    end

    # Send the pause request but don't block.
    r = @framework.pause( false )
    @rpc_pause_request ||= r

    if !@framework.has_slaves?
        block.call( true )
        return
    end

    each = proc { |instance, iter| instance.service.pause { iter.next } }
    each_slave( each, proc { block.call true } )
end

#paused? ⇒ Boolean

Returns:

  • (Boolean) 


248
249
250
 
# File 'lib/arachni/rpc/server/instance.rb', line 248

def paused?
    @framework.paused?
end

#progress(options = {}, &block) ⇒ Hash

Recommended usage

Please request from the method only the things you are going to actually use, otherwise you'll just be wasting bandwidth. In addition, ask to not be served data you already have, like issues or error messages.

To be kept completely up to date on the progress of a scan (i.e. receive new issues and error messages asap) in an efficient manner, you will need to keep track of the issues and error messages you already have and explicitly tell the method to not send the same data back to you on subsequent calls.

Retrieving errors (:errors option) without duplicate data

This is done by telling the method how many error messages you already have and you will be served the errors from the error-log that are past that line. So, if you were to use a loop to get fresh progress data it would look like so:

error_cnt = 0
i = 0
while sleep 1
    # Test method, triggers an error log...
    instance.service.error_test "BOOM! #{i+=1}"

    # Only request errors we don't already have
    errors = instance.service.progress( with: { errors: error_cnt } )[:errors]
    error_cnt += errors.size

    # You will only see new errors
    puts errors.join("\n")
end

Retrieving issues without duplicate data

In order to be served only new issues you will need to let the method know which issues you already have. This is done by providing a list of digests for the issues you already know about.

issue_digests = []
while sleep 1
    issues = instance.service.progress(
                 with: :issues,
                 # Only request issues we don't already have
                 without: { issues: issue_digests  }
             )[:issues]

    issue_digests |= issues.map { |issue| issue['digest'] }

    # You will only see new issues
    issues.each do |issue|
        puts "  * #{issue['name']} in '#{issue['vector']['type']}' input '#{issue['vector']['affected_input_name']}' at '#{issue['vector']['action']}'."
    end
end

Parameters:

  • options (Hash) (defaults to: {}) —

    Options about what progress data to retrieve and return.

Options Hash (options):

  • :with (Array<Symbol, Hash>) —

    Specify data to include:

    • :issues -- Discovered issues as hashes.
    • :instances -- Statistics and info for slave instances.
    • :errors -- Errors and the line offset to use for #errors. Pass as a hash, like: { errors: 10 }
  • :without (Array<Symbol, Hash>) —

    Specify data to exclude:

    • :statistics -- Don't include runtime statistics.
    • :issues -- Don't include issues with the given digests. Pass as a hash, like: { issues: [...] }

Returns:

  • (Hash) —
    • statistics -- General runtime statistics (merged when part of Grid) (enabled by default)
    • status -- #status
    • busy -- #busy?
    • issues -- Discovered issues as hashes. (disabled by default)
    • instances -- Raw statistics for each running instance (only when part of Grid) (disabled by default)
    • errors -- #errors (disabled by default)
    • sitemap -- #sitemap (disabled by default)


430
431
432
 
# File 'lib/arachni/rpc/server/instance.rb', line 430

def progress( options = {}, &block )
    progress_handler( options.merge( as_hash: true ), &block )
end

#report ⇒ Hash

Returns Arachni::Report#to_h.

Returns:



328
329
330
 
# File 'lib/arachni/rpc/server/instance.rb', line 328

def report
    @framework.report.to_h
end

#report_as(name) ⇒ Object

Parameters:

  • name (String) —

    Name of the report component to run, as presented by #list_reporters's :shortname key.



337
338
339
 
# File 'lib/arachni/rpc/server/instance.rb', line 337

def report_as( name )
    @framework.report_as( name )
end

#restore(snapshot) ⇒ Object

See Also:



187
188
189
190
191
 
# File 'lib/arachni/rpc/server/instance.rb', line 187

def restore( snapshot )
    @framework.restore snapshot
    @framework.run
    true
end

#resume(&block) ⇒ Object

Resumes a paused scan right away.



273
274
275
276
277
278
279
280
281
282
283
284
285
286
 
# File 'lib/arachni/rpc/server/instance.rb', line 273

def resume( &block )
    return block.call( false ) if !@rpc_pause_request

    @framework.resume( @rpc_pause_request )

    if !@framework.has_slaves?
        @rpc_pause_request = nil
        block.call true
        return
    end

    each = proc { |instance, iter| instance.service.resume { iter.next } }
    each_slave( each, proc { @rpc_pause_request = nil; block.call true } )
end

#scan(opts = {}, &block) ⇒ Object

Note:

Options marked with an asterisk are required.

Note:

Options which expect patterns will interpret their arguments as regular expressions regardless of their type.

Configures and runs a scan.

Parameters:

  • opts (Hash) (defaults to: {}) —

    Scan options to be passed to Options#update (along with some extra ones to keep configuration in one place).

    The options presented here are the most commonly used ones, in actuality, you can use anything supported by Options#update.

Options Hash (opts):

  • *:url (String) —

    Target URL to audit.

  • :authorized_by (String) — default: nil —

    The e-mail address of the person who authorized the scan.

    john.doe@bigscanners.com
  • :audit (Hash) —

    Audit options.

  • :scope (Hash) —

    Scope options.

  • :http (Hash) —

    HTTP options.

  • :login (Hash) —

    Session options.

  • :checks (String, Array<String>) — default: [] —

    Checks to load, by name.

    # To load all checks use the wildcard on its own
'*'

# To load all XSS and SQLi checks:
[ 'xss*', 'sql_injection*' ]
  • :plugins (Hash<Hash>) — default: {} —

    Plugins to load, by name, along with their options.

    {
    'proxy'      => {}, # empty options
    'autologin'  => {
        'url'         => 'http://demo.testfire.net/bank/login.aspx',
        'parameters' => 'uid=jsmith&passw=Demo1234',
        'check'       => 'MY ACCOUNT'
    },
}
  • :platforms (String, Symbol, Array<String, Symbol>) — default: [] —

    Initialize the fingerprinter with the given platforms.

    The fingerprinter cannot identify database servers so specifying the remote DB backend will greatly enhance performance and reduce bandwidth consumption.

  • :no_fingerprinting (Bool) — default: false —

    Disable platform fingerprinting and include all payloads in the audit.

    Use this option in addition to the :platforms one to restrict the audit payloads to explicitly specified platforms.

  • :grid (Bool) — default: false —

    Use the Dispatcher Grid to load-balance scans across the available nodes.

    If set to true, it serves as a shorthand for:

    grid_mode: :balance
  • :grid_mode (String, Symbol) — default: nil —

    Grid mode to use, available modes are:

    • nil -- No grid.
    • :balance -- Slave Instances will be provided by the least burdened grid members to keep the overall Grid workload even across all Dispatchers.
    • :aggregate -- Used to perform a multi-Instance scan and will only request Instances from Grid members with different Pipe-IDs, resulting in application-level bandwidth aggregation.
  • :spawns (Integer) — default: 0 —

    The amount of slaves to spawn. The behavior of this option changes depending on the grid_mode setting:

    • nil -- All slave Instances will be spawned by this Instance directly, and thus reside in the same machine.
    • :balance -- Slaves will be provided by the least burdened Grid Dispatchers.
    • :aggregate -- Slaves will be provided by Grid Dispatchers with unique Pipe-IDs and the value of this option will be treated as a possible maximum rather than a hard setting. Actual spawn count will be determined by Dispatcher availability at the time.


528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
 
# File 'lib/arachni/rpc/server/instance.rb', line 528

def scan( opts = {}, &block )
    # If the instance isn't clean bail out now.
    if busy? || @called
        block.call false
        return false
    end

    # Normalize this sucker to have symbols as keys.
    opts = opts.my_symbolize_keys( false )

    slaves      = opts.delete(:slaves) || []
    spawn_count = opts[:spawns]
    spawn_count = spawn_count.to_i

    if (platforms = opts.delete(:platforms))
        begin
            Platform::Manager.new( [platforms].flatten.compact )
        rescue => e
            fail ArgumentError, e.to_s
        end
    end

    opts[:dispatcher] ||= {}
    opts[:scope]      ||= {}

    if opts[:grid] || opts[:grid_mode]
        if spawn_count <= 0
            fail ArgumentError,
                 'Option \'spawns\' must be greater than 1 for Grid scans.'
        end

        if [opts[:scope]['restrict_paths']].flatten.compact.any?
            fail ArgumentError,
                 'Scope option \'restrict_paths\' is not supported when in' <<
                     ' multi-Instance mode.'
        end
    end

    # There may be follow-up/retry calls by the client in cases of network
    # errors (after the request has reached us) so we need to keep minimal
    # track of state in order to bail out on subsequent calls.
    @called = @scan_initializing = true

    # Plugins option needs to be a hash...
    if opts[:plugins] && opts[:plugins].is_a?( Array )
        opts[:plugins] = opts[:plugins].inject( {} ) { |h, n| h[n] = {}; h }
    end

    if opts.include?( :grid )
        @framework.options.dispatcher.grid = opts.delete(:grid)
    end

    if opts.include?( :grid_mode )
        @framework.options.dispatcher.grid_mode = opts.delete(:grid_mode)
    end

    @active_options.set( opts )

    if @framework.options.url.to_s.empty?
        fail ArgumentError, 'Option \'url\' is mandatory.'
    end

    @framework.checks.load opts[:checks] if opts[:checks]
    @framework.plugins.load opts[:plugins] if opts[:plugins]

    # Starts the scan after all necessary options have been set.
    after = proc { block.call @framework.run; @scan_initializing = false }

    if @framework.options.dispatcher.grid?
        # If a Grid scan has been selected then just set us as the master,
        # the Framework will sort out the rest.
        @framework.set_as_master

        # Rock n' roll!
        after.call
    else
        # Handles each spawn, enslaving it for a multi-Instance scan.
        each = proc do |slave, iter|
            @framework.enslave( slave ){ iter.next }
        end

        spawn( spawn_count ) do |spawns|
            # Add our spawns to the slaves list which was passed as an option.
            slaves |= spawns

            # Process the Instances.
            Reactor.global.create_iterator( slaves, slaves.empty? ? 1 : slaves.size ).
                each( each, after )
        end
    end

    true
end

#shutdown(&block) ⇒ Object

Makes the server go bye-bye...Lights out!



623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
 
# File 'lib/arachni/rpc/server/instance.rb', line 623

def shutdown( &block )
    if @shutdown
        block.call if block_given?
        return
    end
    @shutdown = true

    print_status 'Shutting down...'

    # We're shutting down services so we need to use a concurrent way but
    # without going through the Reactor.
    Thread.new do
        t = []

        if browser_cluster
            # We can't block until the browser cluster shuts down cleanly
            # (i.e. wait for any running jobs) but we don't need to anyways.
            t << Thread.new { browser_cluster.shutdown false }
        end

        if browser
            t << Thread.new { browser.shutdown }
        end

        @framework.instance_eval do
            next if !has_slaves?

            @slaves.each do |instance|
                t << Thread.new { connect_to_instance( instance ).service.shutdown }
            end
        end

        t.each(&:join)
        @server.shutdown

        block.call true if block_given?
    end

    true
end

#sitemap(index = 0) ⇒ Object 



223
224
225
 
# File 'lib/arachni/rpc/server/instance.rb', line 223

def sitemap( index = 0 )
    @framework.sitemap_entries( index )
end

#snapshot_path ⇒ String?

Returns Path to the snapshot of the suspended scan, nil if not #suspended?.

Returns:

See Also:



163
164
165
166
 
# File 'lib/arachni/rpc/server/instance.rb', line 163

def snapshot_path
    return if !suspended?
    @framework.snapshot_path
end

#status ⇒ Object 



342
343
344
 
# File 'lib/arachni/rpc/server/instance.rb', line 342

def status
    @framework.status
end

#suspend ⇒ Object

Note:

The path to the snapshot can be retrieved via #snapshot_path.

Writes a Snapshot to disk and aborts the scan.

See Also:



173
174
175
176
177
178
179
180
 
# File 'lib/arachni/rpc/server/instance.rb', line 173

def suspend
    if !@framework.solo?
        fail State::Framework::Error::StateNotSuspendable,
             'Cannot suspend a multi-Instance scan.'
    end

    @framework.suspend false
end

#suspended? ⇒ Boolean

Returns:

  • (Boolean)

See Also:



195
196
197
 
# File 'lib/arachni/rpc/server/instance.rb', line 195

def suspended?
    @framework.suspended?
end