Class: Arachni::RPC::Server::Framework

Inherits:
Framework show all
Includes:
MultiInstance, Utilities
Defined in:
lib/arachni/rpc/server/framework.rb,
lib/arachni/rpc/server/framework/slave.rb,
lib/arachni/rpc/server/framework/master.rb,
lib/arachni/rpc/server/framework/distributor.rb,
lib/arachni/rpc/server/framework/multi_instance.rb

Overview

Note:

Ignore:

  • Inherited methods and attributes – only public methods of this class are

    accessible over RPC.

  • ‘block` parameters, they are an RPC implementation detail for methods which

    perform asynchronous operations.

Wraps the framework of the local instance and the frameworks of all its slaves (when it is a Master in multi-Instance mode) into a neat, easy to handle package.

Author:

Defined Under Namespace

Modules: Distributor, Master, MultiInstance, Slave Classes: Error

Constant Summary

Constants included from Distributor

Distributor::MAX_CONCURRENCY, Distributor::MIN_PAGES_PER_INSTANCE

Constants inherited from Framework

Framework::AUDIT_PAGE_MAX_TRIES, Framework::REVISION

Instance Attribute Summary

Attributes inherited from Framework

#failures, #http, #modules, #opts, #page_queue_total_size, #plugins, #reports, #session, #sitemap, #spider, #trainer, #url_queue_total_size

Instance Method Summary collapse

Methods included from MultiInstance

#errors, #multi_self_url, #progress, #restrict_to_elements, #solo?, #update_page_queue

Methods included from Master

#enslave, #master?, #set_as_master, #slave_done, #slave_sitrep, #update_element_ids_per_url, #update_issues

Methods included from Slave

#set_master, #slave?

Methods included from Distributor

#connect_to_instance

Methods included from Utilities

#available_port, #cookie_encode, #cookies_from_document, #cookies_from_file, #cookies_from_response, #exception_jail, #exclude_path?, #extract_domain, #follow_protocol?, #form_decode, #form_encode, #form_parse_request_body, #forms_from_document, #forms_from_response, #generate_token, #get_path, #html_decode, #html_encode, #include_path?, #links_from_document, #links_from_response, #normalize_url, #page_from_response, #page_from_url, #parse_query, #parse_set_cookie, #parse_url_vars, #path_in_domain?, #path_too_deep?, #port_available?, #rand_port, #redundant_path?, #remove_constants, #seed, #skip_page?, #skip_path?, #skip_resource?, #to_absolute, #uri_decode, #uri_encode, #uri_parse, #uri_parser, #url_sanitize

Methods inherited from Framework

#audit_page, #audit_store, #link_count_limit_reached?, #list_modules, #list_platforms, #on_audit_page, #paused?, #push_to_page_queue, #push_to_url_queue, #report_as, reset, #reset, #reset_spider, #reset_trainer, #revision, #running?, #status, #version

Methods included from Mixins::Observable

#clear_observers, #method_missing

Methods included from UI::Output

#debug?, #debug_off, #debug_on, #disable_only_positives, #error_logfile, #flush_buffer, #log_error, #mute, #muted?, old_reset_output_options, #only_positives, #only_positives?, #print_bad, #print_debug, #print_debug_backtrace, #print_debug_pp, #print_error, #print_error_backtrace, #print_info, #print_line, #print_ok, #print_status, #print_verbose, #reroute_to_file, #reroute_to_file?, reset_output_options, #set_buffer_cap, #set_error_logfile, #uncap_buffer, #unmute, #verbose, #verbose?

Constructor Details

#initialize ⇒ Framework

Returns a new instance of Framework.



86
87
88
89
90
91
92
93
 
# File 'lib/arachni/rpc/server/framework.rb', line 86

def initialize( * )
    super

    # Override standard framework components with their RPC-server counterparts.
    @modules = Module::Manager.new( self )
    @plugins = Plugin::Manager.new( self )
    @spider  = Spider.new( self )
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method in the class Arachni::Mixins::Observable

Instance Method Details

#busy?(&block) ⇒ Bool

Returns ‘true` If the system is scanning, `false` if #run hasn’t been called yet or if the scan has finished.

Returns:

  • (Bool) —

    ‘true` If the system is scanning, `false` if #run hasn’t been called yet or if the scan has finished.



120
121
122
123
124
125
126
127
128
129
 
# File 'lib/arachni/rpc/server/framework.rb', line 120

def busy?( &block )
    # If we have a block it means that it was called via RPC, so use the
    # status variable to determine if the scan is done.
    if block_given?
        block.call @prepared && @status != :done
        return
    end

    !!@extended_running
end

#clean_up(&block) ⇒ Object

If the scan needs to be aborted abruptly this method takes care of any unfinished business (like signaling running plug-ins to finish).

Should be called before grabbing the #auditstore, especially when running in HPG mode as it will take care of merging the plug-in results of all instances.

You don’t need to call this if you’ve let the scan complete.



173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
 
# File 'lib/arachni/rpc/server/framework.rb', line 173

def clean_up( &block )
    if @cleaned_up
        block.call false if block_given?
        return false
    end

    @cleaned_up       = true
    @extended_running = false
    r = super

    if !block_given?
        @status = :done
        return r
    end

    if !has_slaves?
        @status = :done
        block.call r
        return
    end

    foreach = proc do |instance, iter|
        instance.framework.clean_up do
            instance.plugins.results do |res|
                iter.return( !res.rpc_exception? ? res : nil )
            end
        end
    end
    after = proc do |results|
        @plugins.merge_results( results.compact )
        @status = :done
        block.call true
    end
    map_slaves( foreach, after )
end

#error_test(str, &block) ⇒ Object 



337
338
339
340
341
342
343
 
# File 'lib/arachni/rpc/server/framework.rb', line 337

def error_test( str, &block )
    print_error str.to_s
    return block.call if !has_slaves?

    each = proc { |instance, iter| instance.framework.error_test( str ) { iter.next } }
    each_slave( each, &block )
end

#issues ⇒ Array<Arachni::Issue>

Returns First variations of all discovered issues.

Returns:



312
313
314
 
# File 'lib/arachni/rpc/server/framework.rb', line 312

def issues
    auditstore.issues.map { |issue| issue.variations.first || issue }
end

#issues_as_hash ⇒ Array<Hash>

Returns #issues as an array of Hashes.

Returns:

See Also:



318
319
320
 
# File 'lib/arachni/rpc/server/framework.rb', line 318

def issues_as_hash
    issues.map( &:to_h )
end

#list_plugins ⇒ Array<Hash> Also known as: lsplug

Returns Information about all available plugins.

Returns:

  • (Array<Hash>) —

    Information about all available plugins.



96
97
98
99
100
101
102
103
 
# File 'lib/arachni/rpc/server/framework.rb', line 96

def list_plugins
    super.map do |plugin|
        plugin[:options] = [plugin[:options]].flatten.compact.map do |opt|
            opt.to_h.merge( 'type' => opt.type )
        end
        plugin
    end
end

#list_reports ⇒ Array<Hash> Also known as: lsrep

Returns Information about all available reports.

Returns:

  • (Array<Hash>) —

    Information about all available reports.



107
108
109
110
111
112
113
114
 
# File 'lib/arachni/rpc/server/framework.rb', line 107

def list_reports
    super.map do |report|
        report[:options] = [report[:options]].flatten.compact.map do |opt|
            opt.to_h.merge( 'type' => opt.type )
        end
        report
    end
end

#output(&block) ⇒ Array<Hash>

Deprecated.

Merged output of all running instances.

This is going to be wildly out of sync and lack A LOT of messages.

It’s here to give the notion of progress to the end-user rather than provide an accurate depiction of the actual progress.

The returned object will be in the form of:

[ { <type> => <message> } ]

like:

[
    { status: 'Initiating'},
    {   info: 'Some informational msg...'},
]

Possible message types are:

  • ‘status` – Status messages, usually to denote progress.

  • ‘info` – Informational messages, like notices.

  • ‘ok` – Denotes a successful operation or a positive result.

  • ‘verbose` – Verbose messages, extra information about whatever.

  • ‘bad` – Opposite of :ok, an operation didn’t go as expected, something has failed but it’s recoverable.

  • ‘error` – An error has occurred, this is not good.

  • ‘line` – Generic message, no type.

Returns:



270
271
272
273
274
275
276
277
278
279
280
281
282
283
 
# File 'lib/arachni/rpc/server/framework.rb', line 270

def output( &block )
    buffer = flush_buffer

    if !has_slaves?
        block.call( buffer )
        return
    end

    foreach = proc do |instance, iter|
        instance.service.output { |out| iter.return( out ) }
    end
    after = proc { |out| block.call( (buffer | out).flatten ) }
    map_slaves( foreach, after )
end

#pause(&block) ⇒ Object

Pauses the running scan on a best effort basis.



210
211
212
213
214
215
216
217
218
219
220
221
 
# File 'lib/arachni/rpc/server/framework.rb', line 210

def pause( &block )
    r = super
    return r if !block_given?

    if !has_slaves?
        block.call true
        return
    end

    each = proc { |instance, iter| instance.framework.pause { iter.next } }
    each_slave( each, proc { block.call true } )
end

#report ⇒ Hash Also known as: audit_store_as_hash, auditstore_as_hash

Returns Audit results as a hash.

Returns:

See Also:



294
295
296
 
# File 'lib/arachni/rpc/server/framework.rb', line 294

def report
    audit_store.to_h
end

#resume(&block) ⇒ Object

Resumes a paused scan right away.



224
225
226
227
228
229
230
231
232
233
234
235
 
# File 'lib/arachni/rpc/server/framework.rb', line 224

def resume( &block )
    r = super
    return r if !block_given?

    if !has_slaves?
        block.call true
        return
    end

    each = proc { |instance, iter| instance.framework.resume { iter.next } }
    each_slave( each, proc { block.call true } )
end

#run ⇒ Bool

Starts the scan.

Returns:

  • (Bool) —

    ‘false` if already running, `true` otherwise.



136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
 
# File 'lib/arachni/rpc/server/framework.rb', line 136

def run
    # Return if we're already running.
    return false if busy?

    if master? && @opts.restrict_paths.any?
        fail Error::UnsupportedOption,
             'Option \'restrict_paths\' is not supported when in multi-Instance mode.'
    end

    @extended_running = true

    # Prepare the local instance (runs plugins and starts the timer).
    prepare

    # Start the scan  -- we can't block the RPC server so we're using a Thread.
    Thread.abort_on_exception = true
    Thread.new do
        if !solo?
            multi_run
        else
            super
        end
    end

    true
end

#self_url ⇒ String

Returns URL of this instance.

Returns:

  • (String) —

    URL of this instance.



324
325
326
327
328
329
 
# File 'lib/arachni/rpc/server/framework.rb', line 324

def self_url
    @opts.rpc_external_address ||= @opts.rpc_address

    @self_url ||= @opts.rpc_external_address ?
        "#{@opts.rpc_external_address }:#{@opts.rpc_port}" : @opts.rpc_socket
end

#serialized_auditstore ⇒ String

Returns YAML representation of #auditstore.

Returns:



306
307
308
 
# File 'lib/arachni/rpc/server/framework.rb', line 306

def serialized_auditstore
    audit_store.to_yaml
end

#serialized_report ⇒ String

Returns YAML representation of #report.

Returns:



301
302
303
 
# File 'lib/arachni/rpc/server/framework.rb', line 301

def serialized_report
    report.to_yaml
end

#stats(*args) ⇒ Object

See Also:



286
287
288
289
290
 
# File 'lib/arachni/rpc/server/framework.rb', line 286

def stats( *args )
    ss = super( *args )
    ss.tap { |s| s[:sitemap_size] = spider.local_sitemap.size } if !solo?
    ss
end

#token ⇒ String

Returns This instance’s RPC token.

Returns:

  • (String) —

    This instance’s RPC token.



332
333
334
 
# File 'lib/arachni/rpc/server/framework.rb', line 332

def token
    @opts.datastore[:token]
end