Class: Arachni::Support::Queue::Disk

Inherits:
Object
  • Object
show all
Defined in:
lib/arachni/support/queue/disk.rb

Overview

Note:

Not threadsafe.

Note:

Don’t forget to #clear the queue when you’re done with it in order to close the disk buffer.

Disk Queue implementation.

Keeps a max amount of entries in memory but stores the overflow to the disk. Once the in-memory entries run low the Queue fills itself up again with the disk entries.

Author:

Constant Summary collapse

DEFAULT_OPTIONS = 
{
    memory_buffer: 1_000
}

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ Disk

Returns a new instance of Disk.

Parameters:

  • options (Hash) (defaults to: {})

Options Hash (options):

  • :memory_buffer (Integer) —

    How many items to keep in memory.



45
46
47
48
49
50
51
52
 
# File 'lib/arachni/support/queue/disk.rb', line 45

def initialize( options = {} )
    @options     = DEFAULT_OPTIONS.merge( options )
    @memory_buffer = @options[:memory_buffer]

    @size     = 0
    @q        = []
    @buffer   = @options[:disk_buffer] || get_tempfile
end

Instance Method Details

#<<(object) ⇒ Disk Also known as: push

Returns self.

Parameters:

  • object (Object) —

    Object to push to the queue.

Returns:



56
57
58
59
60
61
62
63
64
65
 
# File 'lib/arachni/support/queue/disk.rb', line 56

def <<( object )
    if full?
        push_to_disk object
    else
        @q << object
    end

    @size += 1
    self
end

#clear ⇒ Object

Clears the queue, closes the disk buffer handle and removes its file.



105
106
107
108
109
110
111
112
 
# File 'lib/arachni/support/queue/disk.rb', line 105

def clear
    @q.clear
    @size = 0
    nil
ensure
    @buffer.close
    @buffer.unlink
end

#dup ⇒ Disk

Returns Copy of this queue.

Returns:

  • (Disk) —

    Copy of this queue.



90
91
92
93
94
95
96
97
 
# File 'lib/arachni/support/queue/disk.rb', line 90

def dup
    disk_buffer_copy = get_tempfile
    IO.copy_stream @buffer, disk_buffer_copy

    queue = self.class.new( disk_buffer: disk_buffer_copy )
    queue.copy( @q, size )
    queue
end

#pop ⇒ Object

Note:

Order is not guaranteed.

Returns Removes and returns an object from the queue.

Returns:

  • (Object) —

    Removes and returns an object from the queue.



71
72
73
74
75
76
77
 
# File 'lib/arachni/support/queue/disk.rb', line 71

def pop
    return if @q.empty?
    @size -= 1
    @q.shift
ensure
    fill_from_disk if reached_fill_threshold?
end

#size ⇒ Integer

Returns Size of the queue.

Returns:

  • (Integer) —

    Size of the queue.



100
101
102
 
# File 'lib/arachni/support/queue/disk.rb', line 100

def size
    @size
end

#to_a ⇒ Array

Returns Array with all contents.

Returns:

  • (Array) —

    Array with all contents.



80
81
82
83
84
85
86
87
 
# File 'lib/arachni/support/queue/disk.rb', line 80

def to_a
    queue = dup
    ary = []
    while item = queue.pop
        ary << item
    end
    ary
end