Class: Obuf

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/obuf.rb

Overview

An object buffer for Ruby objects. Use it to sequentially store a shitload of objects on disk and then retreive them one by one. Make sure to call clear when done with it to discard the stored blob. 

a = Obuf.new
parse_big_file do | one_node |
  a.push(one_node)
end

a.size #=> 30932 # We've stored 30 thousand objects on disk without breaking a sweat
a.each do | node_read_from_disk |
   # do something with node that has been recovered from disk
end

a.clear # ensure that the file is deleted

Both reading and writing aim to be threadsafe

Defined Under Namespace

Classes: Lens, ProtectedLens

Constant Summary collapse

VERSION = 
"1.2.1"

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(enumerable = []) {|_self| ... } ⇒ Obuf

Initializes a new Obuf. If an Enumerable argument is passed each element from the Enumerable will be stored in the Obuf (so you can pass an IO for example)

Yields:

  • (_self)

Yield Parameters:

  • _self (Obuf) —

    the object that the method was called on



33
34
35
36
37
38
39
40
41
42
43
44
45
46
 
# File 'lib/obuf.rb', line 33

def initialize(enumerable = [])
  @sem = Mutex.new
  @store = Tempfile.new("obuf")
  @store.binmode
  @size = 0
  
  @lens = Obuf::ProtectedLens.new(@store)
  
  # Store everything from the enumerable in self
  enumerable.each { |e| push(e) }
  
  # ...and yield self for any configuration
  yield self if block_given?
end

Instance Attribute Details

#size ⇒ Object (readonly)

Returns the number of objects stored so far



29
30
31
 
# File 'lib/obuf.rb', line 29

def size
  @size
end

Instance Method Details

#[](slice) ⇒ Object

Retreive a slice of the enumerable at index



82
83
84
 
# File 'lib/obuf.rb', line 82

def [](slice)
  slice.respond_to?(:each) ? slice.map{|i| recover_at(i) } : recover_at(slice)
end

#clear ⇒ Object

Calls close! on the datastore and deletes the objects in it



74
75
76
77
78
79
 
# File 'lib/obuf.rb', line 74

def clear
  @sem.synchronize do
    @store.close!
    @size = 0
  end
end

#each ⇒ Object

Retreive each stored object in succession. All other Enumerable methods are also available (but be careful with Enumerable#map and to_a)



66
67
68
69
70
71
 
# File 'lib/obuf.rb', line 66

def each
  with_separate_read_io do | iterable |
    reading_lens = Obuf::Lens.new(iterable)
    @size.times { yield(reading_lens.recover_object) }
  end
end

#empty? ⇒ Boolean

Tells whether the buffer is empty

Returns:

  • (Boolean) 


49
50
51
 
# File 'lib/obuf.rb', line 49

def empty?
  @size.zero?
end

#push(object_to_store) ⇒ Object Also known as: <<

Store an object



54
55
56
57
58
59
60
 
# File 'lib/obuf.rb', line 54

def push(object_to_store)
  @sem.synchronize {
    @lens << object_to_store
    @size += 1
  }
  object_to_store
end