Class: Obuf

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/obuf.rb

Overview

An object buffer for Ruby objects. Use it to sequentially store a shitload of objects on disk and then retreive them one by one. Make sure to call clear when done with it to discard the stored blob. 

a = Obuf.new
parse_big_file do | one_node |
  a.push(one_node)
end

a.size #=> 30932 # We've stored 30 thousand objects on disk without breaking a sweat
a.each do | node_read_from_disk |
   # do something with node that has been recovered from disk
end

a.clear # ensure that the file is deleted

Both reading and writing aim to be threadsafe

Constant Summary collapse

VERSION = 
"1.0.4"
DELIM = 
"\t"
END_RECORD = 
"\n"

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Obuf

Returns a new instance of Obuf.



32
33
34
35
36
37
38
39
40
 
# File 'lib/obuf.rb', line 32

def initialize
  @sem = Mutex.new
  @store = Tempfile.new("obuf")
  @store.set_encoding(Encoding::BINARY) if @store.respond_to?(:set_encoding)
  @store.binmode
  
  @size = 0
  super
end

Instance Attribute Details

#size ⇒ Object (readonly)

Returns the number of objects stored so far



30
31
32
 
# File 'lib/obuf.rb', line 30

def size
  @size
end

Instance Method Details

#[](slice) ⇒ Object

Retreive a slice of the enumerable at index



79
80
81
 
# File 'lib/obuf.rb', line 79

def [](slice)
  slice.respond_to?(:each) ? slice.map{|i| recover_at(i) } : recover_at(slice)
end

#clear ⇒ Object

Calls close! on the datastore and deletes the objects in it



71
72
73
74
75
76
 
# File 'lib/obuf.rb', line 71

def clear
  @sem.synchronize do
    @store.close!
    @size = 0
  end
end

#each ⇒ Object

Retreive each stored object in succession. All other Enumerable methods are also available (but be careful with Enumerable#map and to_a)



64
65
66
67
68
 
# File 'lib/obuf.rb', line 64

def each
  with_separate_read_io do | iterable |
    @size.times { yield(recover_object_from(iterable)) }
  end
end

#empty? ⇒ Boolean

Tells whether the buffer is empty

Returns:

  • (Boolean) 


43
44
45
 
# File 'lib/obuf.rb', line 43

def empty?
  @size.zero?
end

#push(object_to_store) ⇒ Object Also known as: <<

Store an object



48
49
50
51
52
53
54
55
56
57
58
 
# File 'lib/obuf.rb', line 48

def push(object_to_store)
  blob = marshal_object(object_to_store)
  @sem.synchronize do
    @store.write(blob.size)
    @store.write(DELIM)
    @store.write(blob)
    @store.write(END_RECORD)
    @size += 1
  end
  object_to_store
end