Class: RangesIO

Inherits:
Object
  • Object
show all
Defined in:
lib/ole/ranges_io.rb

Overview

Introduction

RangesIO is a basic class for wrapping another IO object allowing you to arbitrarily reorder slices of the input file by providing a list of ranges. Intended as an initial measure to curb inefficiencies in the Dirent#data method just reading all of a file’s data in one hit, with no method to stream it.

This class will encapuslate the ranges (corresponding to big or small blocks) of any ole file and thus allow reading/writing directly to the source bytes, in a streamed fashion (so just getting 16 bytes doesn’t read the whole thing).

In the simplest case it can be used with a single range to provide a limited io to a section of a file.

Limitations

  • No buffering. by design at the moment. Intended for large reads

TODO

On further reflection, this class is something of a joining/optimization of two separate IO classes. a SubfileIO, for providing access to a range within a File as a separate IO object, and a ConcatIO, allowing the presentation of a bunch of io objects as a single unified whole.

I will need such a ConcatIO if I’m to provide Mime#to_io, a method that will convert a whole mime message into an IO stream, that can be read from. It will just be the concatenation of a series of IO objects, corresponding to headers and boundaries, as StringIO’s, and SubfileIO objects, coming from the original message proper, or RangesIO as provided by the Attachment#data, that will then get wrapped by Mime in a Base64IO or similar, to get encoded on-the- fly. Thus the attachment, in its plain or encoded form, and the message as a whole never exists as a single string in memory, as it does now. This is a fair bit of work to achieve, but generally useful I believe.

This class isn’t ole specific, maybe move it to my general ruby stream project.

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(io, mode = 'r', params = {}) ⇒ RangesIO

io

the parent io object that we are wrapping.

mode

the mode to use

params

hash of params.

  • :ranges - byte offsets, either:

    1. an array of ranges [1..2, 4..5, 6..8] or

    2. an array of arrays, where the second is length [[1, 1], [4, 1], [6, 2]] for the above (think the way String indexing works)

  • :close_parent - boolean to close parent when this object is closed

NOTE: the ranges can overlap.



56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/ole/ranges_io.rb', line 56

def initialize io, mode='r', params={}
	mode, params = 'r', mode if Hash === mode
	ranges = params[:ranges]
	@params = {:close_parent => false}.merge params
	@mode = Ole::IOMode.new mode
	@io = io
	# initial position in the file
	@pos = 0
	self.ranges = ranges || [[0, io.size]]
	# handle some mode flags
	truncate 0 if @mode.truncate?
	seek size if @mode.append?
end

Instance Attribute Details

#io ⇒ Object (readonly)

Returns the value of attribute io.



45
46
47
 
# File 'lib/ole/ranges_io.rb', line 45

def io
  @io
end

#mode ⇒ Object (readonly)

Returns the value of attribute mode.



45
46
47
 
# File 'lib/ole/ranges_io.rb', line 45

def mode
  @mode
end

#pos ⇒ Object Also known as: tell

Returns the value of attribute pos.



45
46
47
 
# File 'lib/ole/ranges_io.rb', line 45

def pos
  @pos
end

#ranges ⇒ Object

Returns the value of attribute ranges.



45
46
47
 
# File 'lib/ole/ranges_io.rb', line 45

def ranges
  @ranges
end

#size ⇒ Object

Returns the value of attribute size.



45
46
47
 
# File 'lib/ole/ranges_io.rb', line 45

def size
  @size
end

Class Method Details

.open(*args, &block) ⇒ Object

add block form. TODO add test for this



71
72
73
74
75
76
77
78
79
80
 
# File 'lib/ole/ranges_io.rb', line 71

def self.open(*args, &block)
	ranges_io = new(*args)
	if block_given?
		begin;  yield ranges_io
		ensure; ranges_io.close
		end
	else
		ranges_io
	end
end

Instance Method Details

#close ⇒ Object 



149
150
151
 
# File 'lib/ole/ranges_io.rb', line 149

def close
	@io.close if @params[:close_parent]
end

#eof? ⇒ Boolean

Returns:

  • (Boolean) 


153
154
155
 
# File 'lib/ole/ranges_io.rb', line 153

def eof?
	@pos == @size
end

#gets ⇒ Object Also known as: readline

i can wrap it in a buffered io stream that provides gets, and appropriately handle pos, truncate. mostly added just to past the tests. FIXME



242
243
244
245
246
247
 
# File 'lib/ole/ranges_io.rb', line 242

def gets
	s = read 1024
	i = s.index "\n"
	self.pos -= s.length - (i+1)
	s[0..i]
end

#inspect ⇒ Object 



250
251
252
 
# File 'lib/ole/ranges_io.rb', line 250

def inspect
	"#<#{self.class} io=#{io.inspect}, size=#{@size}, pos=#{@pos}>"
end

#read(limit = nil) ⇒ Object

read bytes from file, to a maximum of limit, or all available if unspecified.



158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
 
# File 'lib/ole/ranges_io.rb', line 158

def read limit=nil
	data = ''.dup
	return data if eof?
	limit ||= size
	pos, len = @ranges[@active]
	diff = @pos - @offsets[@active]
	pos += diff
	len -= diff
	loop do
		@io.seek pos
		if limit < len
			s = @io.read(limit).to_s
			@pos += s.length
			data << s
			break
		end
		s = @io.read(len).to_s
		@pos += s.length
		data << s
		break if s.length != len
		limit -= len
		break if @active == @ranges.length - 1
		@active += 1
		pos, len = @ranges[@active]
	end
	data
end

#rewind ⇒ Object 



145
146
147
 
# File 'lib/ole/ranges_io.rb', line 145

def rewind
	seek 0
end

#truncate(size) ⇒ Object

you may override this call to update @ranges and @size, if applicable.

Raises:

  • (NotImplementedError) 


187
188
189
 
# File 'lib/ole/ranges_io.rb', line 187

def truncate size
	raise NotImplementedError, 'truncate not supported'
end

#write(data) ⇒ Object Also known as: << 



197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
 
# File 'lib/ole/ranges_io.rb', line 197

def write data
	# duplicates object to avoid side effects for the caller, but do so only if
	# encoding isn't already ASCII-8BIT (slight optimization)
	if data.respond_to?(:encoding) and data.encoding != Encoding::ASCII_8BIT
		data = data.dup.force_encoding(Encoding::ASCII_8BIT)
	end
	return 0 if data.empty?
	data_pos = 0
	# if we don't have room, we can use the truncate hook to make more space.
	if data.length > @size - @pos
		begin
			truncate @pos + data.length
		rescue NotImplementedError
			raise IOError, "unable to grow #{inspect} to write #{data.length} bytes" 
		end
	end
	pos, len = @ranges[@active]
	diff = @pos - @offsets[@active]
	pos += diff
	len -= diff
	loop do
		@io.seek pos
		if data_pos + len > data.length
			chunk = data[data_pos..-1]
			@io.write chunk
			@pos += chunk.length
			data_pos = data.length
			break
		end
		@io.write data[data_pos, len]
		@pos += len
		data_pos += len
		break if @active == @ranges.length - 1
		@active += 1
		pos, len = @ranges[@active]
	end
	data_pos
end