Class: XZ::StreamReader

Inherits:
Stream
  • Object
show all
Defined in:
lib/xz/stream_reader.rb

Overview

An IO-like reader class for XZ-compressed data, allowing you to access XZ-compressed data as if it was a normal IO object, but please note you can’t seek in the data–this doesn’t make much sense anyway. Where would you want to seek? The plain or the XZ data?

A StreamReader object actually wraps another IO object it reads the compressed data from; you can either pass this IO object directly to the ::new method, effectively allowing you to pass any IO-like thing you can imagine (just ensure it is readable), or you can pass a path to a filename to ::new, in which case StreamReader takes care of both opening and closing the file correctly. You can even take it one step further and use the block form of ::new which will automatically call the #close method for you after the block finished. However, if you pass an IO, remember you have to close:

  1. The StreamReader instance.

  2. The IO object you passed to ::new.

Do it in exactly that order, otherwise you may lose data.

See the io-like gem’s documentation for the IO-reading methods available for this class (although you’re probably familiar with them through Ruby’s own IO class ;-)).

Example

In this example, we’re going to use ruby-xz together with the archive-tar-minitar gem that allows to read tarballs. Used together, the two libraries allow us to read XZ-compressed tarballs.

require "xz"
require "archive/tar/minitar"

XZ::StreamReader.open("foo.tar.xz") do |txz|
  # This automatically closes txz
  Archive::Tar::Minitar.unpack(txz, "foo")
end

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(delegate, memory_limit = XZ::LibLZMA::UINT64_MAX, flags = [:tell_unsupported_check]) ⇒ StreamReader

call-seq:

new(delegate, memory_limit = XZ::LibLZMA::UINT64_MAX, flags = [:tell_unsupported_check])  → a_stream_reader
open(delegate, memory_limit = XZ::LibLZMA::UINT64_MAX, flags = [:tell_unsupported_check]) → a_stream_reader

Creates a new StreamReader instance. If you pass an IO, remember you have to close both the resulting instance (via the #close method) and the IO object you pass to flush any internal buffers in order to be able to read all decompressed data.

Parameters

delegate

An IO object to read the data from, or a path

to a file to open. If you’re in an urgent need to
pass a plain string, use StringIO from Ruby’s
standard library. If this is an IO, it must be
opened for reading.

The other parameters are identical to what the XZ::decompress_stream method expects.

Return value

The newly created instance.

Example

# Wrap it around a file
f = File.open("foo.xz")
r = XZ::StreamReader.new(f)

# Ignore any XZ checksums (may result in invalid data being read!)
File.open("foo.xz") do |f|
  r = XZ::StreamReader.new(f, XZ::LibLZMA::UINT64_MAX, [:tell_no_check]
end

# Let StreamReader handle file closing automatically
XZ::StreamReader.new("myfile.xz"){|r| r.raed}

Raises:

  • (ArgumentError) 


101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
 
# File 'lib/xz/stream_reader.rb', line 101

def initialize(delegate, memory_limit = XZ::LibLZMA::UINT64_MAX, flags = [:tell_unsupported_check])
  raise(ArgumentError, "Invalid memory limit set!") unless (0..XZ::LibLZMA::UINT64_MAX).include?(memory_limit)
  flags.each do |flag|
    raise(ArgumentError, "Unknown flag #{flag}!") unless [:tell_no_check, :tell_unsupported_check, :tell_any_check, :concatenated].include?(flag)
  end

  if delegate.respond_to?(:to_io)
    super(delegate)
  else
    @file = File.open(delegate, "rb")
    super(@file)
  end

  @memory_limit = memory_limit
  @flags        = flags

  res = XZ::LibLZMA.lzma_stream_decoder(@lzma_stream,
                                        @memory_limit,
                                        @flags.inject(0){|val, flag| val | XZ::LibLZMA.const_get(:"LZMA_#{flag.to_s.upcase}")})
  XZ::LZMAError.raise_if_necessary(res)

  @input_buffer_p = FFI::MemoryPointer.new(XZ::CHUNK_SIZE)

  # These two are only used in #unbuffered read.
  @__lzma_finished = false
  @__lzma_action   = nil

  if block_given?
    begin
      yield(self)
    ensure
      close unless closed?
    end
  end
end

Instance Attribute Details

#flags ⇒ Object (readonly)

The flags you set for this reader (in ::new).



68
69
70
 
# File 'lib/xz/stream_reader.rb', line 68

def flags
  @flags
end

#memory_limit ⇒ Object (readonly)

The memory limit you set for this reader (in ::new).



66
67
68
 
# File 'lib/xz/stream_reader.rb', line 66

def memory_limit
  @memory_limit
end

Instance Method Details

#close ⇒ Object

Closes this StreamReader instance. Don’t use it afterwards anymore.

Return value

The total number of bytes decompressed.

Example

r.close #=> 6468

Remarks

If you passed an IO to ::new, this method doesn’t close it, so you have to close it yourself.



147
148
149
150
151
152
153
154
155
156
157
158
159
 
# File 'lib/xz/stream_reader.rb', line 147

def close
  super

  # Close the XZ stream
  res = XZ::LibLZMA.lzma_end(@lzma_stream.pointer)
  XZ::LZMAError.raise_if_necessary(res)

  #If we created a File object, close this as well.
  @file.close if @file

  # Return the number of bytes written in total.
  @lzma_stream[:total_out]
end

#pos ⇒ Object Also known as: tell

call-seq:

pos()  → an_integer
tell() → an_integer

Total number of output bytes provided to you yet.



166
167
168
 
# File 'lib/xz/stream_reader.rb', line 166

def pos
  @lzma_stream[:total_out]
end

#rewind ⇒ Object

Instrcuts liblzma to immediately stop decompression, rewinds the wrapped IO object and reinitalizes the StreamReader instance with the same values passed originally to the ::new method. The wrapped IO object must support the rewind method for this method to work; if it doesn’t, this method throws an IOError. After the exception was thrown, the StreamReader instance is in an unusable state. You cannot continue using it (don’t call #close on it either); close the wrapped IO stream and create another instance of this class.

Raises

IOError

The wrapped IO doesn’t support rewinding.

Do not use the StreamReader instance anymore
after receiving this exception.

Remarks

I don’t really like this method, it uses several dirty tricks to circumvent both io-like’s and liblzma’s control mechanisms. I only implemented this because the archive-tar-minitar gem calls this method when unpacking a TAR archive from a stream.



191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
 
# File 'lib/xz/stream_reader.rb', line 191

def rewind
  # HACK: Wipe all data from io-like’s internal read buffer.
  # This heavily relies on io-like’s internal structure.
  # Be always sure to test this when a new version of
  # io-like is released!
  __io_like__internal_read_buffer.clear

  # Forcibly close the XZ stream (internally frees it!)
  res = XZ::LibLZMA.lzma_end(@lzma_stream.pointer)
  XZ::LZMAError.raise_if_necessary(res)

  # Rewind the wrapped IO
  begin
    @delegate_io.rewind
  rescue => e
    raise(IOError, "Delegate IO failed to rewind! Original message: #{e.message}")
  end

  # Reinitialize everything. Note this doesn’t affect @file as it
  # is already set and stays so (we don’t pass a filename here,
  # but rather an IO)
  initialize(@delegate_io, @memory_limit, @flags)
end