Class: NMatrix::IO::Matlab::Mat5Reader::MatrixData

Inherits:
MatrixDataStruct show all
Includes:
Packable
Defined in:
lib/nmatrix/io/mat5_reader.rb

Overview

:nodoc:

Instance Attribute Summary

Attributes inherited from MatrixDataStruct

#cells, #column_index, #complex, #dimensions, #global, #imaginary_part, #logical, #matlab_class, #matlab_name, #nonzero_max, #real_part, #row_index

Instance Method Summary collapse

Instance Method Details

#guess_dtype_from_mdtype ⇒ Object

call-seq:

guess_dtype_from_mdtype -> Symbol

Try to determine what dtype and such to use.

TODO: Needs to be verified that unsigned MATLAB types are being converted to the correct NMatrix signed dtypes.



134
135
136
137
138
139
140
 
# File 'lib/nmatrix/io/mat5_reader.rb', line 134

def guess_dtype_from_mdtype
  dtype = MatReader::MDTYPE_TO_DTYPE[self.real_part.tag.data_type]

  return dtype unless self.complex

  dtype == :float32 ? :complex64 : :complex128
end

#ignore_padding(packedio, bytes) ⇒ Object 



333
334
335
 
# File 'lib/nmatrix/io/mat5_reader.rb', line 333

def ignore_padding(packedio, bytes)
  packedio.read([Integer, {:unsigned => true, :bytes => bytes}]) if bytes > 0
end

#read_packed(packedio, options) ⇒ Object 



287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
 
# File 'lib/nmatrix/io/mat5_reader.rb', line 287

def read_packed(packedio, options)
  flags_class, self.nonzero_max = packedio.read([Element, options]).data

  self.matlab_class   = MatReader::MCLASSES[flags_class % 16]

  self.logical        = (flags_class >> 8) % 2 == 1 ? true : false
  self.global         = (flags_class >> 9) % 2 == 1 ? true : false
  self.complex        = (flags_class >> 10) % 2 == 1 ? true : false

  dimensions_tag_data = packedio.read([Element, options])
  self.dimensions     = dimensions_tag_data.data

  begin
    name_tag_data   = packedio.read([Element, options])
    self.matlab_name = name_tag_data.data.is_a?(Array) ? name_tag_data.data.collect { |i| i.chr }.join('') : name_tag_data.data.chr

  rescue ElementDataIOError => e
    STDERR.puts "ERROR: Failure while trying to read Matlab variable name: #{name_tag_data.inspect}"
    STDERR.puts 'Element Tag:'
    STDERR.puts "    #{e.tag}"
    STDERR.puts 'Previously, I read these dimensions:'
    STDERR.puts "    #{dimensions_tag_data.inspect}"
    STDERR.puts "Unpack options were: #{options.inspect}"
    raise(e)
  end

  if self.matlab_class == :mxCELL
    # Read what may be a series of matrices
    self.cells = []
    STDERR.puts("Warning: Cell array does not yet support reading multiple dimensions") if dimensions.size > 2 || (dimensions[0] > 1 && dimensions[1] > 1)
    number_of_cells = dimensions.inject(1) { |prod,i| prod * i }
    number_of_cells.times { self.cells << packedio.read([Element, options]) }

  else
    read_opts = [RawElement, {:bytes => options[:bytes], :endian => :native}]

    if self.matlab_class == :mxSPARSE
      self.column_index = packedio.read(read_opts)
      self.row_index    = packedio.read(read_opts)
    end

    self.real_part   = packedio.read(read_opts)
    self.imaginary_part = packedio.read(read_opts) if self.complex
  end
end

#repacked_data(to_dtype = nil) ⇒ Object

Unpacks and repacks data into the appropriate format for NMatrix.

If data is already in the appropriate format, does not unpack or repack, just returns directly.

Complex is always unpacked and repacked, as the real and imaginary components must be merged together (MATLAB stores them separately for some crazy reason).

Used only for Yale storage creation. For dense, see unpacked_data.

This function calls repack and complex_merge, which are both defined in io.cpp.



185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
 
# File 'lib/nmatrix/io/mat5_reader.rb', line 185

def repacked_data(to_dtype = nil)

  real_mdtype = self.real_part.tag.data_type

  # Figure out what dtype to use based on the MATLAB data-types
  # (mdtypes). They could be different for real and imaginary, so call
  # upcast to figure out what to use.

  components = [] # real and imaginary parts or just the real part

  if self.complex
    imag_mdtype = self.imaginary_part.tag.data_type

    # Make sure we convert both mdtypes do the same dtype
    to_dtype ||= NMatrix.upcast(MatReader::MDTYPE_TO_DTYPE[real_mdtype], MatReader::MDTYPE_TO_DTYPE[imag_mdtype])

    # Let's make sure we don't try to send NMatrix complex integers. We need complex floating points.
    unless [:float32, :float64].include?(to_dtype)
      to_dtype = NMatrix.upcast(to_dtype, :float32)
    end

    STDERR.puts "imag: Requesting dtype #{to_dtype.inspect}"
    # Repack the imaginary part
    components[1] = ::NMatrix::IO::Matlab.repack( self.imaginary_part.data, imag_mdtype, :dtype => to_dtype )

  else

    to_dtype ||= MatReader::MDTYPE_TO_DTYPE[real_mdtype]

    # Sometimes repacking isn't necessary -- sometimes the format is already good
    if MatReader::NO_REPACK.include?(real_mdtype)
      STDERR.puts "No repack"
      return [self.real_part.data, to_dtype]
    end

  end

  # Repack the real part
  STDERR.puts "real: Requesting dtype #{to_dtype.inspect}"
  components[0] = ::NMatrix::IO::Matlab.repack( self.real_part.data, real_mdtype, :dtype => to_dtype )

  # Merge the two parts if complex, or just return the real part.
  [self.complex ? ::NMatrix::IO::Matlab.complex_merge( components[0], components[1], to_dtype ) : components[0],
   to_dtype]
end

#repacked_indices ⇒ Object

Unpacks and repacks index data into the appropriate format for NMatrix.

If data is already in the appropriate format, does not unpack or repack, just returns directly.



235
236
237
238
239
240
 
# File 'lib/nmatrix/io/mat5_reader.rb', line 235

def repacked_indices
  repacked_row_indices = ::NMatrix::IO::Matlab.repack( self.row_index.data, :miINT32, :itype )
  repacked_col_indices = ::NMatrix::IO::Matlab.repack( self.column_index.data, :miINT32, :itype )

  [repacked_row_indices, repacked_col_indices]
end

#to_nm(dtype = nil) ⇒ Object

call-seq:

to_nm(dtype = nil) -> NMatrix

Create an NMatrix from a MATLAB .mat (v5) matrix.

This function matches the storage type exactly. That is, a regular matrix in MATLAB will be a dense NMatrix, and a sparse (old Yale) one in MATLAB will be a :yale (new Yale) matrix in NMatrix.

Note that NMatrix has no old Yale type, so this uses a semi-hidden version of the NMatrix constructor to pass in — as directly as possible – the stored bytes in a MATLAB sparse matrix. This constructor should also be used for other IO formats that want to create sparse matrices from IA and JA vectors (e.g., SciPy).

This is probably not the fastest code. An ideal solution would be a C plugin of some sort for reading the MATLAB .mat file. However, .mat v5 is a really complicated format, and lends itself to an object-oriented solution.



263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
 
# File 'lib/nmatrix/io/mat5_reader.rb', line 263

def to_nm(dtype = nil)
  # Hardest part is figuring out from_dtype, from_index_dtype, and dtype.
  dtype   ||= guess_dtype_from_mdtype
  from_dtype = MatReader::MDTYPE_TO_DTYPE[self.real_part.tag.data_type]

  # Create the same kind of matrix that MATLAB saved.
  case matlab_class
  when :mxSPARSE
    raise(NotImplementedError, "expected .mat row indices to be of type :miINT32") unless row_index.tag.data_type == :miINT32
    raise(NotImplementedError, "expected .mat column indices to be of type :miINT32") unless column_index.tag.data_type == :miINT32
    #require 'pry'
    #binding.pry

    # MATLAB always uses :miINT32 for indices according to the spec
    ia_ja                     = repacked_indices
    data_str, repacked_dtype  = repacked_data(dtype)
    NMatrix.new(:yale, self.dimensions.reverse, repacked_dtype, ia_ja[0], ia_ja[1], data_str, repacked_dtype)

  else
    # Call regular dense constructor.
    NMatrix.new(:dense, self.dimensions.reverse, unpacked_data, dtype).transpose
  end
end

#to_ruby ⇒ Object

call-seq:

to_ruby -> NMatrix
to_ruby -> Array

Figure out the appropriate Ruby type to convert to, and do it. There are basically two possible types: NMatrix and Array. This method is recursive, so an Array is going to contain other Arrays and/or NMatrix objects.

mxCELL types (cells) will be converted to the Array type.

mxSPARSE and other types will be converted to NMatrix, with the appropriate stype (:yale or :dense, respectively).

See also to_nm, which is responsible for NMatrix instantiation.



119
120
121
122
123
124
125
 
# File 'lib/nmatrix/io/mat5_reader.rb', line 119

def to_ruby
  case matlab_class
  when :mxSPARSE then return to_nm
  when :mxCELL  then return self.cells.collect { |c| c.to_ruby }
  else         return to_nm
  end
end

#unpacked_data(real_mdtype = nil, imag_mdtype = nil) ⇒ Object

call-seq:

unpacked_data(real_mdtype = nil, imag_mdtype = nil) ->

Unpacks data without repacking it.

Used only for dense matrix creation. Yale matrix creation uses repacked_data.



151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
 
# File 'lib/nmatrix/io/mat5_reader.rb', line 151

def unpacked_data(real_mdtype = nil, imag_mdtype = nil)
  # Get Matlab data type and unpack args
  real_mdtype ||= self.real_part.tag.data_type
  real_unpack_args = MatReader::MDTYPE_UNPACK_ARGS[real_mdtype]

  # zip real and complex components together, or just return real component
  if self.complex
    imag_mdtype ||= self.imaginary_part.tag.data_type
    imag_unpack_args = MatReader::MDTYPE_UNPACK_ARGS[imag_mdtype]

    unpacked_real = self.real_part.data.unpack(real_unpack_args)
    unpacked_imag = self.imaginary_part.data.unpack(imag_unpack_args)

    unpacked_real.zip(unpacked_imag).flatten
  else
    length = self.dimensions.inject(1) { |a,b| a * b } # get the product
    self.real_part.data.unpack(*(real_unpack_args*length))
  end

end

#write_packed(packedio, options) ⇒ Object

Raises:

  • (NotImplementedError) 


99
100
101
102
 
# File 'lib/nmatrix/io/mat5_reader.rb', line 99

def write_packed(packedio, options)
  raise NotImplementedError
  packedio << [info, {:bytes => padded_bytes}.merge(options)]
end