Class: CTypes::Union

Inherits:
Object
  • Object
show all
Extended by:
Type
Defined in:
lib/ctypes/union.rb

Overview

This class represents a C union in ruby. It provides methods for unpacking unions from their binary representation into a modifiable Ruby instance, and methods for repacking them into binary.

The ruby representation of a union does not get the same memory overlap benefits as experienced in C. As a result, the ruby Union must unpack the binary string each time a different union member is accessed. To support modification, this also means every time we switch between union members, any active union member must be packed into binary format, then unpacked at the new member. **This is a significant performance penalty when working with read-write Unions.**

To get arond the performance penalty, you can do one of the following:

  • do not swap between multiple union members unless absolutely necessary

  • #freeze the unpacked union instance to eliminate the repacking performance hit

  • figure out a memory-overlayed union implementation in ruby that doesn’t require unpack for every member access (it would be welcomed)

Examples:

# encoding: ASCII-8BIT
require_relative "./lib/ctypes"

# subclass Union to define a union
class Msg < CTypes::Union
  layout do
    # this message uses network-byte order
    endian :big

    # create enum for the message type used in members
    type = enum(uint8, {invalid: 0, hello: 1, read: 2})

    # define union members
    member :hello, struct({type:, version: string})
    member :read, struct({type:, offset: uint64, len: uint64})
    member :type, type
    member :raw, string
  end
end

# unpack a message and access member values
msg = Msg.unpack("\x02" +
                 "\xfe\xfe\xfe\xfe\xfe\xfe\xfe\xfe" +
                 "\xab\xab\xab\xab\xab\xab\xab\xab")
msg.type                      # => :read
msg.read.offset               # => 0xfefefefefefefefe
msg.read.len                  # => 0xabababababababab

# create new messages
Msg.pack({hello: {type: :hello, version: "v1.0"}})
                              # => "\1v1.0\0\0\0\0\0\0\0\0\0\0\0\0"
Msg.pack({read: {type: :read, offset: 0xffff, len: 0x1000}})
                              # => "\2\0\0\0\0\0\0\xFF\xFF\0\0\0\0\0\0\x10\0"

# work with a message instance to create a message
msg = Msg.new
msg.hello.type = :hello
msg.hello.version = "v1.0"
msg.to_binstr                 # => "\1v1.0\0\0\0\0\0\0\0\0\0\0\0\0"

Defined Under Namespace

Classes: Builder

Instance Attribute Summary

Attributes included from Type

#dry_type, #endian

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Type

default_endian, default_value, fixed_size?, greedy?, pack, pread, read, unpack, unpack_all, unpack_one, with_endian, without_endian

Constructor Details

#initialize(buf: "\0" * self.class.size, endian: self.class.default_endian) ⇒ Union

Returns a new instance of Union.

Parameters:

  • buf (String) (defaults to: "\0" * self.class.size) —

    binary String containing Union memory

  • endian (Symbol) (defaults to: self.class.default_endian) —

    byte-order of buf



426
427
428
429
430
431
432
 
# File 'lib/ctypes/union.rb', line 426

def initialize(
  buf: "\0" * self.class.size,
  endian: self.class.default_endian
)
  @buf = buf
  @endian = endian
end

Class Method Details

.==(other) ⇒ Object

check if another Union subclass has the same members as this Union



373
374
375
376
377
378
379
 
# File 'lib/ctypes/union.rb', line 373

def self.==(other)
  return true if super
  return false unless other.is_a?(Class) && other < Union
  other.field_layout == @fields &&
    other.default_endian == default_endian &&
    other.size == size
end

.builder ⇒ Object

get an instance of the Builder



99
100
101
 
# File 'lib/ctypes/union.rb', line 99

def self.builder
  Builder.new
end

.export_type(q) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future. 



400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
 
# File 'lib/ctypes/union.rb', line 400

def self.export_type(q)
  q << "CTypes::Union.builder()"
  q.break
  q.nest(2) do
    q << ".endian(%p)\n" % [@endian] if @endian
    @fields.each do |name, type|
      case name
      when Symbol
        q << ".member(%p, " % [name]
        q << type
        q << ")"
      when ::Array
        q << ".member("
        q << type
        q << ")"
      else
        raise Error, "unsupported field name type: %p" % [name]
      end
      q.break
    end
    q << ".build()"
  end
end

.field_layout ⇒ Object

get the list of members in this Union

Returns:

  • array of field layout



362
363
364
 
# File 'lib/ctypes/union.rb', line 362

def self.field_layout
  @fields
end

.fields ⇒ ::Array<Symbol>

get the list of members in this Union

Returns:

  • (::Array<Symbol>) —

    member names



355
356
357
 
# File 'lib/ctypes/union.rb', line 355

def self.fields
  @field_types.keys
end

.fixed_size? ⇒ Boolean

check if this is a fixed-size Union

Returns:

  • (Boolean) 


336
337
338
 
# File 'lib/ctypes/union.rb', line 336

def self.fixed_size?
  @fixed_size
end

.greedy? ⇒ Boolean

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns:

  • (Boolean) 


331
332
333
 
# File 'lib/ctypes/union.rb', line 331

def self.greedy?
  @greedy
end

.has_field?(member) ⇒ Boolean

check if the Union has a given member

Parameters:

  • member (Symbol) —

    member name

Returns:

  • (Boolean) 


368
369
370
 
# File 'lib/ctypes/union.rb', line 368

def self.has_field?(member)
  @field_types.has_key?(member)
end

.layout(&block) ⇒ Object

define the layout of this union

Examples:

class Msg < CTypes::Union
  layout do
    # this message uses network-byte order
    endian :big

    # create enum for the message type used in members
    type = enum(uint8, {invalid: 0, hello: 1, read: 2})

    # define union members
    member :hello, struct({type:, version: string})
    member :read, struct({type:, offset: uint64, len: uint64})
    member :type, type
    member :raw, string
  end
end

Raises:

See Also:



91
92
93
94
95
96
 
# File 'lib/ctypes/union.rb', line 91

def self.layout(&block)
  raise Error, "no block given" unless block
  builder = Builder.new(&block)
  builder.instance_eval(&block)
  apply_layout(builder)
end

.pack(value, endian: default_endian, validate: true, pad_bytes: nil) ⇒ ::String

Note:

do not provide multiple member values to this method; only zero or one member values are supported.

encode a ruby Hash into a String containing the binary representation of the Union

Examples:

include CTypes::Helpers
t = union(word: uint32, bytes: array(uint8, 4))
t.pack({word: 0xfeedface})        # => "\xCE\xFA\xED\xFE"
t.pack({word: 0xfeedface}, endian: :big)
                                  # => "\xFE\xED\xFA\xCE"

t.pack({bytes: [1, 2, 3, 4]})     # => "\x01\x02\x03\x04"
t.pack({bytes: [1, 2, 3, 4]}, endian: :big)
                                  # => "\x01\x02\x03\x04"

t.pack({bytes: [1, 2]})           # => "\x01\x02\x00\x00"
t.pack({bytes: []})               # => "\x00\x00\x00\x00"
t.pack({bytes: nil})              # => "\x00\x00\x00\x00"
t.pack({})                        # => "\x00\x00\x00\x00"
t.pack({word: 20, bytes: []})     # => CTypes::Error

using pad_bytes

t = union { member :u8, uint8, member :u32, uint32 }
t.pack({u8: 0}, pad_bytes: "ABCD") # => "\0BCD"

Parameters:

  • value (Hash) —

    value to be encoded; must have size <= 1

  • endian (Symbol) (defaults to: default_endian) —

    endian to pack with

  • validate (Boolean) (defaults to: true) —

    set to false to disable value validation

  • pad_bytes (String) (defaults to: nil) —

    bytes to used to pad; defaults to null bytes

Returns:

  • (::String) —

    binary encoding for value



175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
 
# File 'lib/ctypes/union.rb', line 175

def self.pack(value, endian: default_endian, validate: true, pad_bytes: nil)
  value = value.to_hash.freeze
  unknown_keys = value&.keys
  members = @fields.filter_map do |name, type|
    case name
    when Symbol
      unknown_keys.delete(name)
      [name, type, value[name]] if value.has_key?(name)
    when ::Array
      unknown_keys.reject! { |k| name.include?(k) }
      [name, type, value.slice(*name)] if name.any? { |n| value.has_key?(n) }
    else
      raise Error, "unsupported field name type: %p" % [name]
    end
  end

  # raise an error if they provided multiple member values
  if members.size > 1
    raise Error, <<~MSG % [members.map { |name, _| name }]
      conflicting values for Union#pack; only supply one union member: %p
    MSG

  # raise an error if they provided extra keys that aren't for a member
  elsif !unknown_keys.empty?
    raise Error, "unknown member names: %p" % [unknown_keys]

  # if they didn't provide any key, use the first member
  elsif members.empty?
    name, type = @fields.first
    members << [name, type, type.default_value]
  end

  # we have a single member value to pack; let's grab the type & value and
  # pack it
  _, type, val = members[0]
  out = if type.respond_to?(:ancestors) && type.ancestors.include?(Union)
    type.pack(val, endian: type.endian || endian, validate:, pad_bytes:)
  else
    type.pack(val, endian: type.endian || endian, validate:)
  end

  # @size has two different behaviors.  When @size is a proc, then the size
  # is an absolute length.  Otherwise, size is a minimum length.  To start
  # with, let's calculate a minimum length.
  #
  # @size has two different behaviors.  When @size is a Proc, it represents
  # an absolute length.  Otherwise, size represents a minimum length.  We
  # do this to support unions with greedy members without the union itself
  # being greedy.
  #
  # So grab the minimum length of the output string and expand the output
  # to be at least that long.
  min_length = if @size.is_a?(Proc)

    # Run the size proc with a Union made of our output.  Yes we end up
    # unpacking what we just packed in this case, but it's the cost of
    # supporting pack of dynamically sized unions.
    begin
      instance_exec(new(buf: out, endian:).freeze, &@size)

    # so there's a chance that the packed union value has fewer bytes
    # than what is required to evaluate the size proc.  If we get a missing
    # bytes error, let's pad the out string with the needed number of
    # bytes.  This may happen multiple times as we have no idea what is
    # in the size proc.
    rescue CTypes::MissingBytesError => ex
      out << if pad_bytes && out.size < pad_bytes.size
        pad_bytes.byteslice(out.size, ex.need)
      else
        "\0" * ex.need
      end
      retry
    end
  else
    @size
  end

  # when we need to pad the output, use pad_bytes first
  if out.size < min_length && pad_bytes
    out << pad_bytes.byteslice(out.size, min_length - out.size)
  end

  # if we still need more bytes, pad with zeros
  if out.size < min_length
    out << "\0" * (min_length - out.size)
  end

  # Now, if @size is a Proc only return the absolute length bytes of our
  # output string.  Everything else gets the full output string
  @size.is_a?(Proc) ? out.byteslice(0, min_length) : out
end

.pretty_print(q) ⇒ Object

:nodoc:



381
382
383
384
385
386
387
388
389
390
391
392
393
 
# File 'lib/ctypes/union.rb', line 381

def self.pretty_print(q) # :nodoc:
  q.ctype("union", @endian) do
    q.seplist(@fields, -> { q.breakable("; ") }) do |name, type|
      case name
      when Symbol
        q.text("member %p, " % name)
      when ::Array
        q.text("member ")
      end
      q.pp(type)
    end
  end
end

.size ⇒ Object

return minimum size of the Union

See Also:

  • Type.size


342
343
344
 
# File 'lib/ctypes/union.rb', line 342

def self.size
  @size
end

.sizeof(member) ⇒ Object

return the size of a member

See Also:

  • Type.size


348
349
350
 
# File 'lib/ctypes/union.rb', line 348

def self.sizeof(member)
  @field_types[member][1].size
end

.type_name ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

return the struct name



326
327
328
 
# File 'lib/ctypes/union.rb', line 326

def self.type_name
  @name
end

.unpack_field(field:, buf:, endian:) ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Raises:



312
313
314
315
316
317
318
319
320
321
322
 
# File 'lib/ctypes/union.rb', line 312

def self.unpack_field(field:, buf:, endian:)
  name, type = @field_types[field]
  raise UnknownMemberError, "unknown member: %p" % [field] unless type

  case name
  when Symbol
    {name => type.with_endian(endian).unpack(buf)}
  when ::Array
    type.with_endian(endian).unpack(buf, endian:)
  end
end

.unpack_one(buf, endian: default_endian) ⇒ ::Array(Union, ::String)

convert a String containing the binary represention of a c union into a ruby type

Examples:

class Msg < CTypes::Union
  layout do
    type = enum(uint8, {invalid: 0, hello: 1, read: 2})
    member :hello, struct({type:, version: string})
    member :read, struct({type:, offset: uint64, len: uint64})
    member :type, type
  end
end

msg, rest = Msg.unpack_one("\x02" +
                           "\xfe\xfe\xfe\xfe\xfe\xfe\xfe\xfe" +
                           "\xab\xab\xab\xab\xab\xab\xab\xab" +
                           "extra_bytes")
msg.type                      # => :read
msg.read.offset               # => 0xfefefefefefefefe
msg.read.len                  # => 0xabababababababab
rest                          # => "extra_bytes"

Parameters:

  • buf (String) —

    bytes that make up the type

  • endian (Symbol) (defaults to: default_endian) —

    endian of data within buf

Returns:

  • (::Array(Union, ::String)) —

    Union, and remaining bytes

See Also:



294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
 
# File 'lib/ctypes/union.rb', line 294

def self.unpack_one(buf, endian: default_endian)
  size = if @size.is_a?(Proc)
    instance_exec(new(buf:, endian:).freeze, &@size)
  else
    @size
  end

  raise missing_bytes_error(input: buf, need: size) if buf.size < size
  if fixed_size? || @size.is_a?(Proc)
    buf, rest = buf.byteslice(0, size), buf.byteslice(size..)
  else
    rest = ""
  end

  [new(buf:, endian:), rest]
end

Instance Method Details

#[](name) ⇒ Object

Note:

only the value for the most recently accessed member is cached

Note:

WARNING: accessing any member will erase any modifications made to other members of the union

get a member value

Examples:

include CTypes::Helpers
t = union(word: uint32, bytes: array(uint8, 4), str: string)
u = t.unpack("hello world")
u[:str]                           # => "hello world"
u[:word]                          # => 1819043176
u[:bytes]                         # => [104, 101, 108, 108]

nested struct

include CTypes::Helpers
t = union(a: struct(a: uint8, b: uint8, c: uint16), raw: string)
u = t.unpack("\x01\x02\xed\xfe")
u.a                               # => #<struct a=1, b=2, c=0xfeed>

ERROR: wiping modified member value by accident

include CTypes::Helpers
t = union(word: uint32, bytes: array(uint8, 4))
u = t.new
u[:bytes] = [1,2,3]
u[:word]                          # ERROR!!!  erases changes to bytes
u[:bytes]                         # => [0, 0, 0, 0]

Parameters:

  • member (Symbol) —

    member name



477
478
479
480
481
482
483
484
485
486
 
# File 'lib/ctypes/union.rb', line 477

def [](name)
  v = active_field(name)[name]
  unless frozen? ||
      v.is_a?(Integer) ||
      v.is_a?(TrueClass) ||
      v.is_a?(FalseClass)
    @changed = true
  end
  v
end

#[]=(name, value) ⇒ Object

Note:

WARNING: modifying any member will erase any modifications made to other members of the union

set a member value

Examples:

include CTypes::Helpers
t = union(word: uint32, bytes: array(uint8, 4), str: string)
u = t.new
u[:bytes] = [1,2,3,4]
u.to_binstr                       # => "\x01\x02\x03\x04"
u[:bytes] = [1,2,3]
u.to_binstr                       # => "\x01\x02\x03\x00"

Parameters:

  • member (Symbol) —

    member name

  • value —

    member value

Raises:

  • (FrozenError) 


502
503
504
505
506
 
# File 'lib/ctypes/union.rb', line 502

def []=(name, value)
  raise FrozenError, "can't modify frozen Union: %p" % [self] if frozen?
  active_field(name)[name] = value
  @changed = true
end

#freeze ⇒ Object

freeze the values within the Union

This is used to eliminate the pack/unpack performance penalty when accessing multiple members in a read-only Union. By freezing the Union we can avoid packing the existing member when accessing another memeber.



439
440
441
442
 
# File 'lib/ctypes/union.rb', line 439

def freeze
  @frozen = true
  self
end

#frozen? ⇒ Boolean

Returns:

  • (Boolean) 


444
445
446
 
# File 'lib/ctypes/union.rb', line 444

def frozen?
  @frozen == true
end

#has_key?(name) ⇒ Boolean

Returns:

  • (Boolean) 


508
509
510
 
# File 'lib/ctypes/union.rb', line 508

def has_key?(name)
  self.class.has_field?(name)
end

#pretty_print(q) ⇒ Object

:nodoc:



548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
 
# File 'lib/ctypes/union.rb', line 548

def pretty_print(q) # :nodoc:
  # before printing, apply any changes to the buffer
  apply_changes!

  active = to_h

  open = if (name = self.class.type_name || self.class.name)
    "union #{name} {"
  else
    "union {"
  end
  q.group(4, open, "}") do
    q.seplist(self.class.fields, -> { q.breakable("") }) do |name|
      q.text(".#{name} = ")
      unless active.has_key?(name)
        begin
          v = self.class.unpack_field(field: name, buf: @buf, endian: @endian)
          active.merge!(v)
        rescue Error => ex
          active[name] = "[unpack failed: %p]" % [ex]
        end
      end

      q.pp(active[name])
      q.text(", ")
    end
  end
end

#to_binstr(endian: @endian) ⇒ String

return the binary representation of this Union

This method calls [Union.pack] on the most recentlu accessed member of the Union. If no member has been accessed, it returns the original String it was initialized with

Examples:

include CTypes::Helpers
t = union(word: uint32, bytes: array(uint8, 4), str: string)
u = t.new
u.bytes = [1,2,3,4]
u.to_binstr                       # => "\x01\x02\x03\x04"

accessing member after modifying other member resets members

include CTypes::Helpers
t = union(word: uint32, bytes: array(uint8, 4), str: string)
u = t.new
u.bytes = [1,2,3,4]
u.to_binstr                       # => "\x01\x02\x03\x04"
u.word                            # => ERROR: resets changes to .bytes
u.to_binstr                       # => "\x00\x00\x00\x00"

Returns:

  • (String) —

    binary representation of union



601
602
603
604
605
606
607
608
609
610
 
# File 'lib/ctypes/union.rb', line 601

def to_binstr(endian: @endian)
  endian ||= self.class.default_endian

  if endian != @endian
    self.class.pack((@active_field || active_field).to_h, endian:)
  else
    apply_changes!
    @buf.dup
  end
end

#to_h(shallow: false) ⇒ Hash Also known as: to_hash

return a Hash representation of the Union

Examples:

include CTypes::Helpers
t = union(bytes: array(uint8, 4),
          str: string,
          nested: struct(a: uint8, b: uint8, c: uint16))
u = t.unpack("hello world")
u.to_h                            # => {:bytes=>[104, 101, 108, 108],
                                  #     :str=>"hello world",
                                  #     :nested=>{
                                  #       :a=>104, :b=>101, :c=>27756
                                  #     }}

Parameters:

  • shallow (Boolean) (defaults to: false) —

    set to true to disable deep traversal

Returns:

  • (Hash) 


527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
 
# File 'lib/ctypes/union.rb', line 527

def to_h(shallow: false)
  # grab the cached active field, or the default one
  out = @active_field || active_field
  out = out.is_a?(Hash) ? out.dup : out.to_h

  # now convert all the values to hashes unless we're doing a shallow to_h
  unless shallow
    out.transform_values! do |v|
      case v
      when ::Array
        v
      else
        v.respond_to?(:to_h) ? v.to_h : v
      end
    end
  end

  out
end