Class: Offroad::CargoStreamer

Inherits:
Object
  • Object
show all
Defined in:
lib/cargo_streamer.rb

Overview

Class for encoding data to, and extracting data from, specially-formatted HTML comments which are called “cargo sections”. Each such section has a name, an md5sum for verification, and some base64-encoded zlib-compressed json data. Multiple cargo sections can have the same name; when the cargo is later read, requests for that name will be yielded each section in turn. The data must always be in the form of arrays of ActiveRecord, or things that walk sufficiently like ActiveRecord

Instance Method Summary collapse

Constructor Details

#initialize(ioh, mode) ⇒ CargoStreamer

Creates a new CargoStreamer on the given stream, which will be used in the given mode (must be “w” or “r”). If the mode is “r”, the file is immediately scanned to determine what cargo it contains.

Raises:



21
22
23
24
25
26
27
28
29
30
31
32
33
 
# File 'lib/cargo_streamer.rb', line 21

def initialize(ioh, mode)
  raise CargoStreamerError.new("Invalid mode: must be 'w' or 'r'") unless ["w", "r"].include?(mode)
  @mode = mode
  
  if ioh.is_a? String
    raise CargoStreamerError.new("Cannot accept string as ioh in write mode") unless @mode == "r"
    @ioh = StringIO.new(ioh, "r")
  else
    @ioh = ioh
  end
  
  scan_for_cargo if @mode == "r"
end

Instance Method Details

#cargo_section_names ⇒ Object

Returns a list of cargo section names available to be read



101
102
103
 
# File 'lib/cargo_streamer.rb', line 101

def cargo_section_names
  return @cargo_locations.keys
end

#each_cargo_section(name) ⇒ Object

Reads, verifies, and decodes each cargo section with a given name, passing each section’s decoded data to the block

Raises:



124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
 
# File 'lib/cargo_streamer.rb', line 124

def each_cargo_section(name)
  raise CargoStreamerError.new("Mode must be 'r' to read cargo data") unless @mode == "r"
  locations = @cargo_locations[name] or return
  locations.each do |seek_location|
    @ioh.seek(seek_location)
    digest = ""
    encoded_data = ""
    @ioh.each_line do |line|
      line.chomp!
      if line == CARGO_END
        break
      elsif digest == ""
        digest = line
      else
        encoded_data += line
      end
    end
    
    yield verify_and_decode_cargo(digest, encoded_data)
  end
end

#first_cargo_element(name) ⇒ Object

Returns the first element from the return value of first_cargo_section



118
119
120
121
 
# File 'lib/cargo_streamer.rb', line 118

def first_cargo_element(name)
  arr = first_cargo_section(name)
  return (arr && arr.size > 0) ? arr[0] : nil
end

#first_cargo_section(name) ⇒ Object

Reads, verifies, decodes, and returns the first cargo section with a given name



111
112
113
114
115
 
# File 'lib/cargo_streamer.rb', line 111

def first_cargo_section(name)
  each_cargo_section(name) do |data|
    return data
  end
end

#has_cargo_named?(name) ⇒ Boolean

Returns true if cargo with a given name is available

Returns:

  • (Boolean) 


106
107
108
 
# File 'lib/cargo_streamer.rb', line 106

def has_cargo_named?(name)
  return @cargo_locations.has_key? name
end

#write_cargo_section(name, value, options = {}) ⇒ Object

Writes a cargo section with the given name and value to the IO stream. Options:

  • :human_readable => true - Before writing the cargo section, writes a comment with human-readable data.

  • :include => [:assoc, :other_assoc] - Includes these first-level associations in the encoded data

Raises:



39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
 
# File 'lib/cargo_streamer.rb', line 39

def write_cargo_section(name, value, options = {})
  raise CargoStreamerError.new("Mode must be 'w' to write cargo data") unless @mode == "w"
  raise CargoStreamerError.new("CargoStreamer section names must be strings") unless name.is_a? String
  raise CargoStreamerError.new("Invalid cargo name '" + name + "'") unless name == clean_for_html_comment(name)
  raise CargoStreamerError.new("Cargo name cannot include newlines") if name.include?("\n")
  raise CargoStreamerError.new("Value must be an array") unless value.is_a? Array
  [:to_xml, :attributes=, :valid?].each do |message|
    unless value.all? { |e| e.respond_to? message }
      raise CargoStreamerError.new("All elements must respond to #{message}") 
    end
  end
  unless value.all? { |e| e.class.respond_to?(:safe_to_load_from_cargo_stream?) && e.class.safe_to_load_from_cargo_stream? }
    raise CargoStreamerError.new("All element classes must be models which are safe_to_load_from_cargo_stream")
  end

  unless options[:skip_validation]
    unless value.all?(&:valid?)
      raise CargoStreamerError.new("All elements must be valid")
    end
  end

  if options[:human_readable]
    human_data = value.map{ |rec|
      rec.attributes.map{ |k, v| "#{k.to_s.titleize}: #{v.to_s}" }.join("\n")
    }.join("\n\n")
    @ioh.write "<!--\n"
    @ioh.write name.titleize + "\n"
    @ioh.write "\n"
    @ioh.write clean_for_html_comment(human_data) + "\n"
    @ioh.write "-->\n"
  end
  
  name = name.chomp
  
  assoc_list = options[:include] || []
  
  xml = Builder::XmlMarkup.new
  xml_data = "<records>%s</records>" % value.map {
    |r| r.to_xml(
      :skip_instruct => true,
      :skip_types => true,
      :root => "record",
      :indent => 0,
      :include => assoc_list
    ) do |xml|
      xml.cargo_streamer_type r.class.name
      assoc_info = assoc_list.reject{|a| r.send(a) == nil}.map{|a| "#{a.to_s}=#{r.send(a).class.name}"}.join(",")
      xml.cargo_streamer_includes assoc_info
    end
  }.join()
  deflated_data = Zlib::Deflate::deflate(xml_data)
  b64_data = Base64.encode64(deflated_data).chomp
  digest = Digest::MD5::hexdigest(deflated_data).chomp
  
  @ioh.write CARGO_BEGIN + "\n"
  @ioh.write name + "\n"
  @ioh.write digest + "\n"
  @ioh.write b64_data + "\n"
  @ioh.write CARGO_END + "\n"
end