Class: OpenC3::ReducerModel

Inherits:
Object show all
Defined in:
lib/openc3/models/reducer_model.rb

Overview

Tracks the files which are being stored in buckets for data reduction purposes. Files are stored in a Redis set by splitting their filenames and storing in a set named SCOPE__TARGET__reducer__TYPE, e.g. DEFAULT__INST__reducer__decom Where TYPE can be ‘decom’, ‘minute’, or ‘hour’. ‘day’ is not necessary because day is the final reduction state. As files are reduced they are removed from the set. Thus the sets contain the active set of files to be reduced.

Class Method Summary collapse

Class Method Details

.add_file(bucket_key) ⇒ Object 



33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/openc3/models/reducer_model.rb', line 33

def self.add_file(bucket_key)
  # Only reduce tlm files
  bucket_key_split = bucket_key.split('/')
  if bucket_key_split[2] == 'tlm'
    # bucket_key is formatted like STARTTIME__ENDTIME__SCOPE__TARGET__PACKET__TYPE.bin
    # e.g. 20211229191610578229500__20211229192610563836500__DEFAULT__INST__HEALTH_STATUS__rt__decom.bin
    _, _, scope, target, _ = File.basename(bucket_key).split('__')
    case bucket_key
    when /__decom\.bin.gz$/
      Store.sadd("#{scope}__#{target}__reducer__decom", bucket_key)
    when /__reduced_minute\.bin.gz$/
      Store.sadd("#{scope}__#{target}__reducer__minute", bucket_key)
    when /__reduced_hour\.bin.gz$/
      Store.sadd("#{scope}__#{target}__reducer__hour", bucket_key)
    end
    # No else clause because add_file is called with raw files which are ignored
  end
end

.all_files(type:, target:, scope:) ⇒ Object 



68
69
70
 
# File 'lib/openc3/models/reducer_model.rb', line 68

def self.all_files(type:, target:, scope:)
  Store.smembers("#{scope}__#{target}__reducer__#{type.downcase}").sort
end

.rm_file(bucket_key) ⇒ Object 



52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
 
# File 'lib/openc3/models/reducer_model.rb', line 52

def self.rm_file(bucket_key)
  _, _, scope, target, _ = bucket_key.split('__')
  case bucket_key
  when /__decom\.bin.gz$/
    Store.srem("#{scope}__#{target}__reducer__decom", bucket_key)
  when /__reduced_minute\.bin.gz$/
    Store.srem("#{scope}__#{target}__reducer__minute", bucket_key)
  when /__reduced_hour\.bin.gz$/
    Store.srem("#{scope}__#{target}__reducer__hour", bucket_key)
  else
    # We should only remove files that were previously in the set
    # Thus if we don't match the bucket_key it is an error
    raise "Unknown file #{bucket_key}"
  end
end