Class: IMW::Tools::Aggregator

Inherits:
Object
  • Object
show all
Defined in:
lib/imw/tools/aggregator.rb

Overview

Aggregates resources into a single local directory.

The directory should already exist.

Any local resources will be copied into the directory.

Any remote resources will be downloaded into the directory.

If any of the resources are archives, they will first be extracted, with only their contents winding up in the final directory (the file hierarchy of the archive will be preserved).

If any of the resources are compressed, they will first be uncompressed before being added to the directory.

As an example:

aggregator = IMW::Tools::Aggregator.new '/path/to/agg_dir'
aggregator.aggregate '/path/to/my/regular_file.tsv', '/path/to/an/archive.tar.bz2', '/path/to/my_compressed_file.gz', 'http://mywebsite.com/index.html'

This will create a directory at /path/to/agg_dir which looks like

path_to_agg_dir
|-- regular_file.tsv
|-- archive
|   |-- internal_archive_file_1
|   |-- internal_archive_file_2
|   ...
|   `-- internal_archive_file_N
|-- my_compressed_file
`-- index.html

Notice that

  • the local file was copied over

  • the remote file was downloaded and copied over

  • the tar archive was first exctracted

  • the compressed file was aggregated

This process can take a while when the constituent files are large.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(dir) ⇒ Aggregator

Returns a new instance of Aggregator.



55
56
57
 
# File 'lib/imw/tools/aggregator.rb', line 55

def initialize dir
  self.dir = IMW.open(dir)
end

Instance Attribute Details

#dir ⇒ Object

Returns the value of attribute dir.



53
54
55
 
# File 'lib/imw/tools/aggregator.rb', line 53

def dir
  @dir
end

Instance Method Details

#aggregate(*paths_or_inputs) ⇒ IMW::Tools::Aggregator

Aggregate the given inputs into this Aggregator’s dir.

Parameters:

Returns:



91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/imw/tools/aggregator.rb', line 91

def aggregate *paths_or_inputs
  @errors = []
  paths_or_inputs.flatten.compact.each do |path_or_input|
    input = IMW.open(path_or_input)
    if input.is_local?
      aggregate_local_input(input)
    else
      download = download_remote_input(input)
      if download.is_compressed? || download.is_archive?
        aggregate_local_input(download)
        download.rm!
      end
    end
  end
end

#errors ⇒ Array

Return a list of error messages for this Aggregator.

Returns:

  • (Array) —

    the error messages



76
77
78
 
# File 'lib/imw/tools/aggregator.rb', line 76

def errors
  @errors ||= []      
end

#success? ⇒ true, false

Was this archiver successful (did it not have any errors)?

Returns:

  • (true, false) 


83
84
85
 
# File 'lib/imw/tools/aggregator.rb', line 83

def success?
  errors.empty?
end