Class: Cumulus::CSV::DataFileManager

Inherits:
Object
  • Object
show all
Defined in:
lib/cumulus_csv/data_file_manager.rb

Overview

DataFileManager is the gatekeeper for sending your data files to S3, and for iterating over them later.

In the constructor, It takes the same authentication parameters as aws-s3:

DataFileManager.new(:access_key_id => 'abc',:secret_access_key => '123')

For storing your csv data file on S3, you need to setup a controller to send your uploaded files through this interface:

DataFileManager.new(connection_params).store_uploaded_file!(params[:uploaded_file])

The file will be posted to S3 in a bucket set aside for this gem (it will be created upon connection if it doesn’t exist already)

When you’re ready to iterate over this csv file later in a background job (or wherever), you’ll use this:

DataFileManager.new(connection_params).each_row_of(name) {|row| #...whatever processing you need }

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(connect_params) ⇒ DataFileManager

Returns a new instance of DataFileManager.



23
24
25
26
 
# File 'lib/cumulus_csv/data_file_manager.rb', line 23

def initialize(connect_params)
  AWS::S3::Base.establish_connection!(connect_params)
  cache_bucket
end

Instance Attribute Details

#bucket ⇒ Object (readonly)

Returns the value of attribute bucket.



21
22
23
 
# File 'lib/cumulus_csv/data_file_manager.rb', line 21

def bucket
  @bucket
end

Instance Method Details

#each_row_of(file_name) ⇒ Object 



34
35
36
37
 
# File 'lib/cumulus_csv/data_file_manager.rb', line 34

def each_row_of(file_name) 
  data = AWS::S3::S3Object.value(file_name,BUCKET_NAME)
  ::CSV::Reader.parse(data).each{|row| yield row }
end

#store_uploaded_file!(uploaded_file) ⇒ Object 



28
29
30
31
32
 
# File 'lib/cumulus_csv/data_file_manager.rb', line 28

def store_uploaded_file!(uploaded_file)        
  name = File.basename(uploaded_file.original_filename)   
  AWS::S3::S3Object.store(name,uploaded_file.read,BUCKET_NAME)    
  return name
end