Module: Technoweenie::AttachmentFu::Backends::S3Backend

Defined in:
lib/technoweenie/attachment_fu/backends/s3_backend.rb

Overview

AWS::S3 Storage Backend

Enables use of Amazon’s Simple Storage Service as a storage mechanism

Requirements

Requires the aws-sdk-v1 gem.

Configuration

Configuration is done via #{Rails.root}/config/amazon_s3.yml and is loaded according to the #{Rails.env}. The minimum connection options that you must specify are a bucket name, your access key id and your secret access key. If you don’t already have your access keys, all you need to sign up for the S3 service is an account at Amazon. You can sign up for S3 and get access keys by visiting aws.amazon.com/s3.

If you wish to use Amazon CloudFront to serve the files, you can also specify a distibution domain for the bucket. To read more about CloudFront, visit aws.amazon.com/cloudfront

Example configuration (#Rails.root/config/amazon_s3.yml)

development:
  bucket_name: appname_development
  access_key_id: <your key>
  secret_access_key: <your key>
  distribution_domain: XXXX.cloudfront.net

test:
  bucket_name: appname_test
  access_key_id: <your key>
  secret_access_key: <your key>
  distribution_domain: XXXX.cloudfront.net

production:
  bucket_name: appname
  access_key_id: <your key>
  secret_access_key: <your key>
  distribution_domain: XXXX.cloudfront.net

You can change the location of the config path by passing a full path to the :s3_config_path option.

has_attachment :storage => :s3, :s3_config_path => (#{Rails.root} + '/config/s3.yml')

Required configuration parameters

  • :access_key_id - The access key id for your S3 account. Provided by Amazon.

  • :secret_access_key - The secret access key for your S3 account. Provided by Amazon.

  • :bucket_name - A unique bucket name (think of the bucket_name as being like a database name).

If any of these required arguments is missing, a MissingAccessKey exception will be raised from AWS::S3.

About bucket names

Bucket names have to be globaly unique across the S3 system. And you can only have up to 100 of them, so it’s a good idea to think of a bucket as being like a database, hence the correspondance in this implementation to the development, test, and production environments.

The number of objects you can store in a bucket is, for all intents and purposes, unlimited.

Optional configuration parameters

  • :server - The server to make requests to. Defaults to s3.amazonaws.com.

  • :port - The port to the requests should be made on. Defaults to 80 or 443 if :use_ssl is set.

  • :use_ssl - If set to true, :port will be implicitly set to 443, unless specified otherwise. Defaults to false.

  • :distribution_domain - The CloudFront distribution domain for the bucket. This can either be the assigned

    distribution domain (ie. XXX.cloudfront.net) or a chosen domain using a CNAME. See CloudFront for more details.

Usage

To specify S3 as the storage mechanism for a model, set the acts_as_attachment :storage option to :s3.

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3
end

Customizing the path

By default, files are prefixed using a pseudo hierarchy in the form of :table_name/:id, which results in S3 urls that look like: http(s)://:server/:bucket_name/:table_name/:id/:filename with :table_name representing the customizable portion of the path. You can customize this prefix using the :path_prefix option:

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3, :path_prefix => 'my/custom/path'
end

Which would result in URLs like http(s)://:server/:bucket_name/my/custom/path/:id/:filename.

Using different bucket names on different models

By default the bucket name that the file will be stored to is the one specified by the :bucket_name key in the amazon_s3.yml file. You can use the :bucket_key option to overide this behavior on a per model basis. For instance if you want a bucket that will hold only Photos you can do this:

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3, :bucket_key => :photo_bucket_name
end

And then your amazon_s3.yml file needs to look like this.

 development:
   bucket_name: appname_development
   access_key_id: <your key>
   secret_access_key: <your key>

 test:
   bucket_name: appname_test
   access_key_id: <your key>
   secret_access_key: <your key>

 production:
   bucket_name: appname
   photo_bucket_name: appname_photos
   access_key_id: <your key>
   secret_access_key: <your key>

If the bucket_key you specify is not there in a certain environment then attachment_fu will
default to the <tt>bucket_name</tt> key.  This way you only have to create special buckets
this can be helpful if you only need special buckets in certain environments.

Permissions

By default, files are stored on S3 with public access permissions. You can customize this using the :s3_access option to has_attachment. Available values are :private, :public_read_write, and :authenticated_read.

Other options

Of course, all the usual configuration options apply, such as content_type and thumbnails:

class Photo < ActiveRecord::Base
  has_attachment :storage => :s3, :content_type => ['application/pdf', :image], :resize_to => 'x50'
  has_attachment :storage => :s3, :thumbnails => { :thumb => [50, 50], :geometry => 'x50' }
end

Accessing S3 URLs

You can get an object’s URL using the s3_url accessor. For example, assuming that for your postcard app you had a bucket name like ‘postcard_world_development’, and an attachment model called Photo:

@postcard.s3_url # => http(s)://s3.amazonaws.com/postcard_world_development/photos/1/mexico.jpg

The resulting url is in the form: http(s)://:server/:bucket_name/:table_name/:id/:file. The optional thumbnail argument will output the thumbnail’s filename (if any).

Additionally, you can get an object’s base path relative to the bucket root using base_path:

@photo.file_base_path # => photos/1

And the full path (including the filename) using full_filename:

@photo.full_filename # => photos/

Niether base_path or full_filename include the bucket name as part of the path. You can retrieve the bucket name using the bucket_name method.

Accessing CloudFront URLs

You can get an object’s CloudFront URL using the cloudfront_url accessor. Using the example from above: The resulting url is in the form: :distribution_domain/:table_name/:id/:file

If you set :cloudfront to true in your model, the public_filename will be the CloudFront URL, not the S3 URL.

Defined Under Namespace

Modules: ClassMethods Classes: ConfigFileNotFoundError, RequiredLibraryNotFoundError

Class Method Summary collapse

Instance Method Summary collapse

Class Method Details

.distribution_domain ⇒ Object 



219
220
221
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 219

def self.distribution_domain
  @distribution_domain = s3_config[:distribution_domain]
end

.hostname ⇒ Object 



211
212
213
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 211

def self.hostname
  @hostname ||= s3_config[:server] || 's3.amazonaws.com'
end

.included(base) ⇒ Object

:nodoc:



174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 174

def self.included(base) #:nodoc:
  mattr_reader :bucket_name, :s3_config, :s3_conn, :bucket

  begin
    require 'aws-sdk-v1'
  rescue LoadError
    raise RequiredLibraryNotFoundError.new('aws-sdk-v1 could not be loaded. Make sure the gem is installed.')
  end

  begin
    @@s3_config_path = base.attachment_options[:s3_config_path] || File.join(Rails.root, 'config', 'amazon_s3.yml')
    @@s3_config = @@s3_config = YAML.load(ERB.new(File.read(@@s3_config_path)).result)[Rails.env].symbolize_keys
  #rescue
  #  raise ConfigFileNotFoundError.new('File %s not found' % @@s3_config_path)
  end

  bucket_key = base.attachment_options[:bucket_key]

  if bucket_key and s3_config[bucket_key.to_sym]
    eval_string = "def bucket_name()\n  \"#{s3_config[bucket_key.to_sym]}\"\nend"
  else
    eval_string = "def bucket_name()\n  \"#{s3_config[:bucket_name]}\"\nend"
  end
  base.class_eval(eval_string, __FILE__, __LINE__)

  @@s3_conn = AWS::S3.new(s3_config.slice(:access_key_id, :secret_access_key))
  @@bucket = s3_conn.buckets[s3_config[:bucket_name]]

  #Base.establish_connection!(s3_config.slice(:access_key_id, :secret_access_key, :server, :port, :use_ssl, :persistent, :proxy))

  base.before_update :rename_file
end

.port_string ⇒ Object 



215
216
217
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 215

def self.port_string
  @port_string ||= (s3_config[:port].nil? || s3_config[:port] == (s3_config[:use_ssl] ? 443 : 80)) ? '' : ":#{s3_config[:port]}"
end

.protocol ⇒ Object 



207
208
209
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 207

def self.protocol
  @protocol ||= s3_config[:use_ssl] ? 'https://' : 'http://'
end

Instance Method Details

#attachment_path_id ⇒ Object

The attachment ID used in the full path of a file



248
249
250
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 248

def attachment_path_id
  ((respond_to?(:parent_id) && parent_id) || id).to_s
end

#authenticated_s3_url(*args) ⇒ Object

All private objects are accessible via an authenticated GET request to the S3 servers. You can generate an authenticated url for an object like this:

@photo.authenticated_s3_url

By default authenticated urls expire 5 minutes after they were generated.

Expiration options can be specified either with an absolute time using the :expires option, or with a number of seconds relative to now with the :expires_in option:

# Absolute expiration date (October 13th, 2025)
@photo.authenticated_s3_url(:expires => Time.mktime(2025,10,13).to_i)

# Expiration in five hours from now
@photo.authenticated_s3_url(:expires_in => 5.hours)

You can specify whether the url should go over SSL with the :use_ssl option. By default, the ssl settings for the current connection will be used:

@photo.authenticated_s3_url(:use_ssl => true)

Finally, the optional thumbnail argument will output the thumbnail’s filename (if any):

@photo.authenticated_s3_url('thumbnail', :expires_in => 5.hours, :use_ssl => true)



324
325
326
327
328
329
330
331
332
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 324

def authenticated_s3_url(*args)
  options   = args.extract_options!
  options[:expires] = options[:expires_in].to_i if options[:expires_in]
  options[:secure] = options[:use_ssl] if options[:use_ssl]
  options.delete(:expires_in) if options[:expires_in]
  options.delete(:use_ssl) if options[:use_ssl]
  thumbnail = args.shift
  bucket.objects[full_filename(thumbnail)].url_for(:read, options).to_s
end

#base_path(thumbnail = nil) ⇒ Object

The pseudo hierarchy containing the file relative to the bucket name Example: :table_name/:id



254
255
256
257
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 254

def base_path(thumbnail = nil)
  file_system_path = (thumbnail ? thumbnail_class : self).attachment_options[:path_prefix]
  File.join(file_system_path, attachment_path_id)
end

#cloudfront_distribution_domain ⇒ Object 



358
359
360
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 358

def cloudfront_distribution_domain
  Technoweenie::AttachmentFu::Backends::S3Backend.distribution_domain
end

#cloudfront_url(thumbnail = nil) ⇒ Object

All public objects are accessible via a GET request to CloudFront. You can generate a url for an object using the cloudfront_url method.

@photo.cloudfront_url

The resulting url is in the form: http://:distribution_domain/:table_name/:id/:file using the :distribution_domain variable set in the configuration parameters in #{Rails.root}/config/amazon_s3.yml.

The optional thumbnail argument will output the thumbnail’s filename (if any).



288
289
290
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 288

def cloudfront_url(thumbnail = nil)
  s3_protocol + cloudfront_distribution_domain + "/" + full_filename(thumbnail)
end

#create_temp_file ⇒ Object 



334
335
336
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 334

def create_temp_file
  write_to_temp_file current_data
end

#current_data ⇒ Object 



338
339
340
341
342
343
344
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 338

def current_data
  if attachment_options[:encrypted_storage] && self.respond_to?(:encryption_key) && self.encryption_key != nil
    EncryptedData.decrypt_data(bucket.objects[full_filename].read, self.encryption_key)
  else
    bucket.objects[full_filename].read
  end
end

#filename=(value) ⇒ Object

Overwrites the base filename writer in order to store the old filename



242
243
244
245
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 242

def filename=(value)
  @old_filename = filename unless filename.nil? || @old_filename
  write_attribute :filename, sanitize_filename(value)
end

#full_filename(thumbnail = nil) ⇒ Object

The full path to the file relative to the bucket name Example: :table_name/:id/:filename



261
262
263
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 261

def full_filename(thumbnail = nil)
  File.join(base_path(thumbnail), thumbnail_name_for(thumbnail))
end

#public_filename(*args) ⇒ Object 



292
293
294
295
296
297
298
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 292

def public_filename(*args)
  if attachment_options[:cloudfront]
    cloudfront_url(*args)
  else
    s3_url(*args)
  end
end

#s3_hostname ⇒ Object 



350
351
352
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 350

def s3_hostname
  Technoweenie::AttachmentFu::Backends::S3Backend.hostname
end

#s3_port_string ⇒ Object 



354
355
356
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 354

def s3_port_string
  Technoweenie::AttachmentFu::Backends::S3Backend.port_string
end

#s3_protocol ⇒ Object 



346
347
348
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 346

def s3_protocol
  Technoweenie::AttachmentFu::Backends::S3Backend.protocol
end

#s3_url(thumbnail = nil) ⇒ Object

All public objects are accessible via a GET request to the S3 servers. You can generate a url for an object using the s3_url method.

@photo.s3_url

The resulting url is in the form: http(s)://:server/:bucket_name/:table_name/:id/:file where the :server variable defaults to AWS::S3 URL::DEFAULT_HOST (s3.amazonaws.com) and can be set using the configuration parameters in #{Rails.root}/config/amazon_s3.yml.

The optional thumbnail argument will output the thumbnail’s filename (if any).



275
276
277
 
# File 'lib/technoweenie/attachment_fu/backends/s3_backend.rb', line 275

def s3_url(thumbnail = nil)
  File.join(s3_protocol + s3_hostname + s3_port_string, bucket_name, full_filename(thumbnail))
end