Class: AWS::S3::ObjectCollection

Inherits:
Object
  • Object
show all
Includes:
Core::Model, PrefixAndDelimiterCollection, Enumerable
Defined in:
lib/aws/s3/object_collection.rb

Overview

Represents a collection of S3 objects.

Getting an S3Object by Key

If you know the key of the object you want, you can reference it this way:

# this will not make any requests against S3
object = bucket.objects['foo.jpg']
object.key #=> 'foo.jpg'

Finding objects with a Prefix

Given a bucket with the following keys:

photos/sunset.jpg
photos/sunrise.jpg
photos/winter.jpg
videos/comedy.mpg
videos/dancing.mpg

You can list objects that share a prefix:

bucket.objects.with_prefix('videos').collect(&:key)
#=> ['videos/comedy.mpg', 'videos/dancing.mpg']

Exploring Objects with a Tree Interface

Given a bucket with the following keys:

README.txt
videos/wedding.mpg
videos/family_reunion.mpg
photos/2010/house.jpg
photos/2011/fall/leaves.jpg
photos/2011/summer/vacation.jpg
photos/2011/summer/family.jpg

tree = bucket.objects.with_prefix('photos').as_tree

directories = tree.children.select(&:branch?).collect(&:prefix)
#=> ['photos/2010', 'photos/2011']

Defined Under Namespace

Classes: BatchHelper

Instance Attribute Summary collapse

Attributes included from PrefixedCollection

#prefix

Attributes included from Core::Model

#config

Instance Method Summary collapse

Methods included from PrefixAndDelimiterCollection

#as_tree

Methods included from Core::Collection::Limitable

#each_batch

Methods included from Core::Collection

#each_batch, #enum, #first, #in_groups_of, #page

Methods included from Core::Model

#client, #config_prefix, #inspect

Constructor Details

#initialize(bucket, options = {}) ⇒ ObjectCollection

Returns a new instance of ObjectCollection.

Parameters:

  • bucket (Bucket) —

    The S3 bucket this object collection belongs to.

  • options (Hash) (defaults to: {}) 


67
68
69
70
 
# File 'lib/aws/s3/object_collection.rb', line 67

def initialize(bucket, options = {})
  @bucket = bucket
  super
end

Instance Attribute Details

#bucket ⇒ Bucket (readonly)

Returns The bucket this collection belongs to.

Returns:

  • (Bucket) —

    The bucket this collection belongs to.



73
74
75
 
# File 'lib/aws/s3/object_collection.rb', line 73

def bucket
  @bucket
end

Instance Method Details

#[](key) ⇒ S3Object

Returns an S3Object given its name. For example:

Examples:


object = bucket.objects['file.txt']
object.class #=> S3Object

Parameters:

  • key (String) —

    The object key.

Returns:



97
98
99
 
# File 'lib/aws/s3/object_collection.rb', line 97

def [] key
  S3Object.new(bucket, key.to_s)
end

#create(key, *args) ⇒ S3Object

Writes a new object to S3.

The first param is the key you want to write this object to. All other params/options are documented in S3Object#write.

Parameters:

  • key (String) —

    Where in S3 to write the object.

Returns:

See Also:



84
85
86
 
# File 'lib/aws/s3/object_collection.rb', line 84

def create key, *args
  self[key].write(*args)
end

#delete(*objects) ⇒ nil

Deletes the objects provided in as few requests as possible.

# delete 2 objects (by key) in a single request
bucket.objects.delete('abc', 'xyz')

You can delete objects also by passing their S3Object representation:

to_delete = []
to_delete << buckets.objects['foo']
to_delete << buckets.objects['bar']

bucket.objects.delete(to_delete)

Parameters:

  • objects (Mixed) —

    One or more objects to delete. Each object can be one of the following:

Returns:

  • (nil)

Raises:

  • (BatchDeleteError) —

    If any of the objects failed to delete, a BatchDeleteError will be raised with a summary of the errors.



132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
 
# File 'lib/aws/s3/object_collection.rb', line 132

def delete *objects

  objects = objects.flatten.collect do |obj|
    case obj
    when String        then { :key => obj }
    when Hash          then obj
    when S3Object      then { :key => obj.key }
    when ObjectVersion then { :key => obj.key, :version_id => obj.version_id }
    else  
      msg = "objects must be keys (strings or hashes with :key and " +
            ":version_id), S3Objects or ObjectVersions, got " +
            object.class.name
      raise ArgumentError, msg
    end
  end

  batch_helper = BatchHelper.new(1000) do |batch|
    client_opts = {}
    client_opts[:bucket_name] = bucket.name
    client_opts[:quiet] = true
    client_opts[:objects] = batch
    client.delete_objects(client_opts)
  end

  error_counts = {}
  batch_helper.after_batch do |response|
    response.errors.each do |error|
      error_counts[error.code] ||= 0
      error_counts[error.code] += 1
    end
  end

  objects.each do |object|
    batch_helper.add(object)
  end

  batch_helper.complete!

  raise Errors::BatchDeleteError.new(error_counts) unless 
    error_counts.empty?

  nil

end

#delete_all ⇒ Array

Deletes all objects represented by this collection.

Examples:

Delete all objects from a bucket


bucket.objects.delete_all

Delete objects with a given prefix


bucket.objects.with_prefix('2009/').delete_all

Returns:

  • (Array) —

    Returns an array of results

Raises:

  • (BatchDeleteError) —

    If any of the objects failed to delete, a BatchDeleteError will be raised with a summary of the errors.



231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
 
# File 'lib/aws/s3/object_collection.rb', line 231

def delete_all

  error_counts = {}

  each_batch do |objects|
    begin
      delete(objects)
    rescue Errors::BatchDeleteError => error
      error.error_counts.each_pair do |code,count|
        error_counts[code] ||= 0
        error_counts[code] += count
      end
    end
  end

  raise Errors::BatchDeleteError.new(error_counts) unless 
    error_counts.empty?

  nil

end

#delete_if {|object| ... } ⇒ Object

Deletes each object in the collection that returns a true value from block passed to this method. Deletes are batched for efficiency.

# delete text files in the 2009 "folder"
bucket.objects.with_prefix('2009/').delete_if {|o| o.key =~ /\.txt$/ }

Yield Parameters:

Raises:

  • (BatchDeleteError) —

    If any of the objects failed to delete, a BatchDeleteError will be raised with a summary of the errors.



188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
 
# File 'lib/aws/s3/object_collection.rb', line 188

def delete_if &block

  error_counts = {}

  batch_helper = BatchHelper.new(1000) do |objects| 
    begin
      delete(objects)
    rescue Errors::BatchDeleteError => error
      error.error_counts.each_pair do |code,count|
        error_counts[code] ||= 0
        error_counts[code] += count
      end
    end
  end

  each do |object|
    batch_helper.add(object) if yield(object)
  end

  batch_helper.complete!

  raise Errors::BatchDeleteError.new(error_counts) unless 
    error_counts.empty?

  nil

end

#each(options = {}, &block) ⇒ nil

Iterates the collection, yielding instances of S3Object.

Use break or raise an exception to terminate the enumeration.

Parameters:

  • options (Hash) (defaults to: {})

Options Hash (options):

  • :limit (Integer) — default: nil —

    The maximum number of objects to yield.

  • :batch_size (Integer) — default: 1000 —

    The number of objects to fetch each request to S3. Maximum is 1000 keys at time.

Returns:

  • (nil) 


263
264
265
 
# File 'lib/aws/s3/object_collection.rb', line 263

def each options = {}, &block
  super
end

#with_prefix(prefix, mode = :replace) ⇒ Collection

Returns a new collection with a different prefix

Examples:

objects = collection.with_prefix('photos')
objects.prefix #=> 'photos'

Chaining with_prefix replaces previous prefix

objects = collection.with_prefix('photos').with_prefix('videos')
objects.prefix #=> 'videos'

Chaining with_prefix with :append

objects = collection.with_prefix('a/').with_prefix('b/', :append)
objects.prefix #=> 'a/b/'

Chaining with_prefix with :prepend

objects = collection.with_prefix('a/').with_prefix('b/', :prepend)
objects.prefix #=> 'b/a/'

Parameters:

  • prefix (String) —

    The prefix condition that limits what objects are returned by this collection.

  • mode (Symbol) (defaults to: :replace) —

    (:replace) If you chain calls to #with_prefix the mode affects if the prefix prepends, appends, or replaces. Valid modes are:

    • :replace

    • :append

    • :prepend

Returns:

  • (Collection) —

    Returns a new collection with a modified prefix.



102
103
104
 
# File 'lib/aws/s3/object_collection.rb', line 102

def with_prefix prefix, mode = :replace
  super(prefix, mode)
end