Uppy::S3Multipart
Provides a Rack application that implements endpoints for the AwsS3Multipart Uppy plugin. This enables multipart uploads directly to S3, which is recommended when dealing with large files, as it allows resuming interrupted uploads.
Installation
Add the gem to your Gemfile:
gem "uppy-s3_multipart"
Setup
Once you've created your S3 bucket, you need to set up CORS for it. The
following script sets up minimal CORS configuration needed for multipart
uploads on your bucket using the aws-sdk-s3
gem:
require "aws-sdk-s3"
client = Aws::S3::Client.new(
access_key_id: "<YOUR KEY>",
secret_access_key: "<YOUR SECRET>",
region: "<REGION>",
)
client.put_bucket_cors(
bucket: "<YOUR BUCKET>",
cors_configuration: {
cors_rules: [{
allowed_headers: ["Authorization", "Content-Type", "Origin", "ETag"],
allowed_methods: ["GET", "POST", "PUT", "DELETE"],
allowed_origins: ["*"],
max_age_seconds: 3000,
}]
}
)
Usage
This gem provides a Rack application that you can mount inside your main
application. If you're using Shrine, you can initialize the Rack application
via the uppy_s3_multipart
Shrine plugin, otherwise you can initialize it
directly.
Shrine
In the initializer load the uppy_s3_multipart
plugin:
require "shrine"
require "shrine/storage/s3"
Shrine.storages = {
cache: Shrine::Storage::S3.new(...),
store: Shrine::Storage::S3.new(...),
}
# ...
Shrine.plugin :uppy_s3_multipart # load the plugin
The plugin will provide a Shrine.uppy_s3_multipart
method, which returns an
instance of Uppy::S3Multipart::App
, which is a Rack app that you can mount
inside your main application:
# Rails (config/routes.rb)
Rails.application.routes.draw do
mount Shrine.uppy_s3_multipart(:cache) => "/s3"
end
# Rack (config.ru)
map "/s3" do
run Shrine.uppy_s3_multipart(:cache)
end
This will add the routes that the AwsS3Multipart
Uppy plugin expects:
POST /s3/multipart
GET /s3/multipart/:uploadId
GET /s3/multipart/:uploadId/:partNumber
POST /s3/multipart/:uploadId/complete
DELETE /s3/multipart/:uploadId
Finally, in your Uppy configuration pass your app's URL as the serverUrl
:
// ...
uppy.use(Uppy.AwsS3Multipart, {
serverUrl: "https://your-app.com/",
})
Both the plugin and method accepts :options
for specifying additional options
to the aws-sdk calls (read further for more details on these options):
Shrine.plugin :uppy_s3_multipart, options: {
create_multipart_upload: { acl: "public-read" } # static
}
# OR
Shrine.uppy_s3_multipart(:cache, options: {
create_multipart_upload: -> (request) { { acl: "public-read" } } # dynamic
})
Standalone
You can also initialize Uppy::S3Multipart::App
directly:
require "uppy/s3_multipart"
resource = Aws::S3::Resource.new(
access_key_id: "...",
secret_access_key: "...",
region: "...",
)
bucket = resource.bucket("my-bucket")
UPPY_S3_MULTIPART_APP = Uppy::S3Multipart::App.new(bucket: bucket)
and mount it in your app in the same way:
# Rails (config/routes.rb)
Rails.application.routes.draw do
mount UPPY_S3_MULTIPART_APP => "/s3"
end
# Rack (config.ru)
map "/s3" do
run UPPY_S3_MULTIPART_APP
end
In your Uppy configuration point the serverUrl
to your application:
// ...
uppy.use(Uppy.AwsS3Multipart, {
serverUrl: "https://your-app.com/",
})
The Uppy::S3Mutipart::App
initializer accepts :options
for specifying
additional options to the aws-sdk calls (read further for more details on these
options):
Uppy::S3Multipart::App.new(bucket: bucket, options: {
create_multipart_upload: { acl: "public-read" }
})
# OR
Uppy::S3Multipart::App.new(bucket: bucket, options: {
create_multipart_upload: -> (request) { { acl: "public-read" } }
})
Custom implementation
If you would rather implement the endpoints yourself, you can utilize
Uppy::S3Multipart::Client
to make S3 requests.
require "uppy/s3_multipart/client"
client = Uppy::S3Multipart::Client.new(bucket: bucket)
create_multipart_upload
Initiates a new multipart upload.
client.create_multipart_upload(key: "foo", **options)
#=> { upload_id: "MultipartUploadId", key: "foo" }
Accepts:
:key
-- object key- additional options for
Aws::S3::Client#create_multipart_upload
Returns:
:upload_id
-- id of the created multipart upload:key
-- object key
#list_parts
Retrieves currently uploaded parts of a multipart upload.
client.list_parts(upload_id: "MultipartUploadId", key: "foo", **options)
#=> [ { part_number: 1, size: 5402383, etag: "etag1" },
# { part_number: 2, size: 5982742, etag: "etag2" },
# ... ]
Accepts:
:upload_id
-- multipart upload id:key
-- object key- additional options for
Aws::S3::Client#list_parts
Returns:
array of parts
:part_number
-- position of the part:size
-- filesize of the part:etag
-- etag of the part
#prepare_upload_part
Returns the endpoint that should be used for uploading a new multipart part.
client.prepare_upload_part(upload_id: "MultipartUploadId", key: "foo", part_number: 1, **options)
#=> { url: "https://my-bucket.s3.amazonaws.com/foo?partNumber=1&uploadId=MultipartUploadId&..." }
Accepts:
:upload_id
-- multipart upload id:key
-- object key:part_number
-- number of the next part- additional options for
Aws::S3::Client#upload_part
andAws::S3::Presigner#presigned_url
Returns:
:url
-- endpoint that should be used for uploading a new multipart part via aPUT
request
#complete_multipart_upload
Finalizes the multipart upload and returns URL to the object.
client.complete_multipart_upload(upload_id: upload_id, key: key, parts: [{ part_number: 1, etag: "etag1" }], **options)
#=> { location: "https://my-bucket.s3.amazonaws.com/foo?..." }
Accepts:
:upload_id
-- multipart upload id:key
-- object key:parts
-- list of all uploaded parts, consisting of:part_number
and:etag
- additional options for
Aws::S3::Client#complete_multipart_upload
Returns:
:location
-- URL to the uploaded object
#abort_multipart_upload
Aborts the multipart upload, removing all parts uploaded so far.
client.abort_multipart_upload(upload_id: upload_id, key: key, **options)
#=> {}
Accepts:
:upload_id
-- multipart upload id:key
-- object key- additional options for
Aws::S3::Client#abort_multipart_upload
Contributing
You can run the test suite with
$ bundle exec rake test
This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the Contributor Covenant code of conduct.
License
The gem is available as open source under the terms of the MIT License.