Google Storage using API v2

last updated 30/9/2012

I wrote this gem to work with the newer v2 of the Google Storage API. It will run fine on all platforms and has been tested to work with ruby 1.8.7 and above. Let me know if you find any bugs though and feel free to make contributions.

Setup Guide

This guide assumes you’ve already set up a Google Storage account through Google, if you haven’t already yet, read the ‘Setup your Google Project Client ID’s’ section further below and create one first before trying to setup this gem.

There’s a couple of steps to follow in order to setup this gem. The new Google Storage APIv2 uses OAuth 2.0 to authenticate requests, you can read up on the nitty gritty details if you like, here: Google’s OAuth2 Guide

I’ll try and add client side support also once this gem is up and running properly, but for the moment it’s only setup to support server side applications.

But it’s also possible to run this gem from the command line, or through a plain ruby script if you want.

Rails setup

#Include the google_storage gem in your Gemfile and install config files
gem 'google_storage'
bundle install

#Then in your Rails application directory run:
cd /rails_app
rails generate google_storage:install

#This will generate some rake tasks and a google_storage.yml file in your config directory.
/rails_app/config/google_storage.yml
/rails_app/lib/tasks/google_storage.task

#Before running any of the rake tasks, update the google_storage.yml file with your own Google Storage login details
#and read the commented out code in the google_storage.yml on how to use the rake tasks to acquire a refresh token
#from google

Ruby setup

#Install the google_storage gem first

gem install google_storage

#From the command line run: 'deploy_gs_yml' to generate a google_storage.yml file

cd /tmp
deploy_gs_yml

#This will generate a template file: /tmp/google_storage.yml

#Open the generated google_storage.yml file and enter your own google credentials
#Then run IRB and type the following:

require 'google_storage'
client = GoogleStorage::Client.new(:config_yml => '/tmp/google_storage.yml')

#The authorization_url takes 1 of 3 arguments depending on how much access you want to grant yourself:
#(:read_only, :read_write or :full_control)

puts client.authorization_url(:full_control)

#This will output a really long URL, copy and paste this URL into a browser and you'll be prompted by Google
#to allow access to your google storage account.
#Accept this request and your browser will redirect to a "PAGE NOT FOUND" error most likely. This is fine.
#In your browsers URL now though, you should see something like this:

http://localhost:3000/example?code=4/oSdp5JUOiTVcq9p6SW6rjAfrdkuG.wngsnANBsI4RgrKXntQAax1m82kodAI

#This is an authentication token. Copy the token from this URL and issue one more command in IRB:

puts client.acquire_refresh_token('4/oSdp5JUOiTVcq9p6SW6rjAfrdkuG.wngsnANBsI4RgrKXntQAax1m82kodAI')

#Copy the output result from this command into the bottom of your google_storage.yml file

refresh_token: 1/Lyti64r-CmpYTpmXC6n4zKK84oh__6yArTPfgANTP9f

#You should now be setup and able to perform the commands in the 'Examples' section below.

Setup your Google Project Client ID’s

Visit Google Storage and select activate Google Storage. If you haven’t already got a project set up, it’ll prompt you to create one.

When you have access to your Google APIs Console, you need to enable Google Storage. When you select enable, you’ll be shown terms and conditions and you’ll then need to setup billing. They have full pricing details there for you to check out as well but I think it’s pretty reasonable..

Create a client ID for your project On the left menu select “API Access” and then “Create an OAuth 2.0 client ID” Enter your project’s name and brand information.

Select Application type = Web Application and enter your site’s information. If you’re using localhost and running locally make sure you include the port number you’re using as well.

Enter your Client ID details into your google_storage.yml

This is the example google_storage.yml file that is copied into your config directory of your rails application.

Follow the steps to obtain a refresh token from google.

#Replace the following example ids with your own

google_config:
  x-goog-project-id: 825628431245

#Client ID for web applications
web_applications:
  client_id: 825628431245.apps.googleusercontent.com
  client_secret: lpNYX5SPFDB6N-40lyMIPlQn
  redirect_uris: 'http://localhost:3000/example'
  js_origins: 'http://localhost:3000'

#refresh_token: replace this line with a refresh token from google, read below on how to get one

#You need to acquire a refresh token from google so that the google_storage gem
#can acquire access tokens whenever it needs to make new requests

# 1. Make sure you've signed up for Google Storage and filled in the above client ID details
#    for your web application first
#
# 2. Depending on how much access you want to grant to your application run
#    ONE of the following from your applications root directory. If you intend to be able to create and
#    destroy objects and buckets and also be able to set permissions then use full_control
#
#           rake gs:grant:read_only
#           rake gs:grant:read_write
#           rake gs:grant:full_control
#
# 3. Step 2 will generate a URL for you. Copy and paste the URL into a browser and you should be prompted
#    by google to authorize the request by logging into your browser using the google email account you setup
#    your google storage account with
#
# 4. When you 'allow access' you'll be redirected back to the redirect URL you setup in your client ID
#    Your redirect URL should now include an authorization code. It'll look something like this:
#    http://localhost:3000/example?code=4/WvlklnjtybhRJpaKpmDYrzIhAzyx
#
# 5. Copy that code from your URL and run the following rake task from your application directory
#
#           rake gs:refresh_token['paste_your_auth_code_here']
#  Example: rake gs:refresh_token['4/WvlklnjtybhRJpaKpmDYrzIhAzyx']
#
# 6. If everything worked you should see something that looks like the following:
#
#           refresh_token: 1/x4X-U57snRMkLIWWYHWLCXPbfcnyGsdfx04sWAiG_1k
#
# 7. Copy and paste the refresh_token into this file. Your application should now be able to make calls to your
#    Google Storage API

Once you’ve acquired your refresh_token you can now make calls to the API.

Response Object

A lot of the responses from GS on the backend are a little bit inconsistent, sometimes returning response codes, sometimes returning heavily nested xml data and sometimes just returning information in the response header. I’ve tried to parse all responses that are returned from GS into a fairly consistent type of response Hash object. Usually this is in the form of something like this:

#successful responses will always include a :success => true key/value pair

client.list_buckets
=> {:success=>true,
        :buckets=>[ {"Name"=>"bucket_1_example", "CreationDate"=>"2011-06-07T07:11:18.480Z"},
                    {"Name"=>"bucket_2_example", "CreationDate"=>"2011-05-31T10:58:08.097Z"},
                    {"Name"=>"bucket_3_example", "CreationDate"=>"2011-06-06T22:47:10.728Z"}],
        :raw=>{ THIS :raw FIELD WILL RETURN A HASH OF THE UNPARSED RESPONSE IN CASE YOU NEED IT }
    }

#Unsuccessful responses will return an "Error" Hash

client.get_bucket("bucket_4_example")
=> {"Error"=>{
        "Code"=>"NoSuchBucket",
        "Message"=>"The specified bucket does not exist."
    }}

Examples

Configuration

require ‘google_storage’

The following will look for google_storage.yml in your rails config directory

client = GoogleStorage::Client.new

Otherwise you can pass in the path to the google_storage.yml

client = GoogleStorage::Client.new(:config_yml => ‘C:/rails_app/config/google_storage.yml’)

Service Requests

GET Service

client.list_buckets

Bucket Requests

GET Access Control List for Bucket

client.bucket_acls(‘bucket_name’)

PUT Bucket

client.create_bucket(‘bucket_name’)                                # < private bucket
client.create_bucket(‘bucket_name’, :x_goog_acl => ‘public-read’)  # < public bucket  

GET Bucket

client.get_bucket(‘bucket_name’)

DELETE Bucket

client.delete_bucket(‘bucket_name’)

Object Requests

GET Access Control List for Object

client.object_acls(‘bucket_name’, ‘filename.jpg’)

GET Object

client.get_object(‘bucket_name’, ‘filename.jpg’)
client.get_object(‘bucket_name’, ‘filename.jpg’, :write_to_file => ‘c:/example/new_file_name.jpg’)

POST Object

Sorry, not including a ‘post’ object method as it still requires use of the old legacy access,
Please use the ‘put’ object method below instead to upload files to a bucket.

PUT Object

client.put_object(‘bucket_name’, ‘filename.jpg’, :data => File.read(‘c:/example/file.jpg’), :x_goog_acl => ‘public-read’)
client.upload_object(‘bucket_name’, ‘filename.jpg’, :path_to_file => ‘c:/example/file.jpg’)

HEAD Object

client.object_head(‘bucket_name’, ‘filename.jpg’)

DELETE Object

client.delete_object(‘bucket_name’, ‘filename.jpg’)

Contributions made by (and thanks to) the following people:

Brian Armstrong Peter Graham ilyakatz karlentwistle

License

MIT License

Copyright © 2011 by Lucas Hills

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.