Class: GoogleStorage::Client

Inherits:
Object
  • Object
show all
Defined in:
lib/google_storage/client.rb,
lib/google_storage/token.rb,
lib/google_storage/bucket.rb,
lib/google_storage/object.rb,
lib/google_storage/request.rb

Overview

Buckets & Objects

Buckets are the basic containers that hold all of your data. There is only one Google namespace so every bucket across the entire namespace has to be uniquely named so you may find that some bucket names have already been taken.

You can have multiple folders and files within a bucket but you can’t nest buckets inside of each other.

Objects are basically just another name for Files, stored within Google Storage. Google Storage stores Objects in 2 parts, holding both the object data and the object metadata. The metadata just holds key value data that describes the objects properties.

Predefined ACL’s (Access Control Lists)

ACL’s allow you to control permission settings on Objects and Buckets

Google Storage uses access control lists (ACLs) to manage object and bucket access. ACLs are the mechanism you use to share objects with other users and allow other users to access your buckets and objects.

At the moment, this google_storage gem only supports the following pre-defined ACL’s. I’ll add support for custom ACL’s soon.

project-private

Gives permission to the project team based on their roles. Anyone who is part of the team has READ permission and project owners and project editors have FULL_CONTROL permission. This is the default ACL that’s applied when you create a bucket.

private

Gives the requester FULL_CONTROL permission for a bucket or object. This is the default ACL that’s applied when you upload an object.

public-read

Gives the requester FULL_CONTROL permission and gives all anonymous users READ permission. When you apply this to an object, anyone on the Internet can read the object without authenticating.

public-read-write

Gives the requester FULL_CONTROL permission and gives all anonymous users READ and WRITE permission. This ACL applies only to buckets.

authenticated-read

Gives the requester FULL_CONTROL permission and gives all authenticated Google account holders READ permission.

bucket-owner-read

Gives the requester FULL_CONTROL permission and gives the bucket owner READ permission. This is used only with objects.

bucket-owner-full-control

Gives the requester FULL_CONTROL permission and gives the bucket owner FULL_CONTROL permission. This is used only with objects.

Instance Method Summary collapse

Constructor Details

#initialize(options = {}) ⇒ Client

You need to initialize a client to be able to make requests with the google_storage gem

Example:

The following will look for google_storage.yml in your rails config directory

client = GoogleStorage::Client.new

Otherwise you can pass in the path to the google_storage.yml

client = GoogleStorage::Client.new(:config_yml => 'C:/example_path/google_storage.yml')

Other options:

:debug => true      <-- This will output all debug information from the HTTP requests to $stderr



63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/google_storage/client.rb', line 63

def initialize(options = {})

  if (options[:config_yml] && !options[:config_yml].nil?) || (defined?(Rails.root) && File.exists?(File.join(Rails.root, 'config', 'google_storage.yml')))
    config_path = options[:config_yml] ? File.expand_path(options[:config_yml]) : File.join(Rails.root, 'config', 'google_storage.yml')
  end

  raise " \nCan't find a google_storage.yml file to initialise with..
\nIf running inside a Rails Application
Please run: rails generate google_storage:install
To generate a google_storage.yml file in your config directory
\nIf running manually within a script or whatever
Initialise GoogleStorage with a path to your config yml file
Example: GoogleStorage::Client.new(:config_yml => 'path to your google storage yml')
\nIf running a rake task try passing: path='path to your google storage yml'
\n\n" unless config_path && File.exists?(config_path)

  config_yml = YAML::load(File.open(config_path))

  @project_id     = config_yml['google_config']['x-goog-project-id']

  @client_id      = config_yml['web_applications']['client_id']
  @client_secret  = config_yml['web_applications']['client_secret']
  @client_secret.force_encoding("UTF-8") if @client_secret.respond_to?(:force_encoding)
  @refresh_token  = config_yml['refresh_token'] if config_yml['refresh_token']

  #TODO Add support for individual permission types
  if config_yml['google_storage_ids']
    @gsid_you       = config_yml['google_storage_ids']['you'] if config_yml['google_storage_ids']['you']
    @gsid_owners    = config_yml['google_storage_ids']['owners'] if config_yml['google_storage_ids']['owners']
    @gsid_editors   = config_yml['google_storage_ids']['editors'] if config_yml['google_storage_ids']['editors']
    @gsid_team      = config_yml['google_storage_ids']['team'] if config_yml['google_storage_ids']['team']
  end

  #TODO - make redirect_uri's support multiple urls
  @redirect_uri   = config_yml['web_applications']['redirect_uris']
  #TODO - maybe add support for API v1 as well... but probably not..
  @api_version    = options[:x_goog_api_version] ? options[:x_goog_api_version] : 2
  @debug          = options[:debug]
  @timeout        = options[:timeout]
  @host           = options[:host] ? options[:host] : 'commondatastorage.googleapis.com'

  @access_token = self.refresh_access_token(@refresh_token)["access_token"]
end

Instance Method Details

#acquire_refresh_token(token, options = {}) ⇒ Object 



27
28
29
30
31
32
 
# File 'lib/google_storage/token.rb', line 27

def acquire_refresh_token(token, options={})
  options['grant_type'] = 'authorization_code'
  response = post_request('accounts.google.com', '/o/oauth2/token', token, options)
  return response["refresh_token"] if response["refresh_token"]
  "Failed to acquire a refresh token. Something went wrong. Try getting a new Auth code."
end

#authorization_url(scope) ⇒ Object 



4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
 
# File 'lib/google_storage/token.rb', line 4

def authorization_url(scope)
  scope_url = case scope
    when :read_only
      'https://www.googleapis.com/auth/devstorage.read_only'
    when :read_write
      'https://www.googleapis.com/auth/devstorage.read_write'
    when :full_control
      'https://www.googleapis.com/auth/devstorage.full_control'
    else
      'https://www.google.com/m8/feeds/'
    end

  auth_url = "https://accounts.google.com/o/oauth2/auth?"
  auth_url += "client_id=#{@client_id}&"
  auth_url += "redirect_uri=#{@redirect_uri}&"
  auth_url += "scope=#{scope_url}&"
  auth_url += "response_type=code&"
  auth_url += "access_type=offline&"
  auth_url += "approval_prompt=force"

  return auth_url
end

#bucket_acls(bucket_name, options = {}) ⇒ Object

Lists the ACL that has been applied to a bucket

Google Ref: code.google.com/apis/storage/docs/reference-methods.html#getbucket

Example:

client.bucket_acls('bucket_name')



39
40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/google_storage/bucket.rb', line 39

def bucket_acls(bucket_name, options={})
  resp = get(bucket_name, '/?acl', options)
  resp_obj = Crack::XML.parse(resp.body)
  if resp_obj["AccessControlList"]
    resp_obj[:success] = true
    resp_obj[:bucket_name] = bucket_name
    resp_obj[:acl] = resp_obj["AccessControlList"]
    resp_obj[:raw] = Crack::XML.parse(resp.body)
    resp_obj.each_key {|key| resp_obj.delete(key) unless key == :success || key == :bucket_name || key == :acl || key == :raw }
  end
  return resp_obj
end

#create_bucket(bucket_name, options = {}) ⇒ Object

Creates a new bucket for your project and applies the ‘project-private’ ACL by default

Google Ref: code.google.com/apis/storage/docs/reference-methods.html#putbucket

You can apply a different ACL to a bucket by passing in an :x_goog_acl option and applying one of the predefined ACL’s

Example:

client.create_bucket('bucket_name')                                <-- private bucket
client.create_bucket('bucket_name', :x_goog_acl => 'public-read')  <-- public readable bucket

Available Options:

:x_goog_acl => 'public-read'



71
72
73
74
75
76
77
78
79
80
81
82
 
# File 'lib/google_storage/bucket.rb', line 71

def create_bucket(bucket_name, options={})
  options[:send_goog_project_id] = true
  resp = put(bucket_name, '/', options)
  resp_obj = Crack::XML.parse(resp.body)
  if resp.code == "200"
    resp_obj.clear
    resp_obj[:success] = true
    resp_obj[:bucket_name] = bucket_name
    resp_obj[:message] = "Bucket created"
  end
  return resp_obj
end

#delete_bucket(bucket_name, options = {}) ⇒ Object

Deletes a specified bucket from your project

Google Ref: code.google.com/apis/storage/docs/reference-methods.html#deletebucket

Note: You can only delete an empty bucket

Example:

client.delete_bucket('bucket_name')



127
128
129
130
131
132
133
134
135
 
# File 'lib/google_storage/bucket.rb', line 127

def delete_bucket(bucket_name, options={})
  resp = delete(bucket_name, '/', options)
  return Crack::XML.parse(resp.body) unless resp.code == "204"
  resp_obj = {}
  resp_obj[:success] = true
  resp_obj[:bucket_name] = bucket_name
  resp_obj[:message] = "Bucket deleted"
  return resp_obj
end

#delete_object(bucket_name, filename, options = {}) ⇒ Object

Deletes an Object from your bucket

Google Ref: code.google.com/apis/storage/docs/reference-methods.html#deleteobject

Example:

client.delete_object('bucket_name', 'file.jpg')



175
176
177
178
179
180
181
182
183
 
# File 'lib/google_storage/object.rb', line 175

def delete_object(bucket_name, filename, options={})
  filename.gsub!(/^\//, "")
  resp = delete(bucket_name, "/#{filename}", options)
  return Crack::XML.parse(resp.body) unless resp.code == "204"
  resp_obj = {}
  resp_obj[:success]          = true
  resp_obj[:message]          = "Object deleted successfully"
  return resp_obj
end

#get_bucket(bucket_name, options = {}) ⇒ Object

Returns a list of all Objects within a specified bucket

Google Ref: code.google.com/apis/storage/docs/reference-methods.html#getbucket

Example:

client.get_bucket('bucket_name')



96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
 
# File 'lib/google_storage/bucket.rb', line 96

def get_bucket(bucket_name, options={})
  resp = get(bucket_name, '/', options)
  resp_obj = Crack::XML.parse(resp.body)
  if resp.code == "200"
    resp_obj[:success] = true
    resp_obj[:bucket_name] = bucket_name
    contents = resp_obj["ListBucketResult"]["Contents"] ? Array.new : nil
    resp_obj["ListBucketResult"]["Contents"].is_a?(Array) ? \
      (contents = resp_obj["ListBucketResult"]["Contents"]) : \
      (contents[0] = resp_obj["ListBucketResult"]["Contents"]) unless contents.nil?
    resp_obj[:contents] = contents
    resp_obj[:raw] = Crack::XML.parse(resp.body)
    resp_obj.each_key {|key| resp_obj.delete(key) unless key == :success || key == :bucket_name || key == :contents || key == :raw  }
  end
  return resp_obj
end

#get_object(bucket_name, filename, options = {}) ⇒ Object

Returns a Google Storage Object inside of a Hash

Google Ref: code.google.com/apis/storage/docs/reference-methods.html#getobject

Example:

Returns a Hash containing your object
  client.get_object('bucket_name', 'example_image.jpg')

Or write the file directly to your file system
  client.get_object('bucket_name', 'example_image.jpg', :write_to_file => 'C:/example/file.jpg')



23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
 
# File 'lib/google_storage/object.rb', line 23

def get_object(bucket_name, filename, options={})
  filename.gsub!(/^\//, "")
  resp = get(bucket_name, "/#{filename}", options)
  return Crack::XML.parse(resp.body) unless resp.code == "200"
  resp_obj = {}
  if options[:write_to_file]
    begin
      File.open(options[:write_to_file], 'wb') {|f| f.write(resp.body) }
    rescue Exception => msg
      return {"Error" => msg}
    end
    resp_obj.clear
    resp_obj[:success]      = true
    resp_obj[:message]      = "File created"
    resp_obj[:path_to_file] = options[:write_to_file]
    return resp_obj
  end
  resp_obj[:success]   = true
  resp_obj[:filename] = filename
  resp_obj[:body]     = resp.body
  resp_obj[:type]     = resp.header["content-type"]
  resp_obj[:size]     = resp.header["content-length"]

  return resp_obj



end

#list_buckets ⇒ Object

Lists all buckets available within your project

Google Ref: code.google.com/apis/storage/docs/reference-methods.html#getservice



13
14
15
16
17
18
19
20
21
22
23
24
25
 
# File 'lib/google_storage/bucket.rb', line 13

def list_buckets
  options = {}
  options[:send_goog_project_id] = true
  resp = get(nil, '/', options)
  resp_obj = Crack::XML.parse(resp.body)
  if resp_obj["ListAllMyBucketsResult"]
    resp_obj[:success] = true
    resp_obj[:buckets] = resp_obj["ListAllMyBucketsResult"]["Buckets"].nil? ? [] : resp_obj["ListAllMyBucketsResult"]["Buckets"]["Bucket"]
    resp_obj[:raw] = Crack::XML.parse(resp.body)
    resp_obj.each_key {|key| resp_obj.delete(key) unless key == :success || key == :buckets || key == :raw }
  end
  return resp_obj
end

#object_acls(bucket_name, filename, options = {}) ⇒ Object

Lists the ACL that has been applied to a particular object

Google Ref: code.google.com/apis/storage/docs/reference-methods.html#getobject

Example:

client.object_acls('bucket_name', 'file.jpg')



121
122
123
124
125
126
127
128
129
130
131
132
133
 
# File 'lib/google_storage/object.rb', line 121

def object_acls(bucket_name, filename, options={})
  filename.gsub!(/^\//, "")
  resp = get(bucket_name, "/#{filename}?acl", options)
  resp_obj = Crack::XML.parse(resp.body)
  if resp_obj["AccessControlList"]
    resp_obj[:success] = true
    resp_obj[:object_name] = filename
    resp_obj[:acl] = resp_obj["AccessControlList"]
    resp_obj[:raw] = Crack::XML.parse(resp.body)
    resp_obj.each_key {|key| resp_obj.delete(key) unless key == :success || key == :object_name || key == :acl || key == :raw }
  end
  return resp_obj
end

#object_head(bucket_name, filename, options = {}) ⇒ Object

Returns the metadata of an Object stored

Google Ref: code.google.com/apis/storage/docs/reference-methods.html#headobject

Example:

client.object_head('bucket_name', 'file.jpg')



147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
 
# File 'lib/google_storage/object.rb', line 147

def object_head(bucket_name, filename, options={})
  filename.gsub!(/^\//, "")
  resp = head(bucket_name, "/#{filename}", options)
  return resp.header unless resp.code == "200"
  resp_obj = {}
  resp_obj[:success]          = true
  resp_obj[:filename]         = filename
  resp_obj[:last_modified]    = resp.header['last-modified'] if resp.header['last-modified']
  resp_obj[:etag]             = resp.header['etag'] if resp.header['etag']
  resp_obj[:content_length]   = resp.header['content-length'] if resp.header['content-length']
  resp_obj[:content_type]     = resp.header['content-type'] if resp.header['content-type']
  resp_obj[:cache_control]    = resp.header['cache-control'] if resp.header['cache-control']
  resp_obj[:date]             = resp.header['date'] if resp.header['date']
  return resp_obj
end

#put_object(bucket_name, filename, options = {}) ⇒ Object Also known as: upload_object

Uploads an Object to Google Storage, or updates if using the same filename

If no :x_goog_acl option is supplied, a ‘private’ ACL is applied by default

Google Ref: code.google.com/apis/storage/docs/reference-methods.html#putobject

Note: If no content type is specified then Google defaults to using ‘binary/octet-stream’

Example:

client.put_object('bucket_name', 'file.jpg', :path_to_file => 'C:/example/file.jpg')
client.put_object('bucket_name', 'file.jpg', :data => File.read('C:/example/file.jpg'))

Available Options:

:x_goog_acl => 'public-read'
:content_type => 'image/jpeg'     <-- It's recommended to always include the content type
:path_to_file => 'path_to_file_you_want_to_upload'
:data => [binary_data]



77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
 
# File 'lib/google_storage/object.rb', line 77

def put_object(bucket_name, filename, options={})
  filename.gsub!(/^\//, "")
  if options[:path_to_file]
    begin
      uploaded_file = File.open(options[:path_to_file])
      if uploaded_file.respond_to?(:get_input_stream)
        uploaded_file.get_input_stream { |io| @data = io.read }
      else
        uploaded_file.binmode
        @data = uploaded_file.read
      end
    rescue Exception => msg
      return {"Error" => msg}
    end
    options[:data] = @data
  end
  resp = put(bucket_name, "/#{filename}", options)
  public_file = (options[:x_goog_acl] && options[:x_goog_acl].match(/public/))
  return Crack::XML.parse(resp.body) unless resp.code == "200"
  resp_obj = {}
  resp_obj[:success]      = true
  resp_obj[:message]      = "Object added successfully"
  resp_obj[:filename]     = filename
  resp_obj[:content_type] = options[:content_type] ? options[:content_type] : 'binary/octet-stream'
  resp_obj[:url]          = public_file ? "http://#{@host}/#{bucket_name}/#{filename}" : \
                                          "https://sandbox.google.com/storage/#{bucket_name}/#{filename}"
  resp_obj[:url_type]     = public_file ? "public" : "private"
  return resp_obj
end