Module: Openstack::Swift::Api
Constant Summary collapse
- MAX_SIZE =
4 * 1024 ** 3
Instance Method Summary collapse
-
#account(url, token) ⇒ Object
Get informations about the currect account used to connect to swift Returns: x-account-bytes-used x-account-object-count x-account-container-count.
-
#auth(proxy, user, password) ⇒ Object
Authentication method to get the url and token to conect to swift Returns: x-storage-url x-storage-token x-auth-token.
-
#containers(url, token, query = {}) ⇒ Object
List containers Note that swift only returns 1000 items, so to list more than this you should use the marker option as the name of the last returned item (1000th item) to return the next sequency (1001 - 2000) query options: marker, prefix, limit.
-
#create_container(url, token, container) ⇒ Object
Create a container on swift from a given name.
-
#create_manifest(url, token, container, file_path) ⇒ Object
Creates the manifest file for a splitted upload Given the container and file path a manifest is created to guide the downloads of this splitted file.
-
#delete_container(url, token, container) ⇒ Object
Delete a container for a given name from swift.
-
#delete_object(url, token, container, object) ⇒ Object
Delete a object for a given container and object name.
-
#delete_objects_from_manifest(url, token, container, manifest) ⇒ Object
Delete all files listed on a manifest following the swift standard order What it does is try to delete all files in sequence from a given manifest until there are no files to delete returning a 404 error code.
-
#download_object(url, token, container, object, file_name = nil) ⇒ Object
Downloads an object (file) to disk and returns the saved file path.
-
#object_stat(url, token, container, object) ⇒ Object
Get the object stat given the object name and the container this object is in.
-
#objects(url, token, container, query = {}) ⇒ Object
Get all objects for a given container Query options: marker prefix limit delimiter.
- #upload_object(url, token, container, file_path, options = {}) ⇒ Object
-
#upload_segment(url, token, container, file_path, options = {}) ⇒ Object
Uploads a given object to a given container.
Instance Method Details
#account(url, token) ⇒ Object
Get informations about the currect account used to connect to swift Returns:
x-account-bytes-used
x-account-object-count
x-account-container-count
25 26 27 28 |
# File 'lib/openstack-swift/api.rb', line 25 def account(url, token) query = {:format => "json"} HTTParty.head(url, :headers => {"X-Auth-Token"=> token}, :query => query).headers end |
#auth(proxy, user, password) ⇒ Object
Authentication method to get the url and token to conect to swift Returns:
x-storage-url
x-storage-token
x-auth-token
13 14 15 16 17 18 |
# File 'lib/openstack-swift/api.rb', line 13 def auth(proxy, user, password) res = HTTParty.get(proxy, :headers => {"X-Auth-User" => user, "X-Auth-Key" => password}) raise AuthenticationError unless res.code == 200 [res.headers["x-storage-url"],res.headers["x-storage-token"],res.headers["x-auth-token"]] end |
#containers(url, token, query = {}) ⇒ Object
List containers Note that swift only returns 1000 items, so to list more than this you should use the marker option as the name of the last returned item (1000th item) to return the next sequency (1001 - 2000) query options: marker, prefix, limit
35 36 37 38 39 |
# File 'lib/openstack-swift/api.rb', line 35 def containers(url, token, query = {}) query = query.merge(:format => "json") res = HTTParty.get(url, :headers => {"X-Auth-Token"=> token}, :query => query) res.to_a end |
#create_container(url, token, container) ⇒ Object
Create a container on swift from a given name
61 62 63 64 65 |
# File 'lib/openstack-swift/api.rb', line 61 def create_container(url, token, container) res = HTTParty.put("#{url}/#{container}", :headers => {"X-Auth-Token"=> token}) raise "Could not create container '#{container}'" if res.code < 200 or res.code >= 300 true end |
#create_manifest(url, token, container, file_path) ⇒ Object
Creates the manifest file for a splitted upload Given the container and file path a manifest is created to guide the downloads of this splitted file
77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 |
# File 'lib/openstack-swift/api.rb', line 77 def create_manifest(url, token, container, file_path) file_name = file_path.match(/.+\/(.+?)$/)[1] file_size = File.size(file_path) file_mtime = File.mtime(file_path).to_f.round(2) manifest_path = "#{container}_segments/#{file_name}/#{file_mtime}/#{file_size}/" res = HTTParty.put("#{url}/#{container}/#{file_name}", :headers => { "X-Auth-Token" => token, "x-object-manifest" => manifest_path, "Content-Type" => "application/octet-stream", "Content-Length" => "0" }) raise "Could not create manifest for '#{file_path}'" if res.code < 200 or res.code >= 300 true end |
#delete_container(url, token, container) ⇒ Object
Delete a container for a given name from swift
54 55 56 57 58 |
# File 'lib/openstack-swift/api.rb', line 54 def delete_container(url, token, container) res = HTTParty.delete("#{url}/#{container}", :headers => {"X-Auth-Token"=> token}) raise "Could not delete container '#{container}'" if res.code < 200 or res.code >= 300 true end |
#delete_object(url, token, container, object) ⇒ Object
Delete a object for a given container and object name
126 127 128 129 130 |
# File 'lib/openstack-swift/api.rb', line 126 def delete_object(url, token, container, object) res = HTTParty.delete("#{url}/#{container}/#{object}", :headers => {"X-Auth-Token"=> token}) raise "Could not delete object '#{object}'" if res.code < 200 or res.code >= 300 true end |
#delete_objects_from_manifest(url, token, container, manifest) ⇒ Object
Delete all files listed on a manifest following the swift standard order What it does is try to delete all files in sequence from a given manifest until there are no files to delete returning a 404 error code. We don’t know how many files are stored for a manifest file.
135 136 137 138 139 140 141 142 143 144 145 |
# File 'lib/openstack-swift/api.rb', line 135 def delete_objects_from_manifest(url, token, container, manifest) manifest_info = object_stat(url, token, container, manifest) files_path = "#{manifest_info["x-object-manifest"]}%08d" manifest_info["content-length"].to_i.times do |index| res = HTTParty.delete("#{url}/#{files_path % index}", :headers => {"X-Auth-Token"=> token}) raise ObjectNotFoundError if res.code == 404 end rescue ObjectNotFoundError true end |
#download_object(url, token, container, object, file_name = nil) ⇒ Object
Downloads an object (file) to disk and returns the saved file path
95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
# File 'lib/openstack-swift/api.rb', line 95 def download_object(url, token, container, object, file_name=nil) file_name ||= "/tmp/swift/#{container}/#{object}" # creating directory if it doesn't exist FileUtils.mkdir_p(file_name.match(/(.+)\/.+?$/)[1]) file = File.open(file_name, "wb") uri = URI.parse("#{url}/#{container}/#{object}") req = Net::HTTP::Get.new(uri.path) req.add_field("X-Auth-Token", token) http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE md5 = Digest::MD5.new http.request(req) do |res| res.read_body do |chunk| file.write chunk md5.update(chunk) end raise "MD5 checksum failed for #{container}/#{object}" if res["x-object-manifest"].nil? && res["etag"] != md5.hexdigest end file_name ensure file.close rescue nil end |
#object_stat(url, token, container, object) ⇒ Object
Get the object stat given the object name and the container this object is in
68 69 70 71 72 |
# File 'lib/openstack-swift/api.rb', line 68 def object_stat(url, token, container, object) url = "#{url}/#{container}/#{object}" query = {:format => "json"} HTTParty.head(url, :headers => {"X-Auth-Token"=> token}, :query => query).headers end |
#objects(url, token, container, query = {}) ⇒ Object
Get all objects for a given container Query options:
marker
prefix
limit
delimiter
47 48 49 50 51 |
# File 'lib/openstack-swift/api.rb', line 47 def objects(url, token, container, query = {}) query = query.merge(:format => "json") res = HTTParty.get("#{url}/#{container}", :headers => {"X-Auth-Token"=> token}, :query => query) res.to_a end |
#upload_object(url, token, container, file_path, options = {}) ⇒ Object
147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 |
# File 'lib/openstack-swift/api.rb', line 147 def upload_object(url, token, container, file_path, ={}) [:segments_size] ||= MAX_SIZE create_container(url, token, container) rescue nil file_name, file_mtime, file_size = file_info(file_path) if file_size > [:segments_size] create_container(url, token, "#{container}_segments") rescue nil segments_minus_one = file_size / [:segments_size] last_piece = file_size - segments_minus_one * [:segments_size] segments_minus_one.times do |segment| upload_segment( url, token, "#{container}_segments", file_path, :size => [:segments_size], :position => [:segments_size] * segment, :object_name => upload_path_for(file_path, segment) ) end upload_segment( url, token, "#{container}_segments", file_path, :size => last_piece, :position => [:segments_size] * segments_minus_one, :object_name => upload_path_for(file_path, segments_minus_one) ) create_manifest(url, token, container, file_path) else upload_segment(url, token, container, file_path) end end |
#upload_segment(url, token, container, file_path, options = {}) ⇒ Object
Uploads a given object to a given container
182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 |
# File 'lib/openstack-swift/api.rb', line 182 def upload_segment(url, token, container, file_path, ={}) [:object_name] ||= file_path.match(/.+\/(.+?)$/)[1] file = File.open(file_path, "rb") file.seek([:position]) if [:position] uri = URI.parse("#{url}/#{container}/#{[:object_name]}") req = Net::HTTP::Put.new(uri.path) req.add_field("X-Auth-Token", token) req.body_stream = file req.content_length = [:size] || File.size(file_path) req.content_type = "application/octet-stream" http = Net::HTTP.new(uri.host, uri.port) http.use_ssl = true http.verify_mode = OpenSSL::SSL::VERIFY_NONE http.request(req) ensure file.close rescue nil end |