Class: Rightscale::HttpConnection
- Inherits:
-
Object
- Object
- Rightscale::HttpConnection
- Defined in:
- lib/right_http_connection.rb
Overview
HttpConnection maintains a persistent HTTP connection to a remote server. Each instance maintains its own unique connection to the HTTP server. HttpConnection makes a best effort to receive a proper HTTP response from the server, although it does not guarantee that this response contains a HTTP Success code.
On low-level errors (TCP/IP errors) HttpConnection invokes a reconnect and retry algorithm. Note that although each HttpConnection object has its own connection to the HTTP server, error handling is shared across all connections to a server. For example, if there are three connections to www.somehttpserver.com, a timeout error on one of those connections will cause all three connections to break and reconnect. A connection will not break and reconnect, however, unless a request becomes active on it within a certain amount of time after the error (as specified by HTTP_CONNECTION_RETRY_DELAY). An idle connection will not break even if other connections to the same server experience errors.
A HttpConnection will retry a request a certain number of times (as defined by HTTP_CONNNECTION_RETRY_COUNT). If all the retries fail, an exception is thrown and all HttpConnections associated with a server enter a probationary period defined by HTTP_CONNECTION_RETRY_DELAY. If the user makes a new request subsequent to entering probation, the request will fail immediately with the same exception thrown on probation entry. This is so that if the HTTP server has gone down, not every subsequent request must wait for a connect timeout before failing. After the probation period expires, the internal state of the HttpConnection is reset and subsequent requests have the full number of potential reconnects and retries available to them.
Constant Summary collapse
- HTTP_CONNECTION_RETRY_COUNT =
Number of times to retry the request after encountering the first error
3
- HTTP_CONNECTION_OPEN_TIMEOUT =
Throw a Timeout::Error if a connection isn’t established within this number of seconds
5
- HTTP_CONNECTION_READ_TIMEOUT =
Throw a Timeout::Error if no data have been read on this connnection within this number of seconds
30
- HTTP_CONNECTION_RETRY_DELAY =
Length of the post-error probationary period during which all requests will fail
15
- @@params =
if a request repeatedly fails. RuntimeError is raised if this parameter is omitted.
{}
Instance Attribute Summary collapse
-
#http ⇒ Object
—————— instance methods ——————.
-
#logger ⇒ Object
Returns the value of attribute logger.
-
#params ⇒ Object
see @@params.
-
#server ⇒ Object
Returns the value of attribute server.
Class Method Summary collapse
-
.params ⇒ Object
Query the global (class-level) parameters.
-
.params=(params) ⇒ Object
Set the global (class-level) parameters.
Instance Method Summary collapse
- #finish(reason = '') ⇒ Object
- #get_param(name) ⇒ Object
-
#initialize(params = {}) ⇒ HttpConnection
constructor
Params hash: :user_agent => ‘www.HostName.com’ String to report as HTTP User agent :ca_file => ‘path_to_file’ A path of a CA certification file in PEM format.
-
#local_read_size=(newsize) ⇒ Object
Set the maximum size (in bytes) of a single read from local data sources like files.
-
#local_read_size? ⇒ Boolean
Query for the maximum size (in bytes) of a single read from local data sources like files.
-
#request(request_params, &block) ⇒ Object
Send HTTP request to server.
-
#socket_read_size=(newsize) ⇒ Object
Set the maximum size (in bytes) of a single read from the underlying socket.
-
#socket_read_size? ⇒ Boolean
Query for the maximum size (in bytes) of a single read from the underlying socket.
Constructor Details
#initialize(params = {}) ⇒ HttpConnection
Params hash:
:user_agent => 'www.HostName.com' String to report as HTTP User agent
:ca_file => 'path_to_file' A path of a CA certification file in PEM format. The file can contain several CA certificates.
:logger => Logger object If omitted, HttpConnection logs to STDOUT
:exception => Exception to raise The type of exception to raise if a request repeatedly fails. RuntimeError is raised if this parameter is omitted.
126 127 128 129 130 131 132 133 |
# File 'lib/right_http_connection.rb', line 126 def initialize(params={}) @params = params @http = nil @server = nil @logger = get_param(:logger) || (RAILS_DEFAULT_LOGGER if defined?(RAILS_DEFAULT_LOGGER)) || Logger.new(STDOUT) end |
Instance Attribute Details
#http ⇒ Object
instance methods
113 114 115 |
# File 'lib/right_http_connection.rb', line 113 def http @http end |
#logger ⇒ Object
Returns the value of attribute logger.
116 117 118 |
# File 'lib/right_http_connection.rb', line 116 def logger @logger end |
#params ⇒ Object
see @@params
115 116 117 |
# File 'lib/right_http_connection.rb', line 115 def params @params end |
#server ⇒ Object
Returns the value of attribute server.
114 115 116 |
# File 'lib/right_http_connection.rb', line 114 def server @server end |
Class Method Details
.params ⇒ Object
Query the global (class-level) parameters
101 102 103 |
# File 'lib/right_http_connection.rb', line 101 def self.params @@params end |
.params=(params) ⇒ Object
Set the global (class-level) parameters
106 107 108 |
# File 'lib/right_http_connection.rb', line 106 def self.params=(params) @@params = params end |
Instance Method Details
#finish(reason = '') ⇒ Object
363 364 365 366 367 368 369 |
# File 'lib/right_http_connection.rb', line 363 def finish(reason = '') if @http && @http.started? reason = ", reason: '#{reason}'" unless reason.blank? @logger.info("Closing #{@http.use_ssl? ? 'HTTPS' : 'HTTP'} connection to #{@http.address}:#{@http.port}#{reason}") @http.finish end end |
#get_param(name) ⇒ Object
135 136 137 |
# File 'lib/right_http_connection.rb', line 135 def get_param(name) @params[name] || @@params[name] end |
#local_read_size=(newsize) ⇒ Object
Set the maximum size (in bytes) of a single read from local data sources like files. This can be used to tune the performance of, for example, a streaming PUT of a large buffer.
163 164 165 |
# File 'lib/right_http_connection.rb', line 163 def local_read_size=(newsize) Net::HTTPGenericRequest.local_read_size=(newsize) end |
#local_read_size? ⇒ Boolean
Query for the maximum size (in bytes) of a single read from local data sources like files. This is important, for example, in a streaming PUT of a large buffer.
156 157 158 |
# File 'lib/right_http_connection.rb', line 156 def local_read_size? Net::HTTPGenericRequest.local_read_size? end |
#request(request_params, &block) ⇒ Object
Send HTTP request to server
request_params hash:
:server => 'www.HostName.com' Hostname or IP address of HTTP server
:port => '80' Port of HTTP server
:protocol => 'https' http and https are supported on any port
:request => 'requeststring' Fully-formed HTTP request to make
Raises RuntimeError, Interrupt, and params[:exception] (if specified in new).
282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 |
# File 'lib/right_http_connection.rb', line 282 def request(request_params, &block) loop do # if we are inside a delay between retries: no requests this time! if error_count > HTTP_CONNECTION_RETRY_COUNT \ && error_time + HTTP_CONNECTION_RETRY_DELAY > Time.now @logger.warn("#{err_header} re-raising same error: #{} " + "-- error count: #{error_count}, error age: #{Time.now.to_i - error_time.to_i}") exception = get_param(:exception) || RuntimeError raise exception.new() end # try to connect server(if connection does not exist) and get response data begin request_params[:protocol] ||= (request_params[:port] == 443 ? 'https' : 'http') # (re)open connection to server if none exists or params has changed unless @http && @http.started? && @server == request_params[:server] && @port == request_params[:port] && @protocol == request_params[:protocol] start(request_params) end # get response and return it request = request_params[:request] request['User-Agent'] = get_param(:user_agent) || '' # Detect if the body is a streamable object like a file or socket. If so, stream that # bad boy. if(request.body && request.body.respond_to?(:read)) body = request.body request.content_length = body.respond_to?(:lstat) ? body.lstat.size : body.size request.body_stream = request.body end response = @http.request(request, &block) error_reset eof_reset return response # We treat EOF errors and the timeout/network errors differently. Both # are tracked in different statistics blocks. Note below that EOF # errors will sleep for a certain (exponentially increasing) period. # Other errors don't sleep because there is already an inherent delay # in them; connect and read timeouts (for example) have already # 'slept'. It is still not clear which way we should treat errors # like RST and resolution failures. For now, there is no additional # delay for these errors although this may change in the future. # EOFError means the server closed the connection on us. rescue EOFError => e @logger.debug("#{err_header} server #{@server} closed connection") @http = nil # if we have waited long enough - raise an exception... if raise_on_eof_exception? exception = get_param(:exception) || RuntimeError @logger.warn("#{err_header} raising #{exception} due to permanent EOF being received from #{@server}, error age: #{Time.now.to_i - eof_time.to_i}") raise exception.new("Permanent EOF is being received from #{@server}.") else # ... else just sleep a bit before new retry sleep(add_eof) end rescue Exception => e # See comment at bottom for the list of errors seen... @http = nil # if ctrl+c is pressed - we have to reraise exception to terminate proggy if e.is_a?(Interrupt) && !( e.is_a?(Errno::ETIMEDOUT) || e.is_a?(Timeout::Error)) @logger.debug( "#{err_header} request to server #{@server} interrupted by ctrl-c") raise elsif e.is_a?(ArgumentError) && e..include?('wrong number of arguments (5 for 4)') # seems our net_fix patch was overriden... exception = get_param(:exception) || RuntimeError raise exception.new('incompatible Net::HTTP monkey-patch') end # oops - we got a banana: log it error_add(e.) @logger.warn("#{err_header} request failure count: #{error_count}, exception: #{e.inspect}") end end end |
#socket_read_size=(newsize) ⇒ Object
Set the maximum size (in bytes) of a single read from the underlying socket. For bulk transfer, especially over fast links, this is value is critical to performance.
149 150 151 |
# File 'lib/right_http_connection.rb', line 149 def socket_read_size=(newsize) Net::BufferedIO.socket_read_size=(newsize) end |
#socket_read_size? ⇒ Boolean
Query for the maximum size (in bytes) of a single read from the underlying socket. For bulk transfer, especially over fast links, this is value is critical to performance.
142 143 144 |
# File 'lib/right_http_connection.rb', line 142 def socket_read_size? Net::BufferedIO.socket_read_size? end |