Class: RestClient::Request

Inherits:
Object
  • Object
show all
Defined in:
lib/restclient/request.rb

Overview

This class is used internally by RestClient to send the request, but you can also call it directly if you’d like to use a method not supported by the main API. For example:

RestClient::Request.execute(:method => :head, :url => 'http://example.com')

Mandatory parameters:

  • :method

  • :url

Optional parameters (have a look at ssl and/or uri for some explanations):

  • :headers a hash containing the request headers

  • :cookies may be a Hash=> String of cookie values, an

    Array<HTTP::Cookie>, or an HTTP::CookieJar containing cookies. These
will be added to a cookie jar before the request is sent.

  • :user and :password for basic auth, will be replaced by a user/password available in the :url

  • :block_response call the provided block with the HTTPResponse as parameter

  • :raw_response return a low-level RawResponse instead of a Response

  • :log Set the log for this request only, overriding RestClient.log, if

    any.

  • :stream_log_percent (Only relevant with :raw_response => true) Customize

    the interval at which download progress is logged. Defaults to every
10% complete.

  • :max_redirects maximum number of redirections (default to 10)

  • :proxy An HTTP proxy URI to use for this request. Any value here (including nil) will override RestClient.proxy.

  • :verify_ssl enable ssl verification, possible values are constants from

    OpenSSL::SSL::VERIFY_*, defaults to OpenSSL::SSL::VERIFY_PEER

  • :read_timeout and :open_timeout are how long to wait for a response and

    to open a connection, in seconds. Pass nil to disable the timeout.

  • :timeout can be used to set both timeouts

  • :ssl_client_cert, :ssl_client_key, :ssl_ca_file, :ssl_ca_path,

    :ssl_cert_store, :ssl_verify_callback, :ssl_verify_callback_warnings

  • :ssl_version specifies the SSL version for the underlying Net::HTTP connection

  • :ssl_ciphers sets SSL ciphers for the connection. See

    OpenSSL::SSL::SSLContext#ciphers=

  • :before_execution_proc a Proc to call before executing the request. This

    proc, like procs from RestClient.before_execution_procs, will be
called with the HTTP request and request params.

Constant Summary collapse

SSLOptionList = 
%w{client_cert client_key ca_file ca_path cert_store
version ciphers verify_callback verify_callback_warnings}

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(args) ⇒ Request

Returns a new instance of Request.



73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
 
# File 'lib/restclient/request.rb', line 73

def initialize args
  @method = normalize_method(args[:method])
  @headers = (args[:headers] || {}).dup
  if args[:url]
    @url = process_url_params(normalize_url(args[:url]), headers)
  else
    raise ArgumentError, "must pass :url"
  end

  @user = @password = nil
  parse_url_with_auth!(url)

  # process cookie arguments found in headers or args
  @cookie_jar = process_cookie_args!(@uri, @headers, args)

  @payload = Payload.generate(args[:payload])

  @user = args[:user] if args.include?(:user)
  @password = args[:password] if args.include?(:password)

  if args.include?(:timeout)
    @read_timeout = args[:timeout]
    @open_timeout = args[:timeout]
  end
  if args.include?(:read_timeout)
    @read_timeout = args[:read_timeout]
  end
  if args.include?(:open_timeout)
    @open_timeout = args[:open_timeout]
  end
  @block_response = args[:block_response]
  @raw_response = args[:raw_response] || false

  @stream_log_percent = args[:stream_log_percent] || 10
  if @stream_log_percent <= 0 || @stream_log_percent > 100
    raise ArgumentError.new(
      "Invalid :stream_log_percent #{@stream_log_percent.inspect}")
  end

  @proxy = args.fetch(:proxy) if args.include?(:proxy)

  @ssl_opts = {}

  if args.include?(:verify_ssl)
    v_ssl = args.fetch(:verify_ssl)
    if v_ssl
      if v_ssl == true
        # interpret :verify_ssl => true as VERIFY_PEER
        @ssl_opts[:verify_ssl] = OpenSSL::SSL::VERIFY_PEER
      else
        # otherwise pass through any truthy values
        @ssl_opts[:verify_ssl] = v_ssl
      end
    else
      # interpret all falsy :verify_ssl values as VERIFY_NONE
      @ssl_opts[:verify_ssl] = OpenSSL::SSL::VERIFY_NONE
    end
  else
    # if :verify_ssl was not passed, default to VERIFY_PEER
    @ssl_opts[:verify_ssl] = OpenSSL::SSL::VERIFY_PEER
  end

  SSLOptionList.each do |key|
    source_key = ('ssl_' + key).to_sym
    if args.has_key?(source_key)
      @ssl_opts[key.to_sym] = args.fetch(source_key)
    end
  end

  # Set some other default SSL options, but only if we have an HTTPS URI.
  if use_ssl?

    # If there's no CA file, CA path, or cert store provided, use default
    if !ssl_ca_file && !ssl_ca_path && !@ssl_opts.include?(:cert_store)
      @ssl_opts[:cert_store] = self.class.default_ssl_cert_store
    end
  end

  @log = args[:log]
  @max_redirects = args[:max_redirects] || 10
  @processed_headers = make_headers headers
  @processed_headers_lowercase = Hash[@processed_headers.map {|k, v| [k.downcase, v]}]
  @args = args

  @before_execution_proc = args[:before_execution_proc]
end

Instance Attribute Details

#args ⇒ Object (readonly)

Returns the value of attribute args.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def args
  @args
end

#headers ⇒ Object (readonly)

Returns the value of attribute headers.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def headers
  @headers
end

#max_redirects ⇒ Object (readonly)

Returns the value of attribute max_redirects.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def max_redirects
  @max_redirects
end

#method ⇒ Object (readonly)

Returns the value of attribute method.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def method
  @method
end

#open_timeout ⇒ Object (readonly)

Returns the value of attribute open_timeout.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def open_timeout
  @open_timeout
end

#password ⇒ Object (readonly)

Returns the value of attribute password.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def password
  @password
end

#payload ⇒ Object (readonly)

Returns the value of attribute payload.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def payload
  @payload
end

#processed_headers ⇒ Object (readonly)

Returns the value of attribute processed_headers.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def processed_headers
  @processed_headers
end

#proxy ⇒ Object (readonly)

Returns the value of attribute proxy.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def proxy
  @proxy
end

#raw_response ⇒ Object (readonly)

Returns the value of attribute raw_response.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def raw_response
  @raw_response
end

#read_timeout ⇒ Object (readonly)

Returns the value of attribute read_timeout.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def read_timeout
  @read_timeout
end

#redirection_history ⇒ Object

An array of previous redirection responses



60
61
62
 
# File 'lib/restclient/request.rb', line 60

def redirection_history
  @redirection_history
end

#ssl_opts ⇒ Object (readonly)

Returns the value of attribute ssl_opts.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def ssl_opts
  @ssl_opts
end

#uri ⇒ Object (readonly)

Returns the value of attribute uri.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def uri
  @uri
end

#url ⇒ Object (readonly)

Returns the value of attribute url.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def url
  @url
end

#user ⇒ Object (readonly)

Returns the value of attribute user.



54
55
56
 
# File 'lib/restclient/request.rb', line 54

def user
  @user
end

Class Method Details

.default_ssl_cert_store ⇒ OpenSSL::X509::Store

Return a certificate store that can be used to validate certificates with the system certificate authorities. This will probably not do anything on OS X, which monkey patches OpenSSL in terrible ways to insert its own validation. On most *nix platforms, this will add the system certifcates using OpenSSL::X509::Store#set_default_paths. On Windows, this will use RestClient::Windows::RootCerts to look up the CAs trusted by the system.

Returns:

  • (OpenSSL::X509::Store) 


501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
 
# File 'lib/restclient/request.rb', line 501

def self.default_ssl_cert_store
  cert_store = OpenSSL::X509::Store.new
  cert_store.set_default_paths

  # set_default_paths() doesn't do anything on Windows, so look up
  # certificates using the win32 API.
  if RestClient::Platform.windows?
    RestClient::Windows::RootCerts.instance.to_a.uniq.each do |cert|
      begin
        cert_store.add_cert(cert)
      rescue OpenSSL::X509::StoreError => err
        # ignore duplicate certs
        raise unless err.message == 'cert already in hash table'
      end
    end
  end

  cert_store
end

.execute(args, &block) ⇒ Object 



62
63
64
 
# File 'lib/restclient/request.rb', line 62

def self.execute(args, & block)
  new(args).execute(& block)
end

Instance Method Details

Returns:

  • (HTTP::CookieJar) 


251
252
253
 
# File 'lib/restclient/request.rb', line 251

def cookie_jar
  @cookie_jar
end

#cookies ⇒ Hash

Render a hash of key => value pairs for cookies in the Request#cookie_jar that are valid for the Request#uri. This will not necessarily include all cookies if there are duplicate keys. It’s safer to use the cookie_jar directly if that’s a concern.

Returns:

  • (Hash)

See Also:



240
241
242
243
244
245
246
247
248
 
# File 'lib/restclient/request.rb', line 240

def cookies
  hash = {}

  @cookie_jar.cookies(uri).each do |c|
    hash[c.name] = c.value
  end

  hash
end

#default_headers ⇒ Hash<Symbol, String>

Default headers set by RestClient. In addition to these headers, servers will receive headers set by Net::HTTP, such as Accept-Encoding and Host.

Returns:

  • (Hash<Symbol, String>) 


586
587
588
589
590
591
 
# File 'lib/restclient/request.rb', line 586

def default_headers
  {
    :accept => '*/*',
    :user_agent => RestClient::Platform.default_user_agent,
  }
end

#execute(&block) ⇒ Object 



160
161
162
163
164
165
166
 
# File 'lib/restclient/request.rb', line 160

def execute & block
  # With 2.0.0+, net/http accepts URI objects in requests and handles wrapping
  # IPv6 addresses in [] for use in the Host request header.
  transmit uri, net_http_request_class(method).new(uri, processed_headers), payload, & block
ensure
  payload.close if payload
end

#inspect ⇒ Object 



69
70
71
 
# File 'lib/restclient/request.rb', line 69

def inspect
  "<RestClient::Request @method=#{@method.inspect}, @url=#{@url.inspect}>"
end

#log ⇒ Object

Default to the global logger if there’s not a request-specific one



536
537
538
 
# File 'lib/restclient/request.rb', line 536

def log
  @log || RestClient.log
end

#log_request ⇒ Object 



540
541
542
543
544
545
546
547
548
549
 
# File 'lib/restclient/request.rb', line 540

def log_request
  return unless log

  out = []

  out << "RestClient.#{method} #{redacted_url.inspect}"
  out << payload.short_inspect if payload
  out << processed_headers.to_a.sort.map { |(k, v)| [k.inspect, v.inspect].join("=>") }.join(", ")
  log << out.join(', ') + "\n"
end

#make_cookie_header ⇒ String?

Render a Cookie HTTP request header from the contents of the @cookie_jar, or nil if the jar is empty.

Returns:

  • (String, nil)

See Also:



262
263
264
265
266
267
268
269
 
# File 'lib/restclient/request.rb', line 262

def make_cookie_header
  return nil if cookie_jar.nil?

  arr = cookie_jar.cookies(url)
  return nil if arr.empty?

  return HTTP::Cookie.cookie_value(arr)
end

#make_headers(user_headers) ⇒ Hash<String, String>

Generate headers for use by a request. Header keys will be stringified using ‘#stringify_headers` to normalize them as capitalized strings.

The final headers consist of:

- default headers from #default_headers
- user_headers provided here
- headers from the payload object (e.g. Content-Type, Content-Lenth)
- cookie headers from #make_cookie_header

BUG: stringify_headers does not alter the capitalization of headers that are passed as strings, it only normalizes those passed as symbols. This behavior will probably remain for a while for compatibility, but it means that the warnings that attempt to detect accidental header overrides may not always work. github.com/rest-client/rest-client/issues/599

Parameters:

  • user_headers (Hash) —

    User-provided headers to include

Returns:

  • (Hash<String, String>) —

    A hash of HTTP headers => values



388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
 
# File 'lib/restclient/request.rb', line 388

def make_headers(user_headers)
  headers = stringify_headers(default_headers).merge(stringify_headers(user_headers))

  # override headers from the payload (e.g. Content-Type, Content-Length)
  if @payload
    payload_headers = @payload.headers

    # Warn the user if we override any headers that were previously
    # present. This usually indicates that rest-client was passed
    # conflicting information, e.g. if it was asked to render a payload as
    # x-www-form-urlencoded but a Content-Type application/json was
    # also supplied by the user.
    payload_headers.each_pair do |key, val|
      if headers.include?(key) && headers[key] != val
        warn("warning: Overriding #{key.inspect} header " +
             "#{headers.fetch(key).inspect} with #{val.inspect} " +
             "due to payload")
      end
    end

    headers.merge!(payload_headers)
  end

  # merge in cookies
  cookies = make_cookie_header
  if cookies && !cookies.empty?
    if headers['Cookie']
      warn('warning: overriding "Cookie" header with :cookies option')
    end
    headers['Cookie'] = cookies
  end

  headers
end

#net_http_do_request(http, req, body = nil, &block) ⇒ Object 



468
469
470
471
472
473
474
475
 
# File 'lib/restclient/request.rb', line 468

def net_http_do_request(http, req, body=nil, &block)
  if body && body.respond_to?(:read)
    req.body_stream = body
    return http.request(req, nil, &block)
  else
    return http.request(req, body, &block)
  end
end

#net_http_object(hostname, port) ⇒ Object 



448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
 
# File 'lib/restclient/request.rb', line 448

def net_http_object(hostname, port)
  p_uri = proxy_uri

  if p_uri.nil?
    # no proxy set
    Net::HTTP.new(hostname, port)
  elsif !p_uri
    # proxy explicitly set to none
    Net::HTTP.new(hostname, port, nil, nil, nil, nil)
  else
    Net::HTTP.new(hostname, port,
                  p_uri.hostname, p_uri.port, p_uri.user, p_uri.password)

  end
end

#net_http_request_class(method) ⇒ Object 



464
465
466
 
# File 'lib/restclient/request.rb', line 464

def net_http_request_class(method)
  Net::HTTP.const_get(method.capitalize, false)
end

#normalize_url(url) ⇒ String

Normalize a URL by adding a protocol if none is present.

If the string has no HTTP-like scheme (i.e. scheme followed by ‘//’), a scheme of ‘http’ will be added. This mimics the behavior of browsers and user agents like cURL.

Parameters:

  • url (String) —

    A URL string.

Returns:

  • (String) 


487
488
489
490
 
# File 'lib/restclient/request.rb', line 487

def normalize_url(url)
  url = 'http://' + url unless url.match(%r{\A[a-z][a-z0-9+.-]*://}i)
  url
end

#process_cookie_args!(uri, headers, args) ⇒ HTTP::CookieJar

Process cookies passed as hash or as HTTP::CookieJar. For backwards compatibility, these may be passed as a :cookies option masquerading inside the headers hash. To avoid confusion, if :cookies is passed in both headers and Request#initialize, raise an error.

:cookies may be a:

  • Hash=> String

  • Array<HTTP::Cookie>

  • HTTP::CookieJar

Passing as a hash:

Keys may be symbols or strings. Values must be strings.
Infer the domain name from the request URI and allow subdomains (as
though '.example.com' had been set in a Set-Cookie header). Assume a
path of '/'.

  RestClient::Request.new(url: 'http://example.com', method: :get,
    :cookies => {:foo => 'Value', 'bar' => '123'}
  )

results in cookies as though set from the server by:

Set-Cookie: foo=Value; Domain=.example.com; Path=/
Set-Cookie: bar=123; Domain=.example.com; Path=/

which yields a client cookie header of:

Cookie: foo=Value; bar=123

Passing as HTTP::CookieJar, which will be passed through directly:

jar = HTTP::CookieJar.new
jar.add(HTTP::Cookie.new('foo', 'Value', domain: 'example.com',
                         path: '/', for_domain: false))

RestClient::Request.new(..., :cookies => jar)

infer the domain name for cookies passed as strings in a hash. To avoid this implicit behavior, pass a full cookie jar or use HTTP::Cookie hash values.

Parameters:

  • uri (URI::HTTP) —

    The URI for the request. This will be used to

  • headers (Hash) —

    The headers hash from which to pull the :cookies option. MUTATION NOTE: This key will be deleted from the hash if present.

  • args (Hash) —

    The options passed to Request#initialize. This hash will be used as another potential source for the :cookies key. These args will not be mutated.

Returns:

  • (HTTP::CookieJar) —

    A cookie jar containing the parsed cookies.



319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
 
# File 'lib/restclient/request.rb', line 319

def process_cookie_args!(uri, headers, args)

  # Avoid ambiguity in whether options from headers or options from
  # Request#initialize should take precedence by raising ArgumentError when
  # both are present. Prior versions of rest-client claimed to give
  # precedence to init options, but actually gave precedence to headers.
  # Avoid that mess by erroring out instead.
  if headers[:cookies] && args[:cookies]
    raise ArgumentError.new(
      "Cannot pass :cookies in Request.new() and in headers hash")
  end

  cookies_data = headers.delete(:cookies) || args[:cookies]

  # return copy of cookie jar as is
  if cookies_data.is_a?(HTTP::CookieJar)
    return cookies_data.dup
  end

  # convert cookies hash into a CookieJar
  jar = HTTP::CookieJar.new

  (cookies_data || []).each do |key, val|

    # Support for Array<HTTP::Cookie> mode:
    # If key is a cookie object, add it to the jar directly and assert that
    # there is no separate val.
    if key.is_a?(HTTP::Cookie)
      if val
        raise ArgumentError.new("extra cookie val: #{val.inspect}")
      end

      jar.add(key)
      next
    end

    if key.is_a?(Symbol)
      key = key.to_s
    end

    # assume implicit domain from the request URI, and set for_domain to
    # permit subdomains
    jar.add(HTTP::Cookie.new(key, val, domain: uri.hostname.downcase,
                             path: '/', for_domain: true))
  end

  jar
end

#process_url_params(url, headers) ⇒ String

Extract the query parameters and append them to the url

Look through the headers hash for a :params option (case-insensitive, may be string or symbol). If present and the value is a Hash or RestClient::ParamsArray, delete the key/value pair from the headers hash and encode the value into a query string. Append this query string to the URL and return the resulting URL.

Parameters:

  • url (String)
  • headers (Hash) —

    An options/headers hash to process. Mutation warning: the params key may be removed if present!

Returns:

  • (String) —

    resulting url with query string



200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
 
# File 'lib/restclient/request.rb', line 200

def process_url_params(url, headers)
  url_params = nil

  # find and extract/remove "params" key if the value is a Hash/ParamsArray
  headers.delete_if do |key, value|
    if key.to_s.downcase == 'params' &&
        (value.is_a?(Hash) || value.is_a?(RestClient::ParamsArray))
      if url_params
        raise ArgumentError.new("Multiple 'params' options passed")
      end
      url_params = value
      true
    else
      false
    end
  end

  # build resulting URL with query string
  if url_params && !url_params.empty?
    query_string = RestClient::Utils.encode_query_string(url_params)

    if url.include?('?')
      url + '&' + query_string
    else
      url + '?' + query_string
    end
  else
    url
  end
end

#proxy_uri ⇒ URI, ...

The proxy URI for this request. If ‘:proxy` was provided on this request, use it over `RestClient.proxy`.

Return false if a proxy was explicitly set and is falsy.

Returns:

  • (URI, false, nil) 


430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
 
# File 'lib/restclient/request.rb', line 430

def proxy_uri
  if defined?(@proxy)
    if @proxy
      URI.parse(@proxy)
    else
      false
    end
  elsif RestClient.proxy_set?
    if RestClient.proxy
      URI.parse(RestClient.proxy)
    else
      false
    end
  else
    nil
  end
end

#redacted_uri ⇒ Object 



521
522
523
524
525
526
527
528
529
 
# File 'lib/restclient/request.rb', line 521

def redacted_uri
  if uri.password
    sanitized_uri = uri.dup
    sanitized_uri.password = 'REDACTED'
    sanitized_uri
  else
    uri
  end
end

#redacted_url ⇒ Object 



531
532
533
 
# File 'lib/restclient/request.rb', line 531

def redacted_url
  redacted_uri.to_s
end

#stringify_headers(headers) ⇒ Object

Return a hash of headers whose keys are capitalized strings

BUG: stringify_headers does not fix the capitalization of headers that are already Strings. Leaving this behavior as is for now for backwards compatibility. github.com/rest-client/rest-client/issues/599



558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
 
# File 'lib/restclient/request.rb', line 558

def stringify_headers headers
  headers.inject({}) do |result, (key, value)|
    if key.is_a? Symbol
      key = key.to_s.split(/_/).map(&:capitalize).join('-')
    end
    if 'CONTENT-TYPE' == key.upcase
      result[key] = maybe_convert_extension(value.to_s)
    elsif 'ACCEPT' == key.upcase
      # Accept can be composed of several comma-separated values
      if value.is_a? Array
        target_values = value
      else
        target_values = value.to_s.split ','
      end
      result[key] = target_values.map { |ext|
        maybe_convert_extension(ext.to_s.strip)
      }.join(', ')
    else
      result[key] = value.to_s
    end
    result
  end
end

#use_ssl? ⇒ Boolean

Return true if the request URI will use HTTPS.

Returns:

  • (Boolean) 


182
183
184
 
# File 'lib/restclient/request.rb', line 182

def use_ssl?
  uri.is_a?(URI::HTTPS)
end

#verify_ssl ⇒ Object

SSL-related options



169
170
171
 
# File 'lib/restclient/request.rb', line 169

def verify_ssl
  @ssl_opts.fetch(:verify_ssl)
end