Class: FTW::Agent
- Inherits:
-
Object
- Object
- FTW::Agent
- Includes:
- Configuration, Protocol
- Defined in:
- lib/ftw/agent.rb
Overview
This should act as a proper web agent.
-
Reuse connections.
-
SSL/TLS.
-
HTTP Upgrade support.
-
HTTP 1.1 (RFC2616).
-
WebSockets (RFC6455).
-
Support Cookies.
All standard HTTP methods defined by RFC2616 are available as methods on this agent: get, head, put, etc.
Example:
agent = FTW::Agent.new
request = agent.get("http://www.google.com/")
response = agent.execute(request)
puts response.body.read
For any standard http method (like ‘get’) you can invoke it with ‘!’ on the end and it will execute and return a FTW::Response object:
agent = FTW::Agent.new
response = agent.get!("http://www.google.com/")
puts response.body.head
TODO(sissel): TBD: implement cookies… delicious chocolate chip cookies.
Defined Under Namespace
Modules: Configuration Classes: TooManyRedirects
Constant Summary collapse
- STANDARD_METHODS =
List of standard HTTP methods described in RFC2616
%w(options get head post put delete trace connect)
Constants included from Configuration
Configuration::REDIRECTION_LIMIT, Configuration::SSL_TRUST_STORE
Constants included from CRLF
Instance Method Summary collapse
-
#execute(request) ⇒ FTW::Response
Execute a FTW::Request in this Agent.
-
#initialize ⇒ Agent
constructor
A new instance of Agent.
-
#request(method, uri, options) ⇒ Object
Build a request.
-
#shutdown ⇒ Object
shutdown this agent.
-
#upgrade!(uri, protocol, options = {}) ⇒ Object
Send the request as an HTTP upgrade.
-
#websocket!(uri, options = {}) ⇒ Object
Make a new websocket connection.
Methods included from Configuration
Methods included from Protocol
#encode_chunked, #read_body, #read_http_body, #read_http_body_chunked, #read_http_body_length, #read_http_message, #write_http_body, #write_http_body_chunked, #write_http_body_normal
Constructor Details
#initialize ⇒ Agent
Returns a new instance of Agent.
60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 |
# File 'lib/ftw/agent.rb', line 60 def initialize @pool = FTW::Pool.new @logger = Cabin::Channel.get configuration[REDIRECTION_LIMIT] = 20 @certificate_store = OpenSSL::X509::Store.new if File.readable?(OpenSSL::X509::DEFAULT_CERT_FILE) @logger.debug("Adding default certificate file", :path => OpenSSL::X509::DEFAULT_CERT_FILE) @certificate_store.add_file(OpenSSL::X509::DEFAULT_CERT_FILE) else # Use some better defaults from http://curl.haxx.se/docs/caextract.html @logger.info("Using upstream ssl certs, possibly untrusty.") default_ca = File.join(File.dirname(__FILE__), "cacert.pem") # JRUBY-6870 - strip 'jar:' prefix if it is present. if default_ca =~ /^jar:file.*!/ default_ca.gsub!(/^jar:/, "") end @certificate_store.add_file(default_ca) end # Handle the local user/app trust store as well. if File.directory?(configuration[SSL_TRUST_STORE]) # This is a directory, so use add_path @logger.debug("Adding SSL_TRUST_STORE", :path => configuration[SSL_TRUST_STORE]) @certificate_store.add_path(configuration[SSL_TRUST_STORE]) end # TODO(sissel): Add custom paths for ssl certs end |
Instance Method Details
#execute(request) ⇒ FTW::Response
Execute a FTW::Request in this Agent.
If an existing, idle connection is already open to the target server of this Request, it will be reused. Otherwise, a new connection is opened.
Redirects are always followed.
302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 |
# File 'lib/ftw/agent.rb', line 302 def execute(request) # TODO(sissel): Make redirection-following optional, but default. connection, error = connect(request.headers["Host"], request.port, request.protocol == "https") if !error.nil? p :error => error raise error end response = request.execute(connection) redirects = 0 # Follow redirects while response.redirect? and response.headers.include?("Location") # RFC2616 section 10.3.3 indicates HEAD redirects must not include a # body. Otherwise, the redirect response can have a body, so let's # throw it away. if request.method == "HEAD" # Head requests have no body connection.release elsif response.content? # Throw away the body response.body = connection # read_body will consume the body and release this connection response.read_http_body { |chunk| } end # TODO(sissel): If this response has any cookies, store them in the # agent's cookie store redirects += 1 if redirects > configuration[REDIRECTION_LIMIT] # TODO(sissel): include original a useful debugging information like # the trace of redirections, etc. raise TooManyRedirects.new("Redirect more than " \ "#{configuration[REDIRECTION_LIMIT]} times, aborting.", response) # I don't like this api from FTW::Agent. I think 'get' and other methods # should return (object, error), and if there's an error end @logger.debug("Redirecting", :location => response.headers["Location"]) request.use_uri(response.headers["Location"]) connection, error = connect(request.headers["Host"], request.port, request.protocol == "https") # TODO(sissel): Do better error handling than raising. if !error.nil? p :error => error raise error end response = request.execute(connection) end # while being redirected # RFC 2616 section 9.4, HEAD requests MUST NOT have a message body. if request.method != "HEAD" response.body = connection else connection.release end # TODO(sissel): If this response has any cookies, store them in the # agent's cookie store return response end |
#request(method, uri, options) ⇒ Object
Build a request. Returns a FTW::Request object.
Arguments:
-
method - the http method
-
uri - the URI to make the request to
-
options - a hash of options
uri can be a valid url or an Addressable::URI object. The uri will be used to choose the host/port to connect to. It also sets the protocol (https, etc). Further, it will set the ‘Host’ header.
The ‘options’ hash supports the following keys:
-
:headers => { string => string, … }. This allows you to set header values.
273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 |
# File 'lib/ftw/agent.rb', line 273 def request(method, uri, ) @logger.info("Creating new request", :method => method, :uri => uri, :options => ) request = FTW::Request.new(uri) request.method = method request.headers.add("Connection", "keep-alive") if .include?(:headers) [:headers].each do |key, value| request.headers.add(key, value) end end if .include?(:body) request.body = [:body] end return request end |
#shutdown ⇒ Object
shutdown this agent.
This will shutdown all active connections.
368 369 370 371 372 373 374 |
# File 'lib/ftw/agent.rb', line 368 def shutdown @pool.each do |identifier, list| list.each do |connection| connection.disconnect("stopping agent") end end end |
#upgrade!(uri, protocol, options = {}) ⇒ Object
Send the request as an HTTP upgrade.
Returns the response and the FTW::Connection for this connection. If the upgrade was denied, the connection returned will be nil.
213 214 215 216 217 218 219 220 221 222 223 224 |
# File 'lib/ftw/agent.rb', line 213 def upgrade!(uri, protocol, ={}) req = request("GET", uri, ) req.headers["Connection"] = "Upgrade" req.headers["Upgrade"] = protocol response = execute(req) if response.status == 101 # Success, return the response object and the connection to hand off. return response, response.body else return response, nil end end |
#websocket!(uri, options = {}) ⇒ Object
Make a new websocket connection.
This will send the http request. If the websocket handshake is successful, a FTW::WebSocket instance will be returned. Otherwise, a FTW::Response will be returned.
See #request for what the ‘uri’ and ‘options’ parameters should be.
233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 |
# File 'lib/ftw/agent.rb', line 233 def websocket!(uri, ={}) # TODO(sissel): Use FTW::Agent#upgrade! ? req = request("GET", uri, ) ws = FTW::WebSocket.new(req) response = execute(req) if ws.handshake_ok?(response) # response.body is a FTW::Connection ws.connection = response.body # There seems to be a bug in http_parser.rb where websocket responses # lead with a newline for some reason. It's like the header terminator # CRLF still has the LF character left in the buffer. Work around it. data = response.body.read if data[0] == "\n" response.body.pushback(data[1..-1]) else response.body.pushback(data) end return ws else return response end end |