Class: RFuzz::HttpClient

Inherits:
Object
  • Object
show all
Includes:
HttpEncoding
Defined in:
lib/rfuzz/client.rb

Overview

The actual HttpClient that does the work with the thinnest layer between you and the protocol. All exceptions and leaks are allowed to pass through since those are important when testing. It doesn't pretend to be a full client, but instead is just enough client to track cookies, form proper HTTP requests, and return HttpResponse hashes with the results.

It's designed so that you create one client, and then you work it with a minimum of parameters as you need. The initialize method lets you pass in defaults for most of the parameters you'll need, and you can simple call the method you want and it'll be translated to an HTTP method (client.get => GET, client.foobar = FOOBAR).

Here's a few examples:

client = HttpClient.new(:head => {"X-DefaultHeader" => "ONE"})
resp = client.post("/test")
resp = client.post("/test", :head => {"X-TestSend" => "Status"}, :body => "TEST BODY")
resp = client.put("/testput", :query => {"q" => "test"}, :body => "SOME JUNK")
client.reset

The HttpClient.reset call clears cookies that are maintained.

It uses method_missing to do the translation of .put to "PUT /testput HTTP/1.1" so you can get into trouble if you're calling unknown methods on it. By default the methods are PUT, GET, POST, DELETE, HEAD. You can change the allowed methods by passing :allowed_methods => [:put, :get, ..] to the initialize for the object.

Notifications

You can register a "notifier" with the client that will get called when different events happen. Right now the Notifier class just has a few functions for the common parts of an HTTP request that each take a symbol and some extra parameters. See RFuzz::Notifier for more information.

Parameters

:head => {K => V}  or {K => [V1,V2]}
:query => {K => V} or {K => [V1,V2]}
:body => "some body" (you must encode for now)
:cookies => {K => V} or {K => [V1, V2]}
:allowed_methods => [:put, :get, :post, :delete, :head]
:notifier => Notifier.new
:redirect => false (give it a number and it'll follow redirects for that count)

Constant Summary collapse

TRANSFER_ENCODING = 
"TRANSFER_ENCODING"
CONTENT_LENGTH = 
"CONTENT_LENGTH"
"SET_COOKIE"
LOCATION = 
"LOCATION"
HOST = 
"HOST"
HTTP_REQUEST_HEADER = 
"%s %s HTTP/1.1\r\n"

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods included from HttpEncoding

#encode_cookies, #encode_field, #encode_headers, #encode_host, #encode_param, #encode_query, #escape, #query_parse, #unescape

Constructor Details

#initialize(host, port, options = {}) ⇒ HttpClient

Doesn't make the connect until you actually call a .put,.get, etc.



210
211
212
213
214
215
216
217
218
 
# File 'lib/rfuzz/client.rb', line 210

def initialize(host, port, options = {})
  @options = options
  @host = host
  @port = port
  @cookies = options[:cookies] || {}
  @allowed_methods = options[:allowed_methods] || [:put, :get, :post, :delete, :head]
  @notifier = options[:notifier]
  @redirect = options[:redirect] || false
end

Dynamic Method Handling

This class handles dynamic methods through the method_missing method

#method_missing(symbol, *args) ⇒ Object

Translates unknown function calls into PUT, GET, POST, DELETE, HEAD methods. The allowed HTTP methods allowed are restricted by the during construction with :allowed_methods => [:put, :get, ...]



396
397
398
399
400
401
402
403
404
405
406
 
# File 'lib/rfuzz/client.rb', line 396

def method_missing(symbol, *args)
  if @allowed_methods.include? symbol
    method = symbol.to_s.upcase
    resp = send_request(method, args[0], args[1] || {})
    resp = redirect(symbol, resp) if @redirect

    return resp
  else
    raise "Invalid method: #{symbol}"
  end
end

Instance Attribute Details

#allowed_methods ⇒ Object

Access to the host, port, default options, and cookies currently in play



207
208
209
 
# File 'lib/rfuzz/client.rb', line 207

def allowed_methods
  @allowed_methods
end

#cookies ⇒ Object

Access to the host, port, default options, and cookies currently in play



207
208
209
 
# File 'lib/rfuzz/client.rb', line 207

def cookies
  @cookies
end

#host ⇒ Object

Access to the host, port, default options, and cookies currently in play



207
208
209
 
# File 'lib/rfuzz/client.rb', line 207

def host
  @host
end

#notifier ⇒ Object

Access to the host, port, default options, and cookies currently in play



207
208
209
 
# File 'lib/rfuzz/client.rb', line 207

def notifier
  @notifier
end

#options ⇒ Object

Access to the host, port, default options, and cookies currently in play



207
208
209
 
# File 'lib/rfuzz/client.rb', line 207

def options
  @options
end

#port ⇒ Object

Access to the host, port, default options, and cookies currently in play



207
208
209
 
# File 'lib/rfuzz/client.rb', line 207

def port
  @port
end

Instance Method Details

#build_request(out, method, uri, req) ⇒ Object

Builds a full request from the method, uri, req, and @cookies using the default @options and writes it to out (should be an IO).

It returns the body that the caller should use (based on defaults resolution).



226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
 
# File 'lib/rfuzz/client.rb', line 226

def build_request(out, method, uri, req)
  ops = @options.merge(req)
  query = ops[:query]

  # merge head differently since that's typically what they mean
  head = req[:head] || {}
  head = ops[:head].merge(head) if ops[:head]

  # setup basic headers we always need
  head[HOST] = encode_host(@host,@port)
  head[CONTENT_LENGTH] = ops[:body] ? ops[:body].length : 0

  # blast it out
  out.write(HTTP_REQUEST_HEADER % [method, encode_query(uri,query)])
  out.write(encode_headers(head))
  out.write(encode_cookies(@cookies.merge(req[:cookies] || {})))
  out.write("\r\n")
  ops[:body] || ""
end

#notify(event) ⇒ Object

Sends the notifications to the registered notifier, taking a block that it runs doing the :begins, :ends states around it.

It also catches errors transparently in order to call the notifier when an attempt fails.



442
443
444
445
446
447
448
449
450
451
452
 
# File 'lib/rfuzz/client.rb', line 442

def notify(event)
  @notifier.send(event, :begins) if @notifier

  begin
    yield
    @notifier.send(event, :ends) if @notifier
  rescue Object
    @notifier.send(event, :error) if @notifier
    raise $!
  end
end

#read_chunked_encoding(resp, sock, parser) ⇒ Object 



281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
 
# File 'lib/rfuzz/client.rb', line 281

def read_chunked_encoding(resp, sock, parser)
  out = StringIO.new
  input = StringIO.new(resp.http_body)

  # read from the http body first, then continue at the socket
  status, result = read_chunks(input, out, parser)

  case status
  when :incomplete_trailer
    if result.nil?
      sock.read(2)
    else
      sock.read(result.length - 2)
    end
  when :incomplete_body
    out.write(sock.read(result))  # read the remaining
    sock.read(2)
  when :incomplete_header
    # push what we read back onto the socket, but backwards
    result.reverse!
    result.each_byte {|b| sock.ungetc(b) }
  when :finished
    # all done, get out
    out.rewind; return out.read
  when :eof_error
    # read everything we could, ignore
  end

  # then continue reading them from the socket
  status, result = read_chunks(sock, out, parser)

  # and now the http_body is the chunk
  out.rewind; return out.read
end

#read_chunks(input, out, parser) ⇒ Object 



246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
 
# File 'lib/rfuzz/client.rb', line 246

def read_chunks(input, out, parser)
  begin
    until input.closed?
      parser.reset
      chunk = HttpResponse.new
      line = input.readline("\r\n")
      nread = parser.execute(chunk, line, 0)

      if !parser.finished?
        # tried to read this header but couldn't
        return :incomplete_header, line
      end

      size = chunk.http_chunk_size ? chunk.http_chunk_size.to_i(base=16) : 0

      if size == 0
        return :finished, nil
      end
      remain =  size -out.write(input.read(size))
      return :incomplete_body, remain if remain > 0

      line = input.read(2)
      if line.nil? or line.length < 2
        return :incomplete_trailer, line
      elsif line != "\r\n"
        raise HttpClientParserError.new("invalid chunked encoding trailer")
      end
    end
  rescue EOFError
    # this is thrown when the header read is attempted and 
    # there's nothing in the buffer
    return :eof_error, nil
  end
end

#read_response(sock) ⇒ Object

Reads an HTTP response from the given socket. It uses readpartial which only appeared in Ruby 1.8.4. The result is a fully formed HttpResponse object for you to play with.

As with other methods in this class it doesn't stop any exceptions from reaching your code. It's for experts who want these exceptions so either write a wrapper, use net/http, or deal with it on your end.



323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
 
# File 'lib/rfuzz/client.rb', line 323

def read_response(sock)
  data, resp = nil, nil
  parser = HttpClientParser.new
  resp = HttpResponse.new

  notify :read_header do
    data = sock.readpartial(1024)
    nread = parser.execute(resp, data, 0)

    while not parser.finished?
      data += sock.readpartial(1024)
      nread += parser.execute(resp, data, nread)
    end
  end

  notify :read_body do
    if resp[TRANSFER_ENCODING] and resp[TRANSFER_ENCODING].index("chunked")
      resp.http_body = read_chunked_encoding(resp, sock, parser)
    elsif resp[CONTENT_LENGTH]
      cl = resp[CONTENT_LENGTH].to_i
      if cl - resp.http_body.length > 0
        resp.http_body += sock.read(cl - resp.http_body.length)
      elsif cl < resp.http_body.length
        STDERR.puts "Web site sucks, they said Content-Length: #{cl}, but sent a longer body length: #{resp.http_body.length}"
      end
    else
      resp.http_body += sock.read
    end
  end

  if resp[SET_COOKIE]
    cookies = query_parse(resp[SET_COOKIE], ';,')
    @cookies.merge! cookies
    @cookies.delete "path"
  end

  notify :close do
    sock.close
  end

  resp
end

#redirect(method, resp, *args) ⇒ Object

Keeps doing requests until it doesn't receive a 3XX request.



409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
 
# File 'lib/rfuzz/client.rb', line 409

def redirect(method, resp, *args)
  @redirect.times do
    break if resp.http_status.index("3") != 0

    host = encode_host(@host,@port)
    location = resp[LOCATION]

    if location.index(host) == 0
      # begins with the host so strip that off
      location = location[host.length .. -1]
    end

    @notifier.redirect(:begins) if @notifier
    resp = self.send(method, location, *args)
    @notifier.redirect(:ends) if @notifier
  end

  return resp
end

#reset ⇒ Object

Clears out the cookies in use so far in order to get a clean slate.



431
432
433
 
# File 'lib/rfuzz/client.rb', line 431

def reset
  @cookies.clear
end

#send_request(method, uri, req) ⇒ Object

Does the socket connect and then build_request, read_response calls finally returning the result.



368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
 
# File 'lib/rfuzz/client.rb', line 368

def send_request(method, uri, req)
  begin
    sock = nil
    notify :connect do
      sock = TCPSocket.new(@host, @port)
    end

    out = StringIO.new
    body = build_request(out, method, uri, req)

    notify :send_request do
      sock.write(out.string + body)
      sock.flush
    end

    return read_response(sock)
  rescue Object
    raise $!
  ensure
    sock.close unless (!sock or sock.closed?)
  end
end