Class: Typhoeus::Hydra

Inherits:
Object
  • Object
show all
Includes:
Addable, Before, BlockConnection, Cacheable, Memoizable, Queueable, Runnable, Stubbable
Defined in:
lib/typhoeus/hydra.rb,
lib/typhoeus/hydra/before.rb,
lib/typhoeus/hydra/addable.rb,
lib/typhoeus/hydra/runnable.rb,
lib/typhoeus/hydra/cacheable.rb,
lib/typhoeus/hydra/queueable.rb,
lib/typhoeus/hydra/stubbable.rb,
lib/typhoeus/hydra/memoizable.rb,
lib/typhoeus/hydra/block_connection.rb

Overview

Note:

Callbacks are going to delay the request execution.

Hydra manages making parallel HTTP requests. This is achieved by using libcurls multi interface: curl.haxx.se/libcurl/c/libcurl-multi.html The benefits are that you don’t have to worry running the requests by yourself.

Hydra will also handle how many requests you can make in parallel. Things will get flakey if you try to make too many requests at the same time. The built in limit is 200. When more requests than that are queued up, hydra will save them for later and start the requests as others are finished. You can raise or lower the concurrency limit through the Hydra constructor.

Regarding the asynchronous behavior of the hydra, it is important to know that this is completely hidden from the developer and you are free to apply whatever technique you want to your code. That should not conflict with libcurls internal concurrency mechanism.

Examples:

Use the hydra to do multiple requests.

hydra = Typhoeus::Hydra.new
requests = (0..9).map{ Typhoeus::Request.new("www.example.com") }
requests.each{ |request| hydra.queue(request) }
hydra.run

Since:

  • 0.5.0

Defined Under Namespace

Modules: Addable, Before, BlockConnection, Cacheable, Memoizable, Queueable, Runnable, Stubbable

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Queueable

#abort, #dequeue, #dequeue_many, #queue, #queue_front, #queued_requests

Methods included from Before

#add

Methods included from Stubbable

#add

Methods included from BlockConnection

#add

Methods included from Cacheable

#add

Methods included from Memoizable

#add, #memory, #run

Methods included from Runnable

#run

Methods included from Addable

#add

Constructor Details

#initialize(options = {}) ⇒ Hydra

Create a new hydra. All Ethon::Multi#initialize options are also available.

Examples:

Create a hydra.

Typhoeus::Hydra.new

Create a hydra with max_concurrency.

Typhoeus::Hydra.new(max_concurrency: 20)

Parameters:

  • options (Hash) (defaults to: {}) —

    The options hash.

Options Hash (options):

  • :max_concurrency (Integer) —

    Number of max concurrent connections to create. Default is 200.

See Also:

Since:

  • 0.5.0



89
90
91
92
93
 
# File 'lib/typhoeus/hydra.rb', line 89

def initialize(options = {})
  @options = options
  @max_concurrency = Integer(@options.fetch(:max_concurrency, 200))
  @multi = Ethon::Multi.new(options.reject{|k,_| k==:max_concurrency})
end

Instance Attribute Details

#max_concurrency ⇒ Object

Examples:

Set max_concurrency.

Typhoeus::Hydra.new(max_concurrency: 20)

Since:

  • 0.5.0



53
54
55
 
# File 'lib/typhoeus/hydra.rb', line 53

def max_concurrency
  @max_concurrency
end

#multi ⇒ Object (readonly)

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Since:

  • 0.5.0



56
57
58
 
# File 'lib/typhoeus/hydra.rb', line 56

def multi
  @multi
end

Class Method Details

.hydra ⇒ Typhoeus::Hydra

Returns a memoized hydra instance.

Examples:

Get a hydra.

Typhoeus::Hydra.hydra

Returns:

Since:

  • 0.5.0



66
67
68
 
# File 'lib/typhoeus/hydra.rb', line 66

def hydra
  Thread.current[:typhoeus_hydra] ||= new
end