Class: Plumnailer::Fetcher

Inherits:
Object
  • Object
show all
Defined in:
lib/plumnailer/fetcher.rb

Overview

Fetch the contents of a url.

Direct Known Subclasses

CachingFetcher

Constant Summary collapse

RedirectLimit =

Follow this many chained HTTP redirects at most.

3

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Fetcher

Returns a new instance of Fetcher.



8
9
10
 
# File 'lib/plumnailer/fetcher.rb', line 8

def initialize
  @user_agent = 'plumnailer (https://github.com/mmb/plumnailer)'
end

Instance Attribute Details

#user_agent ⇒ Object

Returns the value of attribute user_agent.



40
41
42
 
# File 'lib/plumnailer/fetcher.rb', line 40

def user_agent
  @user_agent
end

Instance Method Details

#fetch(url, orig_url = nil, redirect_count = 0) ⇒ Object

Fetch the contents of a url.

  • url - url to fetch, can be string or URI

  • orig_url - passed in from redirects

  • redirect_count - the number of redirects so far



20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
 
# File 'lib/plumnailer/fetcher.rb', line 20

def fetch(url, orig_url=nil, redirect_count=0)
  if redirect_count > RedirectLimit
    raise ArgumentError,
      "too many redirects (#{redirect_count}) for #{orig_url}"
  end

  uri = url.is_a?(URI) ? url : URI(url)
  if uri.is_a?(URI::HTTP)
    http = Net::HTTP::Persistent.new('plumnailer')
    http.headers['User-Agent'] = user_agent
    resp = http.request(uri)
    case resp
      when Net::HTTPSuccess; resp.body
      when Net::HTTPRedirection; fetch(resp['location'], orig_url || url,
        redirect_count + 1)
      else resp.error!
    end
  end
end