Class: TinyGrabber

Inherits:
Object
  • Object
show all
Defined in:
lib/tiny_grabber.rb,
lib/tiny_grabber/version.rb

Overview

Main class for TinyGrabber

Constant Summary collapse

VERSION =

Version number

"0.0.6"

Class Method Summary collapse

Class Method Details

.get(url, params = {}) ⇒ Object

Get Net::HTTP object for content from remote single page

url (string)

Link on content

params (hash)

Addition setting

  • proxy

    Configuration of remote proxy server

    • ip address of remote proxy server

    • port of remote proxy server

    • proxy_type of remote proxy server

      • http for http(s) proxy servers (by default)

      • socks for socks4/5 proxy servers

  • headers

    Headers

Example

TinyGrabber.get url, headers: { 'Content-Type' => 'application/json' }, proxy: { ip: 'xx.xx.xx.xx', port: xx, proxy_type: :socks }

Parameters:

  • url —

    Link for content

  • params (defaults to: {}) —

    Addition setting



29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
 
# File 'lib/tiny_grabber.rb', line 29

def self.get url, params = {}
  uri = URI(URI.escape(url))

  params = convert_params_to_sym params
  if params[:proxy]
    # Socks4(5) proxy
    if ['socks', :socks].include?(params[:proxy][:proxy_type])
      http = Net::HTTP.SOCKSProxy(params[:proxy][:ip], params[:proxy][:port])
    # Http(s) proxy
    else
      http = Net::HTTP::Proxy(params[:proxy][:ip], params[:proxy][:port])
    end
  else
    http = Net::HTTP
  end
  response = http.start(uri.host, uri.port, use_ssl: uri.scheme == 'https') do |http|
    request = Net::HTTP::Get.new uri

    # Add headers
    params[:headers].each { |k, v| request.add_field(k, v) } if params[:headers]
    http.request request
  end
  response
end