Module: Upton::Utils

Defined in:
lib/upton/utils.rb

Overview

This class contains a collection of helpers for Upton

Each method returns a Proc that (with an & ) can be used as the final argument to Upton’s ‘scrape` and `scrape_to_csv`

Class Method Summary collapse

Class Method Details

.list(list_selector, deprecated = nil) ⇒ Object

Scrapes any set of HTML elements into an Array.



35
36
37
38
39
40
 
# File 'lib/upton/utils.rb', line 35

def self.list(list_selector, deprecated=nil)
  return Proc.new do |instance_html|
    html = ::Nokogiri::HTML(instance_html)
    html.search(list_selector).map{|list_element| list_element.text }
  end
end

.table(table_selector, deprecated = nil) ⇒ Object

Scrapes an HTML <table> element into an Array of Arrays. The header, if present, is returned as the first row.



20
21
22
23
24
25
26
27
28
29
30
 
# File 'lib/upton/utils.rb', line 20

def self.table(table_selector, deprecated=nil)
  return Proc.new do |instance_html|
    html = ::Nokogiri::HTML(instance_html)
    output = []
    headers = html.search(table_selector).css("th").map &:text
    output << headers

    table = html.search(table_selector).css("tr").each{|tr| output << tr.css("td").map(&:text) }
    output
  end
end