Module: Upton::Utils

Defined in:: lib/upton/utils.rb

Overview

This class contains a collection of helpers for Upton

Each method returns a Proc that (with an & ) can be used as the final argument to Upton's scrape and scrape_to_csv

Class Method Summary collapse

.list(list_selector, deprecated = nil) ⇒ Object
Scrapes any set of HTML elements into an Array.

.table(table_selector, deprecated = nil) ⇒ Object

Scrapes an HTML

element into an Array of Arrays.

Class Method Details

.list(list_selector, deprecated = nil) ⇒ `Object`

Scrapes any set of HTML elements into an Array.

# File 'lib/upton/utils.rb', line 35

def self.list(list_selector, deprecated=nil)
  return Proc.new do |instance_html|
    html = ::Nokogiri::HTML(instance_html)
    html.search(list_selector).map{|list_element| list_element.text }
  end
end

.table(table_selector, deprecated = nil) ⇒ `Object`

Scrapes an HTML

element into an Array of Arrays. The header, if present, is returned as the first row.

# File 'lib/upton/utils.rb', line 20

def self.table(table_selector, deprecated=nil)
  return Proc.new do |instance_html|
    html = ::Nokogiri::HTML(instance_html)
    output = []
    headers = html.search(table_selector).css("th").map &:text
    output << headers

    table = html.search(table_selector).css("tr").each{|tr| output << tr.css("td").map(&:text) }
    output
  end
end