Method: Grubby::Scraper.scrapes

Defined in:
lib/grubby/scraper.rb

.scrapes(field, **options, &block) ⇒ void

This method returns an undefined value.

Defines an attribute reader method named by field. During #initialize, the given block is called, and the attribute is set to the block’s return value.

By default, raises an exception if the block’s return value is nil. To prevent this behavior, set the :optional option to true. Alternatively, the block can be conditionally evaluated, based on another method’s return value, using the :if or :unless options.

Examples:

Default behavior

class GreetingScraper < Grubby::Scraper
  scrapes(:name) do
    source[/Hello (\w+)/, 1]
  end
end

scraper = GreetingScraper.new("Hello World!")
scraper.name  # == "World"

scraper = GreetingScraper.new("Hello!")  # raises Grubby::Scraper::Error

Optional scraped value

class GreetingScraper < Grubby::Scraper
  scrapes(:name, optional: true) do
    source[/Hello (\w+)/, 1]
  end
end

scraper = GreetingScraper.new("Hello World!")
scraper.name  # == "World"

scraper = GreetingScraper.new("Hello!")
scraper.name  # == nil

Conditional scraped value

class GreetingScraper < Grubby::Scraper
  def hello?
    source.start_with?("Hello ")
  end

  scrapes(:name, if: :hello?) do
    source[/Hello (\w+)/, 1]
  end
end

scraper = GreetingScraper.new("Hello World!")
scraper.name  # == "World"

scraper = GreetingScraper.new("Hello!")  # raises Grubby::Scraper::Error

scraper = GreetingScraper.new("How are you?")
scraper.name  # == nil

Parameters:

  • field (Symbol, String)
  • options (Hash)

Options Hash (**options):

  • :optional (Boolean) — default: false —

    Whether the block should be allowed to return a nil value

  • :if (Symbol) — default: nil —

    Name of predicate method that determines if the block should be evaluated

  • :unless (Symbol) — default: nil —

    Name of predicate method that determines if the block should not be evaluated

Yield Returns:

  • (Object) 


68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
 
# File 'lib/grubby/scraper.rb', line 68

def self.scrapes(field, **options, &block)
  field = field.to_sym
  (self.fields << field).uniq!

  define_method(field) do
    raise "#{self.class}#initialize does not invoke `super`" unless defined?(@scraped)

    if !@scraped.key?(field) && !@errors.key?(field)
      begin
        skip = (options[:if] && !self.send(options[:if])) ||
          (options[:unless] && self.send(options[:unless]))

        if skip
          @scraped[field] = nil
        else
          @scraped[field] = instance_eval(&block)
          if @scraped[field].nil?
            raise FieldValueRequiredError.new(field) unless options[:optional]
            $log.debug("#{self.class}##{field} is nil")
          end
        end
      rescue RuntimeError, IndexError => e
        @errors[field] = e
      end
    end

    if @errors.key?(field)
      raise FieldScrapeFailedError.new(field, @errors[field])
    else
      @scraped[field]
    end
  end
end