Class: EBNF::LL1::Scanner

Inherits:
StringScanner
  • Object
show all
Defined in:
lib/ebnf/ll1/scanner.rb

Overview

Overload StringScanner with file operations

  • Reloads scanner as required until EOF.

  • Loads to a high-water and reloads when remaining size reaches a low-water.

FIXME: Only implements the subset required by the Lexer for now.

Constant Summary collapse

HIGH_WATER = 
10240
LOW_WATER =

Hopefully large enough to deal with long multi-line comments

2048

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(input, options = {}) ⇒ Scanner

Create a scanner, from an IO or String

Parameters:

  • input (String, IO, #read)
  • options (Hash{Symbol => Object}) (defaults to: {})
  • options[Integer] (Hash) —

    a customizable set of options



27
28
29
30
31
32
33
34
35
36
37
 
# File 'lib/ebnf/ll1/scanner.rb', line 27

def initialize(input, options = {})
  @options = options.merge(:high_water => HIGH_WATER, :low_water => LOW_WATER)

  if input.respond_to?(:read)
    @input = input
    super("")
    feed_me
  else
    super(input.to_s)
  end
end

Instance Attribute Details

#input ⇒ IO, StringIO (readonly)

Returns:

  • (IO, StringIO) 


17
18
19
 
# File 'lib/ebnf/ll1/scanner.rb', line 17

def input
  @input
end

Instance Method Details

#rest ⇒ String

Returns the “rest” of the line, or the next line if at EOL (i.e. everything after the scan pointer). If there is no more data (eos? = true), it returns “”.

Returns:

  • (String) 


44
45
46
47
 
# File 'lib/ebnf/ll1/scanner.rb', line 44

def rest
  feed_me
  encode_utf8 super
end

#scan(pattern) ⇒ String

Tries to match with pattern at the current position.

If there is a match, the scanner advances the “scan pointer” and returns the matched string. Otherwise, the scanner returns nil.

If the scanner begins with the multi-line start expression

Examples:

s = StringScanner.new('test string')
p s.scan(/\w+/)   # -> "test"
p s.scan(/\w+/)   # -> nil
p s.scan(/\s+/)   # -> " "
p s.scan(/\w+/)   # -> "string"
p s.scan(/./)     # -> nil

Parameters:

  • pattern (Regexp)

Returns:

  • (String) 


78
79
80
81
 
# File 'lib/ebnf/ll1/scanner.rb', line 78

def scan(pattern)
  feed_me
  encode_utf8 super
end

#skip(pattern) ⇒ Object

Attempts to skip over the given pattern beginning with the scan pointer. If it matches, the scan pointer is advanced to the end of the match, and the length of the match is returned. Otherwise, nil is returned.

similar to scan, but without returning the matched string.

Parameters:

  • pattern (Regexp) 


56
57
58
59
 
# File 'lib/ebnf/ll1/scanner.rb', line 56

def skip(pattern)
  feed_me
  super
end