Class: BracketNotation::Scanner

Inherits:
Object
  • Object
show all
Includes:
Enumerable
Defined in:
lib/bracket_notation/scanner.rb

Overview

This class represents a scanner for sentences annotated with the kind of bracket notation that is commonly used by linguists. The scanner reads the input string and generates a list of Token instances.

Constant Summary collapse

UNRESERVED_CHARACTER = 
/^[^\[\]\s]$/
LBRACKET_CHARACTER = 
"["
RBRACKET_CHARACTER = 
"]"
EOL_CHARACTER = 
nil

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(input) ⇒ Scanner

Saves the input string.



44
45
46
47
48
49
50
 
# File 'lib/bracket_notation/scanner.rb', line 44

def initialize(input)
  @input = input
  @pos = 0
  @chunk_size = 1
  @last_read = "\n"
  @tokens = nil
end

Instance Attribute Details

#input ⇒ Object (readonly)

Returns the value of attribute input.



36
37
38
 
# File 'lib/bracket_notation/scanner.rb', line 36

def input
  @input
end

Instance Method Details

#each(&block) ⇒ Object

Enumerates the list of tokens, passing each in turn to the provided block.



64
65
66
 
# File 'lib/bracket_notation/scanner.rb', line 64

def each(&block)
  scan.each &block
end

#scan ⇒ Object

Returns an array of all the tokens produced by the scanner.



53
54
55
56
57
58
59
60
61
 
# File 'lib/bracket_notation/scanner.rb', line 53

def scan
  return @tokens unless @tokens.nil?
  
  @tokens = []
  token = nil
  @tokens << token while (token = next_token)
  
  return @tokens
end