Class: Prawn::Format::Lexer

Inherits:
Object
  • Object
show all
Defined in:
lib/prawn/format/lexer.rb

Overview

The Lexer class is used by the formatting subsystem to scan a string and extract tokens from it. The tokens it looks for are either text, XML entities, or XML tags.

Note that the lexer only scans for a subset of XML–it is not a true XML scanner, and understands just enough to provide a basic markup language for use in formatting documents.

The subset includes only XML entities and tags–instructions, comments, and the like are not supported.

Defined Under Namespace

Classes: InvalidFormat

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(text) ⇒ Lexer

Create a new lexer that will scan the given text. The text must be UTF-8 encoded, and must consist of well-formed XML in the subset understand by the lexer.



31
32
33
34
35
 
# File 'lib/prawn/format/lexer.rb', line 31

def initialize(text)
  @scanner = StringScanner.new(text)
  @state = :start
  @verbatim = false
end

Instance Attribute Details

#verbatim ⇒ Object

Controls whether whitespace is lexed verbatim or not. If not, adjacent whitespace is compressed into a single space character (this includes newlines).



26
27
28
 
# File 'lib/prawn/format/lexer.rb', line 26

def verbatim
  @verbatim
end

Instance Method Details

#each ⇒ Object

Iterates over each token in the string, until the end of the string is reached. Each token is yielded. See #next for a discussion of the available token types.



65
66
67
68
69
 
# File 'lib/prawn/format/lexer.rb', line 65

def each
  while (token = next_token)
    yield token
  end
end

#next ⇒ Object

Returns the next token from the scanner. If the end of the string has been reached, this will return nil. Otherwise, the token itself is returned as a hash. The hash will always include a :type key, identifying the type of the token. It will be one of :text, :open, or :close.

For :text tokens, the hash will also contain a :text key, which will point to an array of strings. Each element of the array contains either word, whitespace, or some other character at which the line may be broken.

For :open tokens, the hash will contain a :tag key which identifies the name of the tag (as a symbol), and an :options key, which is another hash that contains the options that were given with the tag.

For :close tokens, the hash will contain only a :tag key.



54
55
56
57
58
59
60
 
# File 'lib/prawn/format/lexer.rb', line 54

def next
  if @state == :start && @scanner.eos?
    return nil
  else
    scan_next_token
  end
end