Class: Dhaka::Lexer

Inherits:
Dhaka::LexerSupport::StateMachine show all
Defined in:
lib/dhaka/lexer/lexer.rb

Overview

The lexer generator. To generate a lexer from a lexer specification MyLexerSpecification:

lexer = Dhaka::Lexer.new(MyLexerSpecification)

To compile this lexer as MyLexer to a string of Ruby source:

lexer.compile_to_ruby_source_as(:MyLexer)

Instance Attribute Summary collapse

Attributes inherited from Dhaka::LexerSupport::StateMachine

#start_state

Instance Method Summary collapse

Methods inherited from Dhaka::LexerSupport::StateMachine

#to_dot

Constructor Details

#initialize(specification) ⇒ Lexer

Creates a new lexer from a given specification.



11
12
13
14
15
16
17
18
 
# File 'lib/dhaka/lexer/lexer.rb', line 11

def initialize(specification)
  dfas           = {}
  @specification = specification
  specification.items.each do |pattern, item|
    dfas[pattern] = LexerSupport::DFA.new(pattern)
  end
  super(ItemSet.new(dfas.values.collect{|dfa| dfa.start_state}))
end

Instance Attribute Details

#specification ⇒ Object (readonly)

Returns the value of attribute specification.



8
9
10
 
# File 'lib/dhaka/lexer/lexer.rb', line 8

def specification
  @specification
end

Instance Method Details

#action_for_pattern(pattern) ⇒ Object

:nodoc



38
39
40
 
# File 'lib/dhaka/lexer/lexer.rb', line 38

def action_for_pattern pattern #:nodoc
  @specification.items[pattern].action
end

#compile_to_ruby_source_as(lexer_class_name) ⇒ Object

Compiles the lexer to Ruby code that when executed, reloads all the states and actions of the lexer into a class named lexer_class_name.



22
23
24
25
26
27
28
29
30
31
 
# File 'lib/dhaka/lexer/lexer.rb', line 22

def compile_to_ruby_source_as lexer_class_name
  result  =   "class #{lexer_class_name} < Dhaka::CompiledLexer\n\n"
  result <<   "  self.specification = #{specification.name}\n\n"
  result <<   "  start_with #{start_state.object_id}\n\n"
  @states.each do |key, state|
    result << "#{state.compile_to_ruby_source}\n\n"
  end
  result <<   "end"
  result
end

#lex(input) ⇒ Object

Returns a LexerRun that tokenizes input.



34
35
36
 
# File 'lib/dhaka/lexer/lexer.rb', line 34

def lex input
  LexerRun.new(self, input)
end