Class: HH::Syntax::Token

Inherits:
String show all
Defined in:
lib/ext/highlighter/common.rb

Overview

A single token extracted by a tokenizer. It is simply the lexeme itself, decorated with a ‘group’ attribute to identify the type of the lexeme.

Instance Attribute Summary collapse

Instance Method Summary collapse

Methods inherited from String

#mindex

Constructor Details

#initialize(text, group, instruction = :none) ⇒ Token

Create a new Token representing the given text, and belonging to the given group.



19
20
21
22
23
 
# File 'lib/ext/highlighter/common.rb', line 19

def initialize( text, group, instruction = :none )
  super text
  @group = group
  @instruction = instruction
end

Instance Attribute Details

#group ⇒ Object (readonly)

the type of the lexeme that was extracted.



11
12
13
 
# File 'lib/ext/highlighter/common.rb', line 11

def group
  @group
end

#instruction ⇒ Object (readonly)

the instruction associated with this token (:none, :region_open, or :region_close)



15
16
17
 
# File 'lib/ext/highlighter/common.rb', line 15

def instruction
  @instruction
end