Class: Antelope::Grammar::Token Abstract

Inherits:
Object
  • Object
show all
Includes:
Comparable
Defined in:
lib/antelope/grammar/token.rb,
lib/antelope/grammar/token/error.rb,
lib/antelope/grammar/token/epsilon.rb,
lib/antelope/grammar/token/terminal.rb,
lib/antelope/grammar/token/nonterminal.rb

Overview

This class is abstract.

This class should be inherited to define a real token. A base class does not match any token; however, any token can match the base class.

Defines a token type for productions/rules.

Direct Known Subclasses

Epsilon, Nonterminal, Terminal

Defined Under Namespace

Classes: Epsilon, Error, Nonterminal, Terminal

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(name, type = nil, id = nil, value = nil) ⇒ Token

Initialize.

Parameters:

  • name (Symbol) —

    the name of the token.

  • type (String?) (defaults to: nil) —

    the type of the token. For definitions, this is the given type of the token (for typed language output).

  • id (String?) (defaults to: nil) —

    the id of the token in the production. For some languages, this allows references to the token via the id.

  • value (String?) (defaults to: nil) —

    the value of the token. This is only used in output representation to the developer.



54
55
56
57
58
59
60
61
 
# File 'lib/antelope/grammar/token.rb', line 54

def initialize(name, type = nil, id = nil, value = nil)
  @name  = name
  @value = value
  @type  = type
  @id    = id
  @from  = nil
  @to    = nil
end

Instance Attribute Details

#from ⇒ Recognizer::State

The from state that this token is transitioned from. This is the source. This is used in the constructor in order to handle lookahead sets.

Returns:

  • (Recognizer::State) 


26
27
28
 
# File 'lib/antelope/grammar/token.rb', line 26

def from
  @from
end

#id ⇒ Object

Returns the value of attribute id.



41
42
43
 
# File 'lib/antelope/grammar/token.rb', line 41

def id
  @id
end

#name ⇒ Symbol (readonly)

The name of the token.

Returns:

  • (Symbol) 


19
20
21
 
# File 'lib/antelope/grammar/token.rb', line 19

def name
  @name
end

#to ⇒ Recognizer::State

The to state that this token is transitioned to. This is the destination. This is used in the constructor in order to handle lookahead sets.

Returns:

  • (Recognizer::State) 


33
34
35
 
# File 'lib/antelope/grammar/token.rb', line 33

def to
  @to
end

#type ⇒ String

The type of the token. This is given by a caret argument to the grammar. This is primarily used for generators.

Returns:

  • (String) 


39
40
41
 
# File 'lib/antelope/grammar/token.rb', line 39

def type
  @type
end

Instance Method Details

#<=>(other) ⇒ Numeric

Compares this class to any other object. If the other object is a token, it converts both this class and the other object to an array and compares the array. Otherwise, it delegates the comparison.

Parameters:

  • other (Object) —

    the other object to compare.

Returns:

  • (Numeric) 


168
169
170
171
172
173
174
 
# File 'lib/antelope/grammar/token.rb', line 168

def <=>(other)
  if other.is_a? Token
    to_a <=> other.to_a
  else
    super
  end
end

#===(other) ⇒ Boolean

Compares this class and another object, fuzzily. If the other object is a token, it removes the transitions (to and from) on both objects and compares them like that. Otherwise, it delegates the comparison.

Parameters:

  • other (Object) —

    the other object to compare.

Returns:

  • (Boolean) —

    if they are equal.



185
186
187
188
189
190
191
 
# File 'lib/antelope/grammar/token.rb', line 185

def ===(other)
  if other.is_a? Token
    without_transitions == other.without_transitions
  else
    super
  end
end

#epsilon? ⇒ Boolean

This method is abstract.

Whether or not the token is an epsilon token.

Returns:

  • (Boolean) 


85
86
87
 
# File 'lib/antelope/grammar/token.rb', line 85

def epsilon?
  false
end

#error? ⇒ Boolean

This method is abstract.

Whether or not the token is an error token.

Returns:

  • (Boolean) 


93
94
95
 
# File 'lib/antelope/grammar/token.rb', line 93

def error?
  false
end

#hash ⇒ Object

Note:

This is not intended for use. It is only defined to be compatible with Hashs (and by extension, Sets).

Generates a hash for this class.

Returns:

  • (Object) 


214
215
216
 
# File 'lib/antelope/grammar/token.rb', line 214

def hash
  @_hash ||= to_a.hash
end

#inspect ⇒ String

Returns a nice inspect.

Returns:

  • (String) 


156
157
158
159
 
# File 'lib/antelope/grammar/token.rb', line 156

def inspect
  "#<#{self.class} from=#{from.id if from} to=#{to.id if to} " \
    "name=#{name.inspect} value=#{@value.inspect}>"
end

#invalidate_cache! ⇒ void

This method returns an undefined value.

Invalidates the cache.



203
204
205
206
 
# File 'lib/antelope/grammar/token.rb', line 203

def invalidate_cache!
  @_hash = nil
  @_array = nil
end

#nonterminal? ⇒ Boolean

This method is abstract.

Whether or not the token is a nonterminal.

Returns:

  • (Boolean) 


77
78
79
 
# File 'lib/antelope/grammar/token.rb', line 77

def nonterminal?
  false
end

#terminal? ⇒ Boolean

This method is abstract.

Whether or not the token is a terminal.

Returns:

  • (Boolean) 


69
70
71
 
# File 'lib/antelope/grammar/token.rb', line 69

def terminal?
  false
end

#to_a ⇒ Array<(Recognizer::State, Recognizer::State, Class, Symbol, String?)>

Note:

This is not intended for use. It is only defined to make equality checking easier, and to create a hash.

Creates an array representation of this class.

Returns:

  • (Array<(Recognizer::State, Recognizer::State, Class, Symbol, String?)>) 


226
227
228
 
# File 'lib/antelope/grammar/token.rb', line 226

def to_a
  @_array ||= [to, from, self.class, name, @value]
end

#to_s ⇒ String

Gives a string representation of the token. The output is formatted like so: <data>["(" [<from_id>][:<to_id>] ")"], where <data> is either the value (if it's non-nil) or the name, <from_id> is the from state id, and <to_id> is the to state id. The last part of the format is optional; if neither the from state or to state is non-nil, it's non- existant.

Returns:

  • (String) —

    the string representation.

See Also:



136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
 
# File 'lib/antelope/grammar/token.rb', line 136

def to_s
  buf = if @value
          @value.inspect
        else
          @name.to_s
        end

  if from || to
    buf << '('
    buf << "#{from.id}" if from
    buf << ":#{to.id}"  if to
    buf << ')'
  end

  buf
end

#without_transitions ⇒ Token

Creates a new token without to or from states.

Returns:



196
197
198
 
# File 'lib/antelope/grammar/token.rb', line 196

def without_transitions
  self.class.new(name, @type, @id, @value)
end