Class: RLTK::Token

Inherits:
Object show all
Defined in:
lib/rltk/token.rb

Overview

The Token class is used to represent the output of a RLTK::Lexer and the input of a RLTK::Parser.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(type, value = nil, position = nil) ⇒ Token

Instantiates a new Token object with the values specified.

Parameters:

  • type (Symbol) —

    A symbol representing the type of this Token.

  • value (Object, nil) (defaults to: nil) —

    A value associated with this token.

  • position (StreamPosition, nil) (defaults to: nil) —

    The position of the token in a stream.



65
66
67
68
69
70
 
# File 'lib/rltk/token.rb', line 65

def initialize(type, value = nil, position = nil)
	@type	= type
	@value	= value
	
	@position	= position
end

Instance Attribute Details

#position ⇒ StreamPosition (readonly)

Returns StreamPosition object associated with this token.

Returns:

  • (StreamPosition) —

    StreamPosition object associated with this token.



58
59
60
 
# File 'lib/rltk/token.rb', line 58

def position
  @position
end

#type ⇒ Symbol (readonly)

Returns:

  • (Symbol) 


52
53
54
 
# File 'lib/rltk/token.rb', line 52

def type
  @type
end

#value ⇒ Symbol (readonly)

Returns:

  • (Symbol) 


55
56
57
 
# File 'lib/rltk/token.rb', line 55

def value
  @value
end

Instance Method Details

#==(other) ⇒ Boolean

Compares one token to another. This only tests the token’s type and value and not the location of the token in its source.

Parameters:

  • other (Token) —

    Another Token to compare to.

Returns:

  • (Boolean) 


78
79
80
 
# File 'lib/rltk/token.rb', line 78

def ==(other)
	self.type == other.type and self.value == other.value
end

#to_s ⇒ String

Returns String representing the tokens type and value.

Returns:

  • (String) —

    String representing the tokens type and value.



83
84
85
86
87
88
89
 
# File 'lib/rltk/token.rb', line 83

def to_s
	if value
		"#{self.type}(#{self.value})"
	else
		self.type.to_s
	end
end