Class: RLTK::Token

Inherits:
Object
  • Object
show all
Defined in:
lib/rltk/token.rb

Overview

The Token class is used to represent the output of a RLTK::Lexer and the input of a RLTK::Parser.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(type, value = nil, position = nil) ⇒ Token

Instantiates a new Token object with the values specified.

Parameters:

  • type (Symbol) —

    A symbol representing the type of this Token.

  • value (Object, nil) (defaults to: nil) —

    A value associated with this token.

  • position (StreamPosition, nil) (defaults to: nil) —

    The position of the token in a stream.



64
65
66
67
68
69
 
# File 'lib/rltk/token.rb', line 64

def initialize(type, value = nil, position = nil)
	@type	= type
	@value	= value

	@position	= position
end

Instance Attribute Details

#position ⇒ StreamPosition (readonly)

Returns StreamPosition object associated with this token.

Returns:

  • (StreamPosition) —

    StreamPosition object associated with this token.



57
58
59
 
# File 'lib/rltk/token.rb', line 57

def position
  @position
end

#type ⇒ Symbol (readonly)

Returns:

  • (Symbol) 


51
52
53
 
# File 'lib/rltk/token.rb', line 51

def type
  @type
end

#value ⇒ Symbol (readonly)

Returns:

  • (Symbol) 


54
55
56
 
# File 'lib/rltk/token.rb', line 54

def value
  @value
end

Instance Method Details

#==(other) ⇒ Boolean

Compares one token to another. This only tests the token’s type and value and not the location of the token in its source.

Parameters:

  • other (Token) —

    Another Token to compare to.

Returns:

  • (Boolean) 


77
78
79
 
# File 'lib/rltk/token.rb', line 77

def ==(other)
	self.type == other.type and self.value == other.value
end

#to_s ⇒ String

Returns String representing the tokens type and value.

Returns:

  • (String) —

    String representing the tokens type and value.



82
83
84
85
86
87
88
 
# File 'lib/rltk/token.rb', line 82

def to_s
	if value
		"#{self.type}(#{self.value})"
	else
		self.type.to_s
	end
end