Class: Violet::Token

Inherits:
Hash
  • Object
show all
Defined in:
lib/violet/token.rb

Overview

Public: A specialized ‘Hash` for storing and retrieving information about lexed JavaScript tokens.

Constant Summary collapse

Types =

Public: A ‘Hash` that maps the 14 different token kinds to their corresponding numeric constants.

{
  :pattern => 1,
  :identifier => 2,
  :hexadecimal_number => 3,
  :decimal_number => 4,
  :single_quoted_string => 5,
  :double_quoted_string => 6,
  :line_comment => 7,
  :block_comment => 8,
  :whitespace => 9,
  :line_terminator => 10,
  :punctuator => 11,
  :eof => 12,
  :asi => 13,
  :error => 14
}

Instance Method Summary collapse

Constructor Details

#initialize(lexer, kind, position) ⇒ Token

Public: Creates a new ‘Token`.

lexer - The ‘Lexer` instance associated with this token. kind - The `Symbol`-ized token kind. position - A `Range` that specifies the initial and terminal positions of

the token.



32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
 
# File 'lib/violet/token.rb', line 32

def initialize(lexer, kind, position)
  super()
  self.update({
    :name => Types[kind],
    :start => position.begin,
    :stop => position.end,
    :value => lexer.source[position],
    :size => position.end - position.begin,
    :line => lexer.line,
    :column => lexer.column,
    :lines => 0,
    # The position of the token in the token stream.
    :index => 0
  })
end

Instance Method Details

#inspect ⇒ Object

Public: Returns a human-readable ‘String` representation of the token.



72
73
74
75
 
# File 'lib/violet/token.rb', line 72

def inspect
  name, start, size, line, column, value = values_at(:name, :start, :size, :line, :column, :value)
  "#<Violet::Token(#{name}, position: #{start}, length: #{size}, line: #{line}, column: #{column}) #{value.inspect}>"
end

#store(key, value) ⇒ Object Also known as: []=

Public: Associates the ‘value` with the `key`.

key - The hash key. value - The value to associate with the key.

Returns the ‘Token` instance. Raises an `ArgumentError` if the `:pairs` key is set for a non-regular

expression.



56
57
58
59
60
61
62
63
64
65
66
67
 
# File 'lib/violet/token.rb', line 56

def store(key, value)
  case key
  when :pairs, "pairs"
    # The `pairs` attribute may not be set for non-regular expressions.
    raise ArgumentError, "This token is not a regular expression." unless self[:name] == Types[:pattern]
  when :error, "error"
    # Wrap the error message in a `LexerError` instance unless it is
    # already a `Violet::Error`.
    value = Violet::LexerError.new(value) unless value.kind_of?(Violet::Error)
  end
  super(key, value)
end