Class: Ruty::Datastructure::TokenStream

Inherits:
Object
  • Object
show all
Defined in:
lib/ruty/datastructure.rb

Overview

stream for the tokenize function

Instance Method Summary collapse

Constructor Details

#initialize ⇒ TokenStream

Returns a new instance of TokenStream.



102
103
104
105
106
 
# File 'lib/ruty/datastructure.rb', line 102

def initialize
  @stream = []
  @closed = false
  @pushed = []
end

Instance Method Details

#<<(token) ⇒ Object

add one token to a non closed stream once the stream is closed you can still add tokens to the stream but by pushing back which you can do by calling push.

Raises:

  • (RuntimeError) 


131
132
133
134
 
# File 'lib/ruty/datastructure.rb', line 131

def << token
  raise RuntimeError, 'cannot write to closed stream' if @closed
  @stream << token
end

#close ⇒ Object

close the stream and return self

Raises:

  • (RuntimeError) 


137
138
139
140
141
142
 
# File 'lib/ruty/datastructure.rb', line 137

def close
  raise RuntimeError, 'cannot close closed token stream' if @closed
  @closed = true
  @stream.reverse!
  self
end

#eos? ⇒ Boolean

Returns:

  • (Boolean) 


116
117
118
 
# File 'lib/ruty/datastructure.rb', line 116

def eos?
  @stream.empty?
end

#next ⇒ Object 



108
109
110
111
112
113
114
 
# File 'lib/ruty/datastructure.rb', line 108

def next
  if not @pushed.empty?
    @pushed.pop
  else
    @stream.pop
  end
end

#push(token) ⇒ Object

push a token to a closed or nonclosed stream. pushed tokens are always processed first in reverse order



123
124
125
 
# File 'lib/ruty/datastructure.rb', line 123

def push token
  @pushed << token
end