Class: CodeRay::TokenStream
- Defined in:
- lib/coderay/tokens.rb
Overview
TokenStream
The TokenStream class is a fake Array without elements.
It redirects the method << to a block given at creation.
This allows scanners and Encoders to use streaming (no tokens are saved, the input is highlighted the same time it is scanned) with the same code.
See CodeRay.encode_stream and CodeRay.scan_stream
Constant Summary
Constants inherited from Tokens
Instance Attribute Summary collapse
-
#size ⇒ Object
readonly
The Array is empty, but size counts the tokens given by <<.
Attributes inherited from Tokens
Instance Method Summary collapse
-
#<<(token) ⇒ Object
Calls
block
withtoken
and increments size. -
#dump ⇒ Object
A TokenStream cannot be dumped.
-
#initialize(&block) ⇒ TokenStream
constructor
Creates a new TokenStream that calls
block
whenever its << method is called. -
#optimize ⇒ Object
A TokenStream cannot be optimized.
-
#stream? ⇒ Boolean
Whether the object is a TokenStream.
-
#text_size ⇒ Object
This method is not implemented due to speed reasons.
Methods inherited from Tokens
#each, #each_text_token, #encode, #fix, #fix!, load, #method_missing, #optimize!, #split_into_lines, #split_into_lines!, #text, #to_s
Constructor Details
#initialize(&block) ⇒ TokenStream
Creates a new TokenStream that calls block
whenever its << method is called.
Example:
require 'coderay'
token_stream = CodeRay::TokenStream.new do |text, kind|
puts 'kind: %s, text size: %d.' % [kind, text.size]
end
token_stream << ['/\d+/', :regexp]
#-> kind: rexpexp, text size: 5.
313 314 315 316 317 |
# File 'lib/coderay/tokens.rb', line 313 def initialize &block raise ArgumentError, 'Block expected for streaming.' unless block @callback = block @size = 0 end |
Dynamic Method Handling
This class handles dynamic methods through the method_missing method in the class CodeRay::Tokens
Instance Attribute Details
#size ⇒ Object (readonly)
The Array is empty, but size counts the tokens given by <<.
297 298 299 |
# File 'lib/coderay/tokens.rb', line 297 def size @size end |
Instance Method Details
#<<(token) ⇒ Object
Calls block
with token
and increments size.
Returns self.
322 323 324 325 326 |
# File 'lib/coderay/tokens.rb', line 322 def << token @callback.call(*token) @size += 1 self end |
#dump ⇒ Object
A TokenStream cannot be dumped. Use Tokens.
335 336 337 |
# File 'lib/coderay/tokens.rb', line 335 def dump raise NotImplementedError, 'A TokenStream cannot be dumped.' end |
#optimize ⇒ Object
A TokenStream cannot be optimized. Use Tokens.
340 341 342 |
# File 'lib/coderay/tokens.rb', line 340 def optimize raise NotImplementedError, 'A TokenStream cannot be optimized.' end |
#stream? ⇒ Boolean
Whether the object is a TokenStream.
Returns true.
292 293 294 |
# File 'lib/coderay/tokens.rb', line 292 def stream? true end |
#text_size ⇒ Object
This method is not implemented due to speed reasons. Use Tokens.
329 330 331 332 |
# File 'lib/coderay/tokens.rb', line 329 def text_size raise NotImplementedError, 'This method is not implemented due to speed reasons.' end |