Class: CodeRay::TokenStream
- Defined in:
- lib/coderay/tokens.rb
Overview
TokenStream
The TokenStream class is a fake Array without elements.
It redirects the method << to a block given at creation.
This allows scanners and Encoders to use streaming (no tokens are saved, the input is highlighted the same time it is scanned) with the same code.
See CodeRay.encode_stream and CodeRay.scan_stream
Constant Summary
Constants inherited from Tokens
Instance Attribute Summary collapse
-
#size ⇒ Object
readonly
The Array is empty, but size counts the tokens given by <<.
Attributes inherited from Tokens
Instance Method Summary collapse
-
#<<(token) ⇒ Object
Calls
block
withtoken
and increments size. -
#dump ⇒ Object
A TokenStream cannot be dumped.
-
#initialize(&block) ⇒ TokenStream
constructor
Creates a new TokenStream that calls
block
whenever its << method is called. -
#optimize ⇒ Object
A TokenStream cannot be optimized.
-
#stream? ⇒ Boolean
Whether the object is a TokenStream.
-
#text_size ⇒ Object
This method is not implemented due to speed reasons.
Methods inherited from Tokens
#each, #each_text_token, #encode, #fix, #fix!, load, #method_missing, #optimize!, #split_into_lines, #split_into_lines!, #text, #to_s
Constructor Details
#initialize(&block) ⇒ TokenStream
Creates a new TokenStream that calls block
whenever its << method is called.
Example:
require 'coderay'
token_stream = CodeRay::TokenStream.new do |kind, text|
puts 'kind: %s, text size: %d.' % [kind, text.size]
end
token_stream << [:regexp, '/\d+/']
#-> kind: rexpexp, text size: 5.
311 312 313 314 315 |
# File 'lib/coderay/tokens.rb', line 311 def initialize &block raise ArgumentError, 'Block expected for streaming.' unless block @callback = block @size = 0 end |
Dynamic Method Handling
This class handles dynamic methods through the method_missing method in the class CodeRay::Tokens
Instance Attribute Details
#size ⇒ Object (readonly)
The Array is empty, but size counts the tokens given by <<.
295 296 297 |
# File 'lib/coderay/tokens.rb', line 295 def size @size end |
Instance Method Details
#<<(token) ⇒ Object
Calls block
with token
and increments size.
Returns self.
320 321 322 323 324 |
# File 'lib/coderay/tokens.rb', line 320 def << token @callback.call(*token) @size += 1 self end |
#dump ⇒ Object
A TokenStream cannot be dumped. Use Tokens.
333 334 335 |
# File 'lib/coderay/tokens.rb', line 333 def dump raise NotImplementedError, 'A TokenStream cannot be dumped.' end |
#optimize ⇒ Object
A TokenStream cannot be optimized. Use Tokens.
338 339 340 |
# File 'lib/coderay/tokens.rb', line 338 def optimize raise NotImplementedError, 'A TokenStream cannot be optimized.' end |
#stream? ⇒ Boolean
Whether the object is a TokenStream.
Returns true.
290 291 292 |
# File 'lib/coderay/tokens.rb', line 290 def stream? true end |
#text_size ⇒ Object
This method is not implemented due to speed reasons. Use Tokens.
327 328 329 330 |
# File 'lib/coderay/tokens.rb', line 327 def text_size raise NotImplementedError, 'This method is not implemented due to speed reasons.' end |