Class: Puppet::Pops::Parser::Lexer2WithComments
- Inherits:
-
Lexer2
- Object
- Lexer2
- Puppet::Pops::Parser::Lexer2WithComments
- Defined in:
- lib/puppet-languageserver/puppet_lexer_helper.rb
Overview
This Lexer adds code to create comment tokens. The default lexer just throws them out Ref - github.com/puppetlabs/puppet-specifications/blob/master/language/lexical_structure.md#comments
Constant Summary collapse
- PATTERN_COMMENT_NO_WS =
The PATTERN_COMMENT in lexer2 also consumes the trailing r in the token and we don’t want that.
/#[^\r\n]*/.freeze
- TOKEN_COMMENT =
[:COMMENT, '#', 1].freeze
- TOKEN_MLCOMMENT =
[:MLCOMMENT, nil, 0].freeze
Instance Method Summary collapse
-
#initialize ⇒ Lexer2WithComments
constructor
A new instance of Lexer2WithComments.
Constructor Details
#initialize ⇒ Lexer2WithComments
Returns a new instance of Lexer2WithComments.
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 |
# File 'lib/puppet-languageserver/puppet_lexer_helper.rb', line 17 def initialize super # Remove the selector for line comments so we can add our own @new_selector = @selector.reject { |k, _v| k == '#' } # Add code to scan line comments @new_selector['#'] = lambda { scn = @scanner before = scn.pos value = scn.scan(PATTERN_COMMENT_NO_WS) if value emit_completed([:TOKEN_COMMENT, value[1..-1].freeze, scn.pos - before], before) else # It's probably not possible to EVER get here ... but just incase emit(TOKEN_COMMENT, before) end }.freeze # Add code to scan multi-line comments old_lambda = @new_selector['/'] @new_selector['/'] = lambda { scn = @scanner la = scn.peek(2) if la[1] == '*' before = scn.pos value = scn.scan(PATTERN_MLCOMMENT) return emit_completed([:TOKEN_MLCOMMENT, value[2..-3].freeze, scn.pos - before], before) if value end old_lambda.call }.freeze @new_selector.freeze @selector = @new_selector end |