Class: Puppet::Pops::Parser::Lexer2WithComments

Inherits:
Lexer2
  • Object
show all
Defined in:
lib/puppet-languageserver/puppet_lexer_helper.rb

Overview

This Lexer adds code to create comment tokens. The default lexer just throws them out Ref - github.com/puppetlabs/puppet-specifications/blob/master/language/lexical_structure.md#comments

Constant Summary collapse

PATTERN_COMMENT_NO_WS =

The PATTERN_COMMENT in lexer2 also consumes the trailing r in the token and we don’t want that.

/#[^\r\n]*/.freeze
TOKEN_COMMENT = 
[:COMMENT, '#', 1].freeze
TOKEN_MLCOMMENT = 
[:MLCOMMENT, nil, 0].freeze

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Lexer2WithComments

Returns a new instance of Lexer2WithComments.



17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
 
# File 'lib/puppet-languageserver/puppet_lexer_helper.rb', line 17

def initialize
  super

  # Remove the selector for line comments so we can add our own
  @new_selector = @selector.reject { |k, _v| k == '#' }

  # Add code to scan line comments
  @new_selector['#'] = lambda {
    scn = @scanner
    before = scn.pos
    value = scn.scan(PATTERN_COMMENT_NO_WS)

    if value
      emit_completed([:TOKEN_COMMENT, value[1..-1].freeze, scn.pos - before], before)
    else
      # It's probably not possible to EVER get here ... but just incase
      emit(TOKEN_COMMENT, before)
    end
  }.freeze

  # Add code to scan multi-line comments
  old_lambda = @new_selector['/']
  @new_selector['/'] = lambda {
    scn = @scanner
    la = scn.peek(2)
    if la[1] == '*'
      before = scn.pos
      value = scn.scan(PATTERN_MLCOMMENT)
      return emit_completed([:TOKEN_MLCOMMENT, value[2..-3].freeze, scn.pos - before], before) if value
    end
    old_lambda.call
  }.freeze
  @new_selector.freeze
  @selector = @new_selector
end