Class: Puppet::Pops::Parser::Lexer2WithComments

Inherits:
Lexer2
  • Object
show all
Defined in:
lib/puppet-languageserver/puppet_lexer_helper.rb

Overview

This Lexer adds code to create comment tokens. The default lexer just throws them out Ref - github.com/puppetlabs/puppet-specifications/blob/master/language/lexical_structure.md#comments

Constant Summary collapse

PATTERN_COMMENT_NO_WS =

The PATTERN_COMMENT in lexer2 also consumes the trailing r in the token and we don’t want that.

/#[^\r\n]*/.freeze
TOKEN_COMMENT = 
[:COMMENT, '#', 1].freeze
TOKEN_MLCOMMENT = 
[:MLCOMMENT, nil, 0].freeze

Instance Method Summary collapse

Constructor Details

#initialize ⇒ Lexer2WithComments 



17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
 
# File 'lib/puppet-languageserver/puppet_lexer_helper.rb', line 17

def initialize
  super

  # Remove the selector for line comments so we can add our own
  @new_selector = @selector.reject { |k, _v| k == '#' }

  # Add code to scan line comments
  @new_selector['#'] = lambda {
    scn = @scanner
    before = scn.pos
    value = scn.scan(PATTERN_COMMENT_NO_WS)

    if value
      emit_completed([:TOKEN_COMMENT, value[1..-1].freeze, scn.pos - before], before)
    else
      # It's probably not possible to EVER get here ... but just incase
      emit(TOKEN_COMMENT, before)
    end
  }.freeze

  # Add code to scan multi-line comments
  old_lambda = @new_selector['/']
  @new_selector['/'] = lambda {
    scn = @scanner
    la = scn.peek(2)
    if la[1] == '*'
      before = scn.pos
      value = scn.scan(PATTERN_MLCOMMENT)
      return emit_completed([:TOKEN_MLCOMMENT, value[2..-3].freeze, scn.pos - before], before) if value
    end
    old_lambda.call
  }.freeze
  @new_selector.freeze
  @selector = @new_selector
end