Class: Parser::Lexer::Dedenter

Inherits:
Object
  • Object
show all
Defined in:
lib/parser/lexer/dedenter.rb

Constant Summary collapse

TAB_WIDTH =

Tab (t) counts as 8 spaces

8

Instance Method Summary collapse

Constructor Details

#initialize(dedent_level) ⇒ Dedenter

Returns a new instance of Dedenter.



9
10
11
12
13
 
# File 'lib/parser/lexer/dedenter.rb', line 9

def initialize(dedent_level)
  @dedent_level = dedent_level
  @at_line_begin = true
  @indent_level  = 0
end

Instance Method Details

#dedent(string) ⇒ Object

For a heredoc like

<<-HERE
  a
  b
HERE

this method gets called with “ an” and “ bn”

However, the following heredoc:

<<-HERE
  a\
  b
HERE

calls this method only once with a string “ a\n bn”

This is important because technically it’s a single line, but it has to be concatenated __after__ dedenting.

It has no effect for non-squiggly heredocs, i.e. it simply removes “\n” Of course, lexer could do it but once again: it’s all because of dedenting.



36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
 
# File 'lib/parser/lexer/dedenter.rb', line 36

def dedent(string)
  original_encoding = string.encoding
  # Prevent the following error when processing binary encoded source.
  # "\xC0".split # => ArgumentError (invalid byte sequence in UTF-8)
  lines = string.force_encoding(Encoding::BINARY).split("\\\n")
  if lines.length == 1
    # If the line continuation sequence was found but there is no second
    # line, it was not really a line continuation and must be ignored.
    lines = [string.force_encoding(original_encoding)]
  else
    lines.map! {|s| s.force_encoding(original_encoding) }
  end

  if @at_line_begin
    lines_to_dedent = lines
  else
    _first, *lines_to_dedent = lines
  end

  lines_to_dedent.each do |line|
    left_to_remove = @dedent_level
    remove = 0

    line.each_char do |char|
      break if left_to_remove <= 0
      case char
      when ?\s
        remove += 1
        left_to_remove -= 1
      when ?\t
        break if TAB_WIDTH * (remove / TAB_WIDTH + 1) > @dedent_level
        remove += 1
        left_to_remove -= TAB_WIDTH
      else
        # no more spaces or tabs
        break
      end
    end

    line.slice!(0, remove)
  end

  string.replace(lines.join)

  @at_line_begin = string.end_with?("\n")
end

#interrupt ⇒ Object 



83
84
85
 
# File 'lib/parser/lexer/dedenter.rb', line 83

def interrupt
  @at_line_begin = false
end