Top Level Namespace

Defined Under Namespace

Modules: Generated Classes: AddEnding, AlternationOperator, BailoutTransform, ConcatOperator, Enumerator, ExportableGrammar, FixRepeatedTagAs, FlatIncludes, Grammar, GrammarLinter, GrammarPlugin, GrammarTransform, ImportGrammar, IncludesThenTagAs, LegacyPattern, LookAroundPattern, MatchResultOfPattern, MaybePattern, OneOfPattern, OneOrMoreOfPattern, OrPattern, PatternBase, PatternRange, PlaceholderPattern, RecursivelyMatchPattern, RegexOperator, RepeatablePattern, ResolvePlaceholders, RunPatternTests, StandardNaming, StartMatchEmpty, String, TokenPattern, UnusedUnresolvedLinter, ZeroOrMoreOfPattern

Constant Summary collapse

Pattern =

Pattern is a class alias for RepeatablePattern

class Pattern # <- encase people ctrl+F for “class Pattern”

RepeatablePattern

Instance Method Summary collapse

Instance Method Details

#break_left(arr) {|RegexOperator, String| ... } ⇒ Array<(Array,Array)>

Split arr in two Walks arr from left to right and splits it on the first element that the block returns false this means that the block returned true for all lements in the left half and false for the first element of the right half the order of elements is not changed

Parameters:

  • arr (Array) —

    The array to break

Yields:

Returns:

  • (Array<(Array,Array)>) —

    The two halfs



17
18
19
20
21
22
 
# File 'lib/ruby_grammar_builder/regex_operator.rb', line 17

def break_left(arr)
    left = arr.take_while do |elem|
        next !(yield elem)
    end
    [left, arr[(left.length)..-1]]
end

#break_right(arr) ⇒ Object

(@see break_left) Walks the array from right to left spliting where the block returns false



28
29
30
31
32
33
 
# File 'lib/ruby_grammar_builder/regex_operator.rb', line 28

def break_right(arr)
    right = arr.reverse.take_while do |elem|
        next !(yield elem)
    end.reverse
    [arr[0..-(right.length+1)], right]
end

#filter_options(plugin, pattern, default) ⇒ Hash

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Filters a PatternBase#original_arguments to just the options required for a plugin

Parameters:

  • plugin (GrammarPlugin) —

    The plugin to filter options

  • pattern (PatternBase, Symbol, Hash) —

    the pattern with options to filter

  • default (Hash) —

    the default options to supply to the plugin

Returns:

  • (Hash) —

    the filtered options



179
180
181
182
183
184
185
 
# File 'lib/ruby_grammar_builder/grammar_plugin.rb', line 179

def filter_options(plugin, pattern, default)
    options = {}
    if pattern.is_a? PatternBase
        options = pattern.original_arguments.select { |k| plugin.class.options.include? k }
    end
    options.merge(default)
end

#fixup_value(value) ⇒ PatternBase, ...

Fixes value to be either a PatternBase, Symbol, or Array of either

Parameters:

  • value (*) —

    The value to fixup

Returns:



138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
 
# File 'lib/ruby_grammar_builder/util.rb', line 138

def fixup_value(value)
    is_array = value.is_a? Array
    # ensure array is flat and only contains patterns or symbols
    value = [value].flatten.map do |v|
        next v if v.is_a? Symbol

        if v.is_a? String
            next v if v.start_with?("source.", "text.", "$")
        end

        if v.is_a? Hash
            # check for an implicit legacy pattern
            legacy_keys = [
                :name,
                :contentName,
                :begin,
                :end,
                :while,
                :comment,
                :disabled,
                :patterns,
            ]
            v = LegacyPattern.new(v) unless (v.keys.map(&:to_sym) & legacy_keys).empty?
        end

        v = Pattern.new(v) unless v.is_a? PatternBase
        v
    end

    value = value[0] unless is_array
    value
end

#get_tags(output) ⇒ Set<String>

Converts an output into a set of tags

Parameters:

  • output (Hash) —

    output of Grammar#generate

Returns:

  • (Set<String>) —

    The tags in the grammar



204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
 
# File 'lib/ruby_grammar_builder/util.rb', line 204

def get_tags(output)
    repository = output[:repository]
    repository[:$initial_context] = {patterns: output[:patterns]}
    tags = Set.new
    add_tags = lambda do |rule|
        rule = rule.transform_keys(&:to_sym)
        tags.merge(rule[:name].split(" ")) if rule[:name]
        tags.merge(rule[:contentName].split(" ")) if rule[:contentName]

        rule[:patterns]&.each { |p| add_tags.call(p) }
        rule[:captures]&.values&.each { |p| add_tags.call(p) }
        rule[:beginCaptures]&.values&.each { |p| add_tags.call(p) }
        rule[:endCaptures]&.values&.each { |p| add_tags.call(p) }
        rule[:whileCaptures]&.values&.each { |p| add_tags.call(p) }
    end

    repository.values.each { |p| add_tags.call(p) }

    tags
end

#lookAheadFor(pattern) ⇒ PatternBase

Equivalent to lookAround with type set to :lookAheadFor

Parameters:

  • pattern (Hash) —

    pattern constructor

Returns:



25
26
27
28
29
30
31
32
 
# File 'lib/ruby_grammar_builder/pattern_extensions/look_ahead_for.rb', line 25

def lookAheadFor(pattern)
    if pattern.is_a? Hash
        pattern[:type] = :lookAheadFor
    else
        pattern = {match: pattern, type: :lookAheadFor}
    end
    lookAround(pattern)
end

#lookAheadToAvoid(pattern) ⇒ PatternBase

Equivalent to lookAround with type set to :lookAheadToAvoid

Parameters:

  • pattern (Hash) —

    pattern constructor

Returns:



152
153
154
155
156
157
158
159
 
# File 'lib/ruby_grammar_builder/pattern_extensions/lookaround_pattern.rb', line 152

def lookAheadToAvoid(pattern)
    if pattern.is_a? Hash
        pattern[:type] = :lookAheadToAvoid
    else
        pattern = {match: pattern, type: :lookAheadToAvoid}
    end
    lookAround(pattern)
end

#lookAround(pattern) ⇒ PatternBase

Looks around for the pattern

option [Symbol] :type the look-around type

can be one of :lookAheadFor, :lookAheadToAvoid, :lookBehindFor, :lookBehindToAvoid

Parameters:

  • pattern (Hash) —

    pattern constructor

Returns:



125
126
127
 
# File 'lib/ruby_grammar_builder/pattern_extensions/lookaround_pattern.rb', line 125

def lookAround(pattern)
    LookAroundPattern.new(pattern)
end

#lookBehindFor(pattern) ⇒ PatternBase

Equivalent to lookAround with type set to :lookBehindFor

Parameters:

  • pattern (Hash) —

    pattern constructor

Returns:



24
25
26
27
28
29
30
31
 
# File 'lib/ruby_grammar_builder/pattern_extensions/look_behind_for.rb', line 24

def lookBehindFor(pattern)
    if pattern.is_a? Hash
        pattern[:type] = :lookBehindFor
    else
        pattern = {match: pattern, type: :lookBehindFor}
    end
    lookAround(pattern)
end

#lookBehindToAvoid(pattern) ⇒ PatternBase

Equivalent to lookAround with type set to :lookBehindToAvoid

Parameters:

  • pattern (Hash) —

    pattern constructor

Returns:



132
133
134
135
136
137
138
139
 
# File 'lib/ruby_grammar_builder/pattern_extensions/lookaround_pattern.rb', line 132

def lookBehindToAvoid(pattern)
    if pattern.is_a? Hash
        pattern[:type] = :lookBehindToAvoid
    else
        pattern = {match: pattern, type: :lookBehindToAvoid}
    end
    lookAround(pattern)
end

#matchResultOf(reference) ⇒ PatternBase

Match the result of some other pattern

Parameters:

  • reference (String) —

    a reference to match the result of

Returns:



65
66
67
 
# File 'lib/ruby_grammar_builder/pattern_extensions/match_result_of.rb', line 65

def matchResultOf(reference)
    MatchResultOfPattern.new(reference)
end

#maybe(pattern) ⇒ PatternBase

Optionally match pattern

Parameters:

Returns:



48
49
50
 
# File 'lib/ruby_grammar_builder/pattern_extensions/maybe.rb', line 48

def maybe(pattern)
    MaybePattern.new(pattern)
end

#oneOf(patterns) ⇒ PatternBase

Match one of the supplied patterns

Parameters:

Returns:



105
106
107
 
# File 'lib/ruby_grammar_builder/pattern_extensions/one_of.rb', line 105

def oneOf(patterns)
    OneOfPattern.new(patterns)
end

#oneOrMoreOf(pattern) ⇒ PatternBase

Match pattern one or more times

Parameters:

Returns:



40
41
42
 
# File 'lib/ruby_grammar_builder/pattern_extensions/one_or_more_of.rb', line 40

def oneOrMoreOf(pattern)
    OneOrMoreOfPattern.new(pattern)
end

#placeholder(placeholder) ⇒ PatternBase

Match a pattern that does not exist yet

Parameters:

  • placeholder (Symbol) —

    the name of the pattern to match

Returns:



100
101
102
 
# File 'lib/ruby_grammar_builder/pattern_extensions/placeholder.rb', line 100

def placeholder(placeholder)
    PlaceholderPattern.new(placeholder)
end

#recursivelyMatch(reference) ⇒ PatternBase

Recursively match some outer pattern

Parameters:

  • reference (String) —

    a reference to an outer pattern

Returns:



74
75
76
 
# File 'lib/ruby_grammar_builder/pattern_extensions/recursively_match.rb', line 74

def recursivelyMatch(reference)
    RecursivelyMatchPattern.new(reference)
end

#resolve_require(path) ⇒ String

Note:

this assumes path was successfully required previously

Determine the absolute path that a require statement resolves to

Parameters:

  • path (String) —

    the path to resolve

Returns:

  • (String) —

    the resolved path



180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
 
# File 'lib/ruby_grammar_builder/util.rb', line 180

def resolve_require(path)
    path = Pathname.new path
    return path.to_s if path.absolute? && path.extname != ""

    return path.dirname.glob("#{path.basename}.{rb,so,dll}")[0].to_s if path.absolute?

    $LOAD_PATH.each do |p|
        test_path = Pathname.new(p).join(path)
        return test_path.to_s if path.extname != "" && test_path.exist?

        test_paths = test_path.dirname.glob("#{test_path.basename}.{rb,so,dll}")
        return test_paths[0].to_s unless test_paths.empty?
    end

    ""
end

#string_single_entity?(regex_string) ⇒ Boolean

Note:

single entity means that for the purposes of modification, the expression is atomic, for example if appending a * to the end of regex_string matches only a part of regex string multiple times then it is not a single_entity

determines if a regex string is a single entity

Parameters:

  • regex_string (String) —

    a string representing a regular expression, without the forward slash “/” at the beginning and

Returns:

  • (Boolean) —

    if the string represents an single regex entity



53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
 
# File 'lib/ruby_grammar_builder/util.rb', line 53

def string_single_entity?(regex_string)
    normal_char = '[a-zA-Z0-9_\-@&%#\'"<>=\/\.,`~\s;:!]'
    escape_sequence = '\\\\[\w\W]'
    character_class_that_doesnt_contain_bracket = '\[[^\]]*\]'
    # normal char
    if regex_string =~ /^#{normal_char}$/
        return true
    end
    # escape sequence (all are valid, even stuff like \@ ("\\@") or "\\" + "\n" )
    if regex_string =~ /^#{escape_sequence}$/
        return true
    end
    # character class that doesn't contain ]
    if regex_string =~ /^#{character_class_that_doesnt_contain_bracket}$/
        return true
    end
    
    # fail if more than one of any of the above
    if regex_string =~ /^(#{normal_char}|#{escape_sequence}|#{character_class_that_doesnt_contain_bracket}){2,}$/
        return false
    end
    
    # 
    # more complicated cases
    # 
    escaped = false
    in_set = false
    depth = 0
    regex_string.each_char.with_index do |c, index|
        # allow the first character to be at depth 0
        # NOTE: this automatically makes a single char regexp a single entity
        return false if depth == 0 && index != 0

        if escaped
            escaped = false
            next
        end
        if c == '\\'
            escaped = true
            next
        end
        if in_set
            if c == ']'
                in_set = false
                depth -= 1
            end
            next
        end
        case c
        when "(" then depth += 1
        when ")" then depth -= 1
        when "["
            depth += 1
            in_set = true
        end
    end
    # sanity check
    if depth != 0 or escaped or in_set
        puts "Internal error: when determining if a Regexp is a single entity"
        puts "an unexpected sequence was found. This is a bug with the gem."
        puts "This will not effect the validity of the produced grammar"
        puts "Regexp: #{inspect} depth: #{depth} escaped: #{escaped} in_set: #{in_set}"
        return false
    end
    true
end

#with_no_warnings ⇒ void

This method returns an undefined value.

Disables warnings for the block



9
10
11
12
13
14
15
 
# File 'lib/ruby_grammar_builder/util.rb', line 9

def with_no_warnings
    old_verbose = $VERBOSE
    $VERBOSE = nil
    yield
ensure
    $VERBOSE = old_verbose
end

#wrap_with_anchors(pat) ⇒ PatternBase

Wraps a pattern in start and end anchors

Parameters:

Returns:



127
128
129
 
# File 'lib/ruby_grammar_builder/util.rb', line 127

def wrap_with_anchors(pat)
    Pattern.new(/^/).then(Pattern.new(pat)).then(/$/)
end

#zeroOrMoreOf(pattern) ⇒ PatternBase

Match pattern zero or more times

Parameters:

Returns:



48
49
50
 
# File 'lib/ruby_grammar_builder/pattern_extensions/zero_or_more_of.rb', line 48

def zeroOrMoreOf(pattern)
    ZeroOrMoreOfPattern.new(pattern)
end