Class: Grammar

Inherits:
Object
  • Object
show all
Defined in:
lib/ruby_grammar_builder/grammar.rb,
lib/ruby_grammar_builder/tokens.rb,
lib/ruby_grammar_builder/grammar_plugin.rb

Overview

Represents a Textmate Grammar

Direct Known Subclasses

ExportableGrammar, ImportGrammar

Constant Summary collapse

@@export_grammars =

This classvariable is part of a private API. You should avoid using this classvariable if possible, as it may be removed or be changed in the future.

A mapping of grammar partials that have been exported

{}
@@linters = 
[]
@@transforms = 
{
    before_pre_linter: [],
    after_pre_linter: [],
    before_post_linter: [],
    after_post_linter: [],
}

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(keys) ⇒ Grammar

Create a new Grammar

Parameters:

  • keys (Hash) —

    The grammar keys

Options Hash (keys):

  • :name (String) —

    The name of the grammar

  • :scope_name (String) —

    The scope_name of teh grammar, must start with source. or text.

  • :version (String, :auto) — default: :auto —

    the version of the grammar, :auto uses the current git commit as the version

  • :patterns (Array) — default: [] —

    ignored, will be replaced with the initial context

  • :repository (Hash) — default: {} —

    ignored, will be replaced by the generated rules

  • all (Object) —

    remaining options will be copied to the grammar without change



117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 117

def initialize(keys)
    required_keys = [:name, :scope_name]
    unless required_keys & keys.keys == required_keys
        puts "Missing one or more of the required grammar keys"
        puts "Missing: #{required_keys - (required_keys & keys.keys)}"
        puts "The required grammar keys are: #{required_keys}"
        raise "See above error"
    end

    @name = keys[:name]
    @scope_name = keys[:scope_name]
    @repository = {}
    @ending = keys[:ending] || @scope_name.split('.').drop(1).join('.')

    keys.delete :name
    keys.delete :scope_name

    # auto versioning, when save_to is called grab the latest git commit or "" if not
    # a git repo
    keys[:version] ||= :auto
    @keys = keys.compact
    return if @scope_name == "export" || @scope_name.start_with?("source.", "text.")

    puts "Warning: grammar scope name should start with `source.' or `text.'"
    puts "Examples: source.cpp text.html text.html.markdown source.js.regexp"
end

Instance Attribute Details

#ending ⇒ Object

Returns the value of attribute ending.



24
25
26
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 24

def ending
  @ending
end

#name ⇒ Object

Returns the value of attribute name.



22
23
24
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 22

def name
  @name
end

#repository ⇒ Object

Returns the value of attribute repository.



21
22
23
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 21

def repository
  @repository
end

#scope_name ⇒ Object

Returns the value of attribute scope_name.



23
24
25
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 23

def scope_name
  @scope_name
end

Class Method Details

.fromTmLanguage(path) ⇒ Grammar

Note:

the imported grammar is write only access to imported keys will raise an error

import an existing grammar from a file

Parameters:

  • path (String) —

    path to a json or plist grammar

Returns:

  • (Grammar) —

    The imported grammar



44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 44

def self.fromTmLanguage(path)
    begin
        import_grammar = JSON.parse File.read(path)
    rescue JSON::ParserError
        require 'plist'
        import_grammar = Plist.parse_xml File.read(path)
    end

    grammar = ImportGrammar.new(
        name: import_grammar["name"],
        scope_name: import_grammar["scopeName"],
        version: import_grammar["version"] || "",
        description: import_grammar["description"] || nil,
        information_for_contributors: import_grammar["information_for_contributors"] || nil,
        fileTypes: import_grammar["fileTypes"] || nil,
    )
    # import "patterns" into @repository[:$initial_context]
    grammar.repository[:$initial_context] = import_grammar["patterns"]
    # import the rest of the repository
    import_grammar["repository"].each do |key, value|
        # repository keys are kept as a hash
        grammar.repository[key.to_sym] = value
    end
    grammar
end

.import(path_or_export) ⇒ ExportableGrammar

Note:

the import is “dynamic”, changes made to the grammar partial after the import wil be reflected in the parent grammar

Import a grammar partial

Parameters:

  • path_or_export (String, ExportableGrammar) —

    the grammar partial or the file in which the grammar partial is declared

Returns:



81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 81

def self.import(path_or_export)
    export = path_or_export
    unless path_or_export.is_a? ExportableGrammar
        # allow for relative paths
        if not Pathname.new(path_or_export).absolute?
            relative_path = File.dirname(caller_locations[0].path)
            if not Pathname.new(relative_path).absolute?
                relative_path = File.join(Dir.pwd,relative_path)
            end
            path_or_export = File.join(relative_path, path_or_export)
        end
        require path_or_export
        resolved = File.expand_path resolve_require(path_or_export)

        export = @@export_grammars.dig(resolved, :grammar)
        unless export.is_a? ExportableGrammar
            raise "#{path_or_export} does not create a Exportable Grammar"
        end
    end

    return export.export
end

.new_exportable_grammar ⇒ ExportableGrammar

Create a new Exportable Grammar (Grammar Partial)

Returns:



31
32
33
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 31

def self.new_exportable_grammar
    ExportableGrammar.new
end

.plugins ⇒ Array<GrammarPlugin>

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Gets all registered plugins

Returns:



148
149
150
 
# File 'lib/ruby_grammar_builder/grammar_plugin.rb', line 148

def self.plugins
    @@linters + @@transforms.values.flatten.map { |v| v[:transform] }
end

.register_linter(linter) ⇒ void

This method returns an undefined value.

Register a linter plugin

Parameters:



108
109
110
 
# File 'lib/ruby_grammar_builder/grammar_plugin.rb', line 108

def self.register_linter(linter)
    @@linters << linter
end

.register_transform(transform, priority = 150) ⇒ void

Note:

The priority controls when a transformation runs in relation to other events in addition to ordering transformations priorities < 100 have their pre transform run before pre linters priorities >= 100 have their pre transform run after pre linters priorities >= 200 do not have their pre_transform function ran priorities < 300 have their post transorm run before post linters priorities >= 300 have their post transorm run before post linters

This method returns an undefined value.

Register a transformation plugin

Parameters:

  • transform (GrammarTransform) —

    the transformation plugin

  • priority (Numeric) (defaults to: 150) —

    an optional priority



128
129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/ruby_grammar_builder/grammar_plugin.rb', line 128

def self.register_transform(transform, priority = 150)
    key = if priority < 100 then :before_pre_linter
          elsif priority < 200 then :after_pre_linter
          elsif priority < 300 then :before_post_linter
          else :after_pre_linter
          end

    @@transforms[key] << {
        priority: priority,
        transform: transform,
    }
end

.remove_plugin(plugin) ⇒ void

This method returns an undefined value.

Removes a plugin whose classname matches plugin

Parameters:

  • plugin (#to_s) —

    The plugin name to remove



159
160
161
162
163
164
165
 
# File 'lib/ruby_grammar_builder/grammar_plugin.rb', line 159

def self.remove_plugin(plugin)
    @@linters.delete_if { |linter| linter.class.to_s == plugin.to_s }
    @@transforms[:before_pre_linter].delete_if { |t| t[:transform].class.to_s == plugin.to_s }
    @@transforms[:after_pre_linter].delete_if { |t| t[:transform].class.to_s == plugin.to_s }
    @@transforms[:before_post_linter].delete_if { |t| t[:transform].class.to_s == plugin.to_s }
    @@transforms[:after_post_linter].delete_if { |t| t[:transform].class.to_s == plugin.to_s }
end

Instance Method Details

#[](key) ⇒ PatternBase, ...

Access a pattern in the grammar

Parameters:

  • key (Symbol) —

    The key the pattern is stored in

Returns:



151
152
153
154
155
156
157
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 151

def [](key)
    if key.is_a?(Regexp)
        tokenMatching(key) # see tokens.rb
    else
        @repository.fetch(key, PlaceholderPattern.new(key))
    end
end

#[]=(key, value) ⇒ PatternBase, ...

Store a pattern

A pattern must be stored in the grammar for it to appear in the final grammar

The special key :$initial_context is the pattern that will be matched at the beginning of the document or whenever the root of the grammar is to be matched

Parameters:

  • key (Symbol) —

    The key to store the pattern in

  • value (PatternBase, Symbol, Array<PatternBase, Symbol>) —

    the pattern to store

Returns:



172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 172

def []=(key, value)
    unless key.is_a? Symbol
        raise "Use symbols not strings" unless key.is_a? Symbol
    end

    if key.to_s.start_with?("$") && !([:$initial_context, :$base, :$self].include? key)
        puts "#{key} is not a valid repository name"
        puts "repository names starting with $ are reserved"
        raise "See above error"
    end

    if key.to_s == "repository"
        puts "#{key} is not a valid repository name"
        puts "the name 'repository' is a reserved name"
        raise "See above error"
    end

    # add it to the repository
    @repository[key] = fixup_value(value)
    @repository[key]
end

#auto_version ⇒ String

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Returns the version information

Returns:

  • (String) —

    The version string to use



514
515
516
517
518
519
520
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 514

def auto_version
    return @keys[:version] unless @keys[:version] == :auto

    `git rev-parse HEAD`.strip
rescue StandardError
    ""
end

#generate(options = {}) ⇒ Hash

Convert the grammar into a hash suitable for exporting to a file

Parameters:

  • inherit_or_embedded (Symbol) —

    Is this grammar being inherited from, or will be embedded, this controls if :$initial_context is mapped to “$base” or “$self”

Returns:

  • (Hash) —

    the generated grammar



298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 298

def generate(options={})
    default = {
        inherit_or_embedded: :embedded,
        should_lint: true,
    }
    options = default.merge(options)
    
    repo = @repository.__deep_clone__
    repo = run_pre_transform_stage(repo, :before_pre_linter)

    if options[:should_lint]
        @@linters.each do |linter|
            repo.each do |_, potential_pattern|
                [potential_pattern].flatten.each do |each_potential_pattern|
                    raise "linting failed, see above error" unless linter.pre_lint(
                        each_potential_pattern,
                        filter_options(
                            linter,
                            each_potential_pattern,
                            grammar: self,
                            repository: repo,
                        ),
                    )
                end
            end
        end
    end

    repo = run_pre_transform_stage(repo, :after_pre_linter)

    convert_initial_context = lambda do |potential_pattern|
        if potential_pattern == :$initial_context
            return (options[:inherit_or_embedded] == :embedded) ? :$self : :$base
        end

        if potential_pattern.is_a? Array
            return potential_pattern.map do |nested_potential_pattern|
                convert_initial_context.call(nested_potential_pattern)
            end
        end

        if potential_pattern.is_a? PatternBase
            return potential_pattern.transform_includes do |each_nested_potential_pattern|
                # transform includes will call this block again if each_* is a patternBase
                if each_nested_potential_pattern.is_a? PatternBase
                    next each_nested_potential_pattern
                end

                convert_initial_context.call(each_nested_potential_pattern)
            end
        end

        return potential_pattern
    end
    repo = repo.transform_values do |each_potential_pattern|
        convert_initial_context.call(each_potential_pattern)
    end

    output = {
        name: @name,
        scopeName: @scope_name,
    }

    to_tag = lambda do |potential_pattern|
        case potential_pattern
        when Array
            return {
                "patterns" => potential_pattern.map do |nested_potential_pattern|
                    to_tag.call(nested_potential_pattern)
                end,
            }
        when Symbol then return {"include" => "#" + potential_pattern.to_s}
        when Hash then return potential_pattern
        when String then return {"include" => potential_pattern}
        when PatternBase then return potential_pattern.to_tag
        else raise "Unexpected value: #{potential_pattern.class}"
        end
    end

    output[:repository] = repo.transform_values do |each_potential_pattern|
        to_tag.call(each_potential_pattern)
    end
    # sort repos by key name
    output[:repository] = Hash[output[:repository].sort_by { |key, _| key.to_s }]

    output[:patterns] = output[:repository][:$initial_context]
    output[:patterns] ||= []
    output[:patterns] = output[:patterns]["patterns"] if output[:patterns].is_a? Hash
    output[:repository].delete(:$initial_context)

    output[:version] = auto_version
    output.merge!(@keys) { |_key, old, _new| old }

    output = run_post_transform_stage(output, :before_pre_linter)
    output = run_post_transform_stage(output, :after_pre_linter)
    output = run_post_transform_stage(output, :before_post_linter)

    @@linters.each do |linter|
        raise "linting failed, see above error" unless linter.post_lint(output)
    end

    output = run_post_transform_stage(output, :after_post_linter)

    Hash[
        output.sort_by do |key, _|
            order = {
                information_for_contributors: 0,
                version: 1,
                name: 2,
                scopeName: 3,
                fileTypes: 4,
                unknown_keys: 5,
                patterns: 6,
                repository: 7,
                uuid: 8,
            }
            next order[key.to_sym] if order.has_key? key.to_sym

            order[:unknown_keys]
        end
    ]
end

#import(path_or_export) ⇒ void

Note:

the import is “dynamic”, changes made to the grammar partial after the import wil be reflected in the parent grammar

This method returns an undefined value.

Import a grammar partial into this grammar

Parameters:

  • path_or_export (String, ExportableGrammar) —

    the grammar partial or the file in which the grammar partial is declared



205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 205

def import(path_or_export)
    
    unless path_or_export.is_a? ExportableGrammar
        relative_path = File.dirname(caller_locations[0].path)
        if not Pathname.new(relative_path).absolute?
            relative_path = File.join(Dir.pwd,relative_path)
        end
        # allow for relative paths
        if not Pathname.new(path_or_export).absolute?
            path_or_export = File.join(relative_path, path_or_export)
        end
    end

    export = Grammar.import(path_or_export)
    export.parent_grammar = self

    # import the repository
    @repository = @repository.merge export.repository do |_key, old_val, new_val|
        [old_val, new_val].flatten.uniq
    end
end

#parseTokenSyntax(argument) ⇒ proc

Note:

The syntax for tokenParsing is simple, there are:

  • ‘adjectives` ex: isAClass

  • the ‘not` operator ex: !isAClass

  • the ‘or` operator ex: isAClass || isAPrimitive

  • the ‘and` operator ex: isAClass && isAPrimitive

  • paraentheses ex: (!isAClass) && isAPrimitive

_ anything matching /[a-zA-Z0-9_]+/ is considered an “adjective” whitespace, including newlines, are removed/ignored all other characters are invalid _ using only an adjective, ex: /isAClass/ means to only include Patterns that have that adjective in their adjective list

convert a regex value into a proc filter used to select patterns

Parameters:

  • argument (Regexp) —

    A value that uses the tokenParsing syntax (explained below)

Returns:

  • (proc) —

    a function that accepts a Pattern as input, and returns a boolean of whether or not that pattern should be included



64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
 
# File 'lib/ruby_grammar_builder/tokens.rb', line 64

def parseTokenSyntax(argument)
    # validate input type
    if !argument.is_a?(Regexp)
        raise <<~HEREDOC
            
            
            Trying to call parseTokenSyntax() but the argument isn't Regexp its #{argument.class}
            value: #{argument}
        HEREDOC
    end
    # just remove the //'s from the string
    regex_content = argument.inspect[1...-1]
    
    # remove all invalid characters, make sure length didn't change
    invalid_characters_removed = regex_content.gsub(/[^a-zA-Z0-9_&|\(\)! \n]/, "")
    if invalid_characters_removed.length != regex_content.length
        raise <<~HEREDOC
            
            
            It appears the tokenSyntax #{argument.inspect} contains some invalid characters
            with invalid characters: #{regex_content.inspect}
            without invalid characters: #{invalid_characters_removed.inspect}
        HEREDOC
    end
    
    # find broken syntax
    if regex_content =~ /[a-zA-Z0-9_]+\s+[a-zA-Z0-9_]+/
        raise <<~HEREDOC
            
            Inside a tokenSyntax: #{argument.inspect}
            this part of the syntax is invalid: #{$&.inspect}
            (theres a space between two adjectives)
            My guess is that it was half-edited
            or an accidental space was added
        HEREDOC
    end
    
    # convert all adjectives into inclusion checks
    regex_content.gsub!(/\s+/," ")
    regex_content.gsub!(/[a-zA-Z0-9_]+/, 'pattern.arguments[:adjectives].include?(:\0)')
    # convert it into a proc
    return ->(pattern) do
        puts "regex_content is: #{regex_content} "
        eval(regex_content) if pattern.is_a?(PatternBase) && pattern.arguments[:adjectives].is_a?(Array)
    end
end

#run_post_transform_stage(output, stage) ⇒ Hash

Runs a set of post transformations

Parameters:

  • output (Hash) —

    The generated grammar

  • stage (Symbol) —

    the stage to run

Returns:

  • (Hash) —

    The modified grammar



280
281
282
283
284
285
286
287
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 280

def run_post_transform_stage(output, stage)
    @@transforms[stage]
        .sort { |a, b| a[:priority] <=> b[:priority] }
        .map { |a| a[:transform] }
        .each { |transform| output = transform.post_transform(output) }

    output
end

#run_pre_transform_stage(repository, stage) ⇒ Hash

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Runs a set of pre transformations

Parameters:

  • repository (Hash) —

    The repository

  • stage (:before_pre_linter, :after_pre_linter) —

    the stage to run

Returns:

  • (Hash) —

    the modified repository



237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 237

def run_pre_transform_stage(repository, stage)
    @@transforms[stage]
        .sort { |a, b| a[:priority] <=> b[:priority] }
        .map { |a| a[:transform] }
        .each do |transform|
            repository = repository.transform_values do |potential_pattern|
                if potential_pattern.is_a? Array
                    potential_pattern.map do |each|
                        transform.pre_transform(
                            each,
                            filter_options(
                                transform,
                                each,
                                grammar: self,
                                repository: repository,
                            ),
                        )
                    end
                else
                    transform.pre_transform(
                        potential_pattern,
                        filter_options(
                            transform,
                            potential_pattern,
                            grammar: self,
                            repository: repository,
                        ),
                    )
                end
            end
        end

    repository
end

#save_to(options) ⇒ void

Note:

all keys except :directory is optional

Note:

:directory is optional if both :tag_dir and :syntax_dir are specified

Note:

currently :vscode is an alias for :json

Note:

currently :textmate, :tm_language, and :xml are aliases for :plist

Note:

later the aliased :syntax_type choices may enable compatibility features

This method returns an undefined value.

Save the grammar to a path

Parameters:

  • options (Hash) —

    options to save_to

Options Hash (options):

  • :inherit_or_embedded (Object) — default: :embedded —

    see #generate

  • :generate_tags (Boolean) — default: true —

    generate a list of all :tag_ass

  • :directory (String) —

    the location to generate the files

  • :tag_dir (String) — default: File.join(options[:directory],"language_tags") —

    the directory to generate language tags in

  • :syntax_dir (String) — default: File.join(options[:directory],"syntaxes") —

    the directory to generate the syntax file in

  • :syntax_format (:json, :vscode, :plist, :textmate, :tm_language, :xml) — default: :json —

    The format to generate the syntax file in

  • :syntax_name (String) — default: "#{@name}.tmLanguage" —

    the name of the syntax file to generate without the extension

  • :tag_name (String) — default: "#{@name}-scopes.txt" —

    the name of the tag list file to generate without the extension



447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
 
# File 'lib/ruby_grammar_builder/grammar.rb', line 447

def save_to(options)
    options[:directory] ||= "."
    
    # make the path absolute 
    absolute_path_from_caller = File.dirname(caller_locations[0].path)
    if not Pathname.new(absolute_path_from_caller).absolute?
        absolute_path_from_caller = File.join(Dir.pwd,absolute_path_from_caller)
    end
    if not Pathname.new(options[:directory]).absolute?
        options[:directory] = File.join(absolute_path_from_caller, options[:directory])
    end
    
    default = {
        inherit_or_embedded: :embedded,
        generate_tags: true,
        syntax_format: :json,
        syntax_name: "#{@ending}",
        syntax_dir: options[:directory],
        tag_dir: options[:directory],
        should_lint: true,
    }
    options = default.merge(options)
    options[:tag_name] ||= options[:syntax_name] + "_scopes.txt"
    
    output = generate(options)

    if [:json, :vscode].include? options[:syntax_format]
        file_name = File.join(
            options[:syntax_dir],
            "#{options[:syntax_name]}.tmLanguage.json",
        )
        out_file = File.open(file_name, "w")
        out_file.write(JSON.pretty_generate(output))
        out_file.close
    elsif [:plist, :textmate, :tm_language, :xml].include? options[:syntax_format]
        require 'plist'
        file_name = File.join(
            options[:syntax_dir],
            options[:syntax_name],
        )
        out_file = File.open(file_name, "w")
        out_file.write(Plist::Emit.dump(output))
        out_file.close
    else
        puts "unexpected syntax format #{options[:syntax_format]}"
        puts "expected one of [:json, :vscode, :plist, :textmate, :tm_language, :xml]"
        raise "see above error"
    end

    return unless options[:generate_tags]

    file_name = File.join(
        options[:tag_dir],
        options[:tag_name],
    )
    new_file = File.open(file_name, "w")
    new_file.write(get_tags(output).to_a.sort.join("\n"))
    new_file.close
end

#tokenMatching(token_pattern) ⇒ TokenPattern

Note:

The syntax for tokenParsing is simple, there are:

  • ‘adjectives` ex: isAClass

  • the ‘not` operator ex: !isAClass

  • the ‘or` operator ex: isAClass || isAPrimitive

  • the ‘and` operator ex: isAClass && isAPrimitive

  • paraentheses ex: (!isAClass) && isAPrimitive

_ anything matching /[a-zA-Z0-9_]+/ is considered an “adjective” whitespace, including newlines, are removed/ignored all other characters are invalid _ using only an adjective, ex: /isAClass/ means to only include Patterns that have that adjective in their adjective list

convert a regex value into a proc filter used to select patterns

Parameters:

  • argument (Regexp) —

    A value that uses the tokenParsing syntax (explained below)

Returns:



30
31
32
33
34
35
36
37
38
 
# File 'lib/ruby_grammar_builder/tokens.rb', line 30

def tokenMatching(token_pattern)
    # create the normal pattern that will act as a placeholder until the very end
    token_pattern = TokenPattern.new({
        match: /(?#tokens)/,
        pattern_filter: parseTokenSyntax(token_pattern),
    })
    # tell it what it needs to select-later
    return token_pattern
end