Class: YARD::Parser::SourceParser

Inherits:
Object
  • Object
show all
Defined in:
lib/yard/parser/source_parser.rb

Overview

Responsible for parsing a source file into the namespace. Parsing also invokes handlers to process the parsed statements and generate any code objects that may be recognized.

Custom Parsers

SourceParser allows custom parsers to be registered and called when a certain filetype is recognized. To register a parser and hook it up to a set of file extensions, call SourceParser.register_parser_type

See Also:

Class Attribute Summary collapse

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(parser_type = SourceParser.parser_type, load_order_errors = false) ⇒ SourceParser

Creates a new parser object for code parsing with a specific parser type.

Parameters:

  • parser_type (Symbol) (defaults to: SourceParser.parser_type) —

    the parser type to use

  • load_order_errors (Boolean) (defaults to: false) —

    whether or not to raise the LoadOrderError



186
187
188
189
190
 
# File 'lib/yard/parser/source_parser.rb', line 186

def initialize(parser_type = SourceParser.parser_type, load_order_errors = false)
  @load_order_errors = load_order_errors
  @file = '(stdin)'
  self.parser_type = parser_type
end

Class Attribute Details

.parser_type ⇒ Symbol

Returns the default parser type (defaults to :ruby).

Returns:

  • (Symbol) —

    the default parser type (defaults to :ruby)



36
37
38
 
# File 'lib/yard/parser/source_parser.rb', line 36

def parser_type
  @parser_type
end

.parser_type_extensions=(value) ⇒ Object 



118
 
# File 'lib/yard/parser/source_parser.rb', line 118

def parser_type_extensions=(value) @@parser_type_extensions = value end

.parser_types=(value) ⇒ Object 



110
 
# File 'lib/yard/parser/source_parser.rb', line 110

def parser_types=(value) @@parser_types = value end

Instance Attribute Details

#file ⇒ Object (readonly)

The filename being parsed by the parser.



176
177
178
 
# File 'lib/yard/parser/source_parser.rb', line 176

def file
  @file
end

#parser_type ⇒ Object

The parser type associated with the parser instance. This should be set by the constructor.



180
181
182
 
# File 'lib/yard/parser/source_parser.rb', line 180

def parser_type
  @parser_type
end

Class Method Details

.parse(paths = ["lib/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) ⇒ Object

Parses a path or set of paths

Parameters:

  • paths (String, Array<String>) (defaults to: ["lib/**/*.rb", "ext/**/*.c"]) —

    a path, glob, or list of paths to parse

  • excluded (Array<String, Regexp>) (defaults to: []) —

    a list of excluded path matchers

  • level (Fixnum) (defaults to: log.level) —

    the logger level to use during parsing. See Logger

Returns:

  • the parser object that was used to parse the source.



50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
 
# File 'lib/yard/parser/source_parser.rb', line 50

def parse(paths = ["lib/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level)
  log.debug("Parsing #{paths.inspect} with `#{parser_type}` parser")
  excluded = excluded.map do |path|
    case path
    when Regexp; path
    else Regexp.new(path.to_s, Regexp::IGNORECASE)
    end
  end
  files = [paths].flatten.
    map {|p| File.directory?(p) ? "#{p}/**/*.{rb,c}" : p }.
    map {|p| p.include?("*") ? Dir[p] : p }.flatten.
    reject {|p| !File.file?(p) || excluded.any? {|re| p =~ re } }

  log.enter_level(level) do
    parse_in_order(*files.uniq)
  end
end

.parse_string(content, ptype = parser_type) ⇒ Object

Parses a string content

Parameters:

  • content (String) —

    the block of code to parse

  • ptype (Symbol) (defaults to: parser_type) —

    the parser type to use. See parser_type.

Returns:

  • the parser object that was used to parse content



73
74
75
 
# File 'lib/yard/parser/source_parser.rb', line 73

def parse_string(content, ptype = parser_type)
  new(ptype).parse(StringIO.new(content))
end

.parser_type_for_extension(extension) ⇒ Symbol

Finds a parser type that is registered for the extension. If no type is found, the default Ruby type is returned.

Returns:

  • (Symbol) —

    the parser type to be used for the extension

Since:

  • 0.5.6



125
126
127
128
129
130
 
# File 'lib/yard/parser/source_parser.rb', line 125

def parser_type_for_extension(extension)
  type = parser_type_extensions.find do |t, exts|
    [exts].flatten.any? {|ext| ext === extension }
  end
  validated_parser_type(type ? type.first : :ruby)
end

.register_parser_type(type, parser_klass, extensions = nil) ⇒ void

This method returns an undefined value.

Registers a new parser type.

Examples:

Registering a parser for “java” files

SourceParser.register_parser_type :java, JavaParser, 'java'

Parameters:

  • type (Symbol) —

    a symbolic name for the parser type

  • parser_klass (Base) —

    a class that implements parsing and tokenization

  • extensions (Array<String>, String, Regexp) (defaults to: nil) —

    a list of extensions or a regex to match against the file extension

See Also:



96
97
98
99
100
101
102
 
# File 'lib/yard/parser/source_parser.rb', line 96

def register_parser_type(type, parser_klass, extensions = nil)
  unless Base > parser_klass
    raise ArgumentError, "expecting parser_klass to be a subclass of YARD::Parser::Base"
  end
  parser_type_extensions[type.to_sym] = extensions if extensions
  parser_types[type.to_sym] = parser_klass
end

.tokenize(content, ptype = parser_type) ⇒ Array

Tokenizes but does not parse the block of code

Parameters:

  • content (String) —

    the block of code to tokenize

  • ptype (Symbol) (defaults to: parser_type) —

    the parser type to use. See parser_type.

Returns:

  • (Array) —

    a list of tokens



82
83
84
 
# File 'lib/yard/parser/source_parser.rb', line 82

def tokenize(content, ptype = parser_type)
  new(ptype).tokenize(content)
end

Instance Method Details

#parse(content = __FILE__) ⇒ Object?

The main parser method. This should not be called directly. Instead, use the class methods parse and parse_string.

Parameters:

  • content (String, #read, Object) (defaults to: __FILE__) —

    the source file to parse

Returns:

  • (Object, nil) —

    the parser object used to parse the source



197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
 
# File 'lib/yard/parser/source_parser.rb', line 197

def parse(content = __FILE__)
  case content
  when String
    @file = File.cleanpath(content)
    content = convert_encoding(File.read_binary(file))
    checksum = Registry.checksum_for(content)
    return if Registry.checksums[file] == checksum

    if Registry.checksums.has_key?(file)
      log.info "File '#{file}' was modified, re-processing..."
    end
    Registry.checksums[@file] = checksum
    self.parser_type = parser_type_for_filename(file)
  else
    content = content.read if content.respond_to? :read
  end
  
  @parser = parser_class.new(content, file)
  @parser.parse
  post_process
  @parser
rescue ArgumentError, NotImplementedError => e
  log.warn("Cannot parse `#{file}': #{e.message}")
rescue ParserSyntaxError => e
  log.warn(e.message.capitalize)
end

#tokenize(content) ⇒ Array

Tokenizes but does not parse the block of code using the current #parser_type

Parameters:

  • content (String) —

    the block of code to tokenize

Returns:

  • (Array) —

    a list of tokens



228
229
230
231
 
# File 'lib/yard/parser/source_parser.rb', line 228

def tokenize(content)
  @parser = parser_class.new(content, file)
  @parser.tokenize
end