Class: YARD::Parser::SourceParser
- Inherits:
-
Object
- Object
- YARD::Parser::SourceParser
- Defined in:
- lib/yard/parser/source_parser.rb
Overview
Responsible for parsing a source file into the namespace. Parsing also invokes handlers to process the parsed statements and generate any code objects that may be recognized.
Custom Parsers
SourceParser allows custom parsers to be registered and called when a certain filetype is recognized. To register a parser and hook it up to a set of file extensions, call SourceParser.register_parser_type
Constant Summary collapse
- SHEBANG_LINE =
/\A\s*#!\S+/
- ENCODING_LINE =
/\A(?:\s*#*!.*\r?\n)?\s*#+.*coding\s*[:=]{1,2}\s*(\S+)/i
- ENCODING_BYTE_ORDER_MARKS =
Byte order marks for various encodings
{ 'utf-8' => "\xEF\xBB\xBF", # Not yet supported #'utf-16be' => "\xFE\xFF", #'utf-16le' => "\xFF\xFE", #'utf-32be' => "\x00\x00\xFF\xFE", #'utf-32le' => "\xFF\xFE", }
Parser Callbacks collapse
-
#contents ⇒ String
readonly
The contents of the file to be parsed.
-
#file ⇒ String
readonly
The filename being parsed by the parser.
-
#globals ⇒ OpenStruct
readonly
An open struct containing arbitrary global state shared between files and handlers.
-
#parser_type ⇒ Symbol
readonly
The parser type associated with the parser instance.
Class Attribute Summary collapse
-
.parser_type ⇒ Symbol
The default parser type (defaults to :ruby).
- .parser_type_extensions ⇒ Object writeonly
- .parser_types ⇒ Object writeonly
Parser Callbacks collapse
-
.after_parse_file {|parser| ... } ⇒ Proc
Registers a callback to be called after an individual file is parsed.
-
.after_parse_file_callbacks ⇒ Array<Proc>
The list of callbacks to be called after parsing a file.
-
.after_parse_list {|files, globals| ... } ⇒ Proc
Registers a callback to be called after a list of files is parsed via SourceParser.parse.
-
.after_parse_list_callbacks ⇒ Array<Proc>
The list of callbacks to be called after parsing a list of files.
-
.before_parse_file {|parser| ... } ⇒ Proc
Registers a callback to be called before an individual file is parsed.
-
.before_parse_file_callbacks ⇒ Array<Proc>
The list of callbacks to be called before parsing a file.
-
.before_parse_list {|files, globals| ... } ⇒ Proc
Registers a callback to be called before a list of files is parsed via SourceParser.parse.
-
.before_parse_list_callbacks ⇒ Array<Proc>
The list of callbacks to be called before parsing a list of files.
-
#initialize(parser_type = SourceParser.parser_type, load_order_errors = false, globals = nil) ⇒ SourceParser
constructor
Creates a new parser object for code parsing with a specific parser type.
-
#parse(content = __FILE__) ⇒ Object?
The main parser method.
-
#tokenize(content) ⇒ Array
Tokenizes but does not parse the block of code using the current #parser_type.
Class Method Summary collapse
-
.parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) ⇒ void
Parses a path or set of paths.
-
.parse_string(content, ptype = parser_type) ⇒ Object
Parses a string
content
. -
.parser_type_for_extension(extension) ⇒ Symbol
Finds a parser type that is registered for the extension.
-
.register_parser_type(type, parser_klass, extensions = nil) ⇒ void
Registers a new parser type.
-
.tokenize(content, ptype = parser_type) ⇒ Array
Tokenizes but does not parse the block of code.
Constructor Details
#initialize(parser_type = SourceParser.parser_type, load_order_errors = false, globals = nil) ⇒ SourceParser
Creates a new parser object for code parsing with a specific parser type.
402 403 404 405 406 407 |
# File 'lib/yard/parser/source_parser.rb', line 402 def initialize(parser_type = SourceParser.parser_type, load_order_errors = false, globals = nil) @load_order_errors = load_order_errors @file = '(stdin)' @globals = globals || OpenStruct.new self.parser_type = parser_type end |
Class Attribute Details
.parser_type ⇒ Symbol
Returns the default parser type (defaults to :ruby).
61 62 63 |
# File 'lib/yard/parser/source_parser.rb', line 61 def parser_type @parser_type end |
.parser_type_extensions=(value) ⇒ Object
143 |
# File 'lib/yard/parser/source_parser.rb', line 143 def parser_type_extensions=(value) @@parser_type_extensions = value end |
.parser_types=(value) ⇒ Object
135 |
# File 'lib/yard/parser/source_parser.rb', line 135 def parser_types=(value) @@parser_types = value end |
Instance Attribute Details
#contents ⇒ String (readonly)
Returns the contents of the file to be parsed.
395 396 397 |
# File 'lib/yard/parser/source_parser.rb', line 395 def contents @contents end |
#file ⇒ String (readonly)
Returns the filename being parsed by the parser.
382 383 384 |
# File 'lib/yard/parser/source_parser.rb', line 382 def file @file end |
#globals ⇒ OpenStruct (readonly)
Returns an open struct containing arbitrary global state shared between files and handlers.
391 392 393 |
# File 'lib/yard/parser/source_parser.rb', line 391 def globals @globals end |
#parser_type ⇒ Symbol
Returns the parser type associated with the parser instance. This should be set by the constructor.
386 387 388 |
# File 'lib/yard/parser/source_parser.rb', line 386 def parser_type @parser_type end |
Class Method Details
.after_parse_file {|parser| ... } ⇒ Proc
Registers a callback to be called after an individual file is parsed. The block passed to this method will be called on subsequent parse calls.
To register a callback that is called after the entire list of files is processed, see after_parse_list.
303 304 305 |
# File 'lib/yard/parser/source_parser.rb', line 303 def after_parse_file(&block) after_parse_file_callbacks << block end |
.after_parse_file_callbacks ⇒ Array<Proc>
Returns the list of callbacks to be called after parsing a file. Should only be used for testing.
331 332 333 |
# File 'lib/yard/parser/source_parser.rb', line 331 def after_parse_file_callbacks @after_parse_file_callbacks ||= [] end |
.after_parse_list {|files, globals| ... } ⇒ Proc
Registers a callback to be called after a list of files is parsed via parse. The block passed to this method will be called on subsequent parse calls.
237 238 239 |
# File 'lib/yard/parser/source_parser.rb', line 237 def after_parse_list(&block) after_parse_list_callbacks << block end |
.after_parse_list_callbacks ⇒ Array<Proc>
Returns the list of callbacks to be called after parsing a list of files. Should only be used for testing.
317 318 319 |
# File 'lib/yard/parser/source_parser.rb', line 317 def after_parse_list_callbacks @after_parse_list_callbacks ||= [] end |
.before_parse_file {|parser| ... } ⇒ Proc
Registers a callback to be called before an individual file is parsed. The block passed to this method will be called on subsequent parse calls.
To register a callback that is called before the entire list of files is processed, see before_parse_list.
274 275 276 |
# File 'lib/yard/parser/source_parser.rb', line 274 def before_parse_file(&block) before_parse_file_callbacks << block end |
.before_parse_file_callbacks ⇒ Array<Proc>
Returns the list of callbacks to be called before parsing a file. Should only be used for testing.
324 325 326 |
# File 'lib/yard/parser/source_parser.rb', line 324 def before_parse_file_callbacks @before_parse_file_callbacks ||= [] end |
.before_parse_list {|files, globals| ... } ⇒ Proc
Registers a callback to be called before a list of files is parsed via parse. The block passed to this method will be called on subsequent parse calls.
213 214 215 |
# File 'lib/yard/parser/source_parser.rb', line 213 def before_parse_list(&block) before_parse_list_callbacks << block end |
.before_parse_list_callbacks ⇒ Array<Proc>
Returns the list of callbacks to be called before parsing a list of files. Should only be used for testing.
310 311 312 |
# File 'lib/yard/parser/source_parser.rb', line 310 def before_parse_list_callbacks @before_parse_list_callbacks ||= [] end |
.parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) ⇒ void
This method returns an undefined value.
Parses a path or set of paths
75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 |
# File 'lib/yard/parser/source_parser.rb', line 75 def parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) log.debug("Parsing #{paths.inspect} with `#{parser_type}` parser") excluded = excluded.map do |path| case path when Regexp; path else Regexp.new(path.to_s, Regexp::IGNORECASE) end end files = [paths].flatten. map {|p| File.directory?(p) ? "#{p}/**/*.{rb,c}" : p }. map {|p| p.include?("*") ? Dir[p] : p }.flatten. reject {|p| !File.file?(p) || excluded.any? {|re| p =~ re } } log.enter_level(level) do parse_in_order(*files.uniq) end end |
.parse_string(content, ptype = parser_type) ⇒ Object
Parses a string content
98 99 100 |
# File 'lib/yard/parser/source_parser.rb', line 98 def parse_string(content, ptype = parser_type) new(ptype).parse(StringIO.new(content)) end |
.parser_type_for_extension(extension) ⇒ Symbol
Finds a parser type that is registered for the extension. If no type is found, the default Ruby type is returned.
150 151 152 153 154 155 |
# File 'lib/yard/parser/source_parser.rb', line 150 def parser_type_for_extension(extension) type = parser_type_extensions.find do |t, exts| [exts].flatten.any? {|ext| ext === extension } end validated_parser_type(type ? type.first : :ruby) end |
.register_parser_type(type, parser_klass, extensions = nil) ⇒ void
This method returns an undefined value.
Registers a new parser type.
121 122 123 124 125 126 127 |
# File 'lib/yard/parser/source_parser.rb', line 121 def register_parser_type(type, parser_klass, extensions = nil) unless Base > parser_klass raise ArgumentError, "expecting parser_klass to be a subclass of YARD::Parser::Base" end parser_type_extensions[type.to_sym] = extensions if extensions parser_types[type.to_sym] = parser_klass end |
.tokenize(content, ptype = parser_type) ⇒ Array
Tokenizes but does not parse the block of code
107 108 109 |
# File 'lib/yard/parser/source_parser.rb', line 107 def tokenize(content, ptype = parser_type) new(ptype).tokenize(content) end |
Instance Method Details
#parse(content = __FILE__) ⇒ Object?
The main parser method. This should not be called directly. Instead, use the class methods parse and parse_string.
414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 |
# File 'lib/yard/parser/source_parser.rb', line 414 def parse(content = __FILE__) case content when String @file = File.cleanpath(content) content = convert_encoding(File.read_binary(file)) checksum = Registry.checksum_for(content) return if Registry.checksums[file] == checksum if Registry.checksums.has_key?(file) log.info "File '#{file}' was modified, re-processing..." end Registry.checksums[@file] = checksum self.parser_type = parser_type_for_filename(file) else content = content.read if content.respond_to? :read end @contents = content @parser = parser_class.new(content, file) self.class.before_parse_file_callbacks.each do |cb| return @parser if cb.call(self) == false end @parser.parse post_process self.class.after_parse_file_callbacks.each do |cb| cb.call(self) end @parser rescue ArgumentError, NotImplementedError => e log.warn("Cannot parse `#{file}': #{e.}") log.backtrace(e) if log.level >= Logger::DEBUG rescue ParserSyntaxError => e log.warn(e..capitalize) log.backtrace(e) if log.level >= Logger::DEBUG end |
#tokenize(content) ⇒ Array
Tokenizes but does not parse the block of code using the current #parser_type
458 459 460 461 |
# File 'lib/yard/parser/source_parser.rb', line 458 def tokenize(content) @parser = parser_class.new(content, file) @parser.tokenize end |