Class: YARD::Parser::SourceParser
- Inherits:
-
Object
- Object
- YARD::Parser::SourceParser
- Defined in:
- lib/yard/parser/source_parser.rb
Overview
Responsible for parsing a source file into the namespace. Parsing also invokes handlers to process the parsed statements and generate any code objects that may be recognized.
Custom Parsers
SourceParser allows custom parsers to be registered and called when a certain filetype is recognized. To register a parser and hook it up to a set of file extensions, call SourceParser.register_parser_type
Constant Summary collapse
- SHEBANG_LINE =
/\A\s*#!\S+/
- ENCODING_LINE =
/\A(?:\s*#*!.*\r?\n)?\s*#+.*coding\s*[:=]{1,2}\s*(\S+)/i
- ENCODING_BYTE_ORDER_MARKS =
Byte order marks for various encodings
{ 'utf-8' => "\xEF\xBB\xBF", # Not yet supported #'utf-16be' => "\xFE\xFF", #'utf-16le' => "\xFF\xFE", #'utf-32be' => "\x00\x00\xFF\xFE", #'utf-32le' => "\xFF\xFE", }
Parser Callbacks collapse
-
#contents ⇒ String
readonly
The contents of the file to be parsed.
-
#file ⇒ String
readonly
The filename being parsed by the parser.
-
#globals ⇒ OpenStruct
readonly
An open struct containing arbitrary global state shared between files and handlers.
-
#parser_type ⇒ Symbol
readonly
The parser type associated with the parser instance.
Class Attribute Summary collapse
-
.parser_type ⇒ Symbol
The default parser type (defaults to :ruby).
- .parser_type_extensions ⇒ Object writeonly
- .parser_types ⇒ Object writeonly
Parser Callbacks collapse
-
.after_parse_file {|parser| ... } ⇒ Proc
Registers a callback to be called after an individual file is parsed.
-
.after_parse_file_callbacks ⇒ Array<Proc>
The list of callbacks to be called after parsing a file.
-
.after_parse_list {|files, globals| ... } ⇒ Proc
Registers a callback to be called after a list of files is parsed via SourceParser.parse.
-
.after_parse_list_callbacks ⇒ Array<Proc>
The list of callbacks to be called after parsing a list of files.
-
.before_parse_file {|parser| ... } ⇒ Proc
Registers a callback to be called before an individual file is parsed.
-
.before_parse_file_callbacks ⇒ Array<Proc>
The list of callbacks to be called before parsing a file.
-
.before_parse_list {|files, globals| ... } ⇒ Proc
Registers a callback to be called before a list of files is parsed via SourceParser.parse.
-
.before_parse_list_callbacks ⇒ Array<Proc>
The list of callbacks to be called before parsing a list of files.
-
#initialize(parser_type = SourceParser.parser_type, load_order_errors = false, globals = nil) ⇒ SourceParser
constructor
Creates a new parser object for code parsing with a specific parser type.
-
#parse(content = __FILE__) ⇒ Object?
The main parser method.
-
#tokenize(content) ⇒ Array
Tokenizes but does not parse the block of code using the current #parser_type.
Class Method Summary collapse
-
.parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) ⇒ Object
Parses a path or set of paths.
-
.parse_string(content, ptype = parser_type) ⇒ Object
Parses a string
content
. -
.parser_type_for_extension(extension) ⇒ Symbol
Finds a parser type that is registered for the extension.
-
.register_parser_type(type, parser_klass, extensions = nil) ⇒ void
Registers a new parser type.
-
.tokenize(content, ptype = parser_type) ⇒ Array
Tokenizes but does not parse the block of code.
Constructor Details
#initialize(parser_type = SourceParser.parser_type, load_order_errors = false, globals = nil) ⇒ SourceParser
Creates a new parser object for code parsing with a specific parser type.
393 394 395 396 397 398 |
# File 'lib/yard/parser/source_parser.rb', line 393 def initialize(parser_type = SourceParser.parser_type, load_order_errors = false, globals = nil) @load_order_errors = load_order_errors @file = '(stdin)' @globals = globals || OpenStruct.new self.parser_type = parser_type end |
Class Attribute Details
.parser_type ⇒ Symbol
Returns the default parser type (defaults to :ruby).
52 53 54 |
# File 'lib/yard/parser/source_parser.rb', line 52 def parser_type @parser_type end |
.parser_type_extensions=(value) ⇒ Object
134 |
# File 'lib/yard/parser/source_parser.rb', line 134 def parser_type_extensions=(value) @@parser_type_extensions = value end |
.parser_types=(value) ⇒ Object
126 |
# File 'lib/yard/parser/source_parser.rb', line 126 def parser_types=(value) @@parser_types = value end |
Instance Attribute Details
#contents ⇒ String (readonly)
Returns the contents of the file to be parsed.
386 387 388 |
# File 'lib/yard/parser/source_parser.rb', line 386 def contents @contents end |
#file ⇒ String (readonly)
Returns the filename being parsed by the parser.
373 374 375 |
# File 'lib/yard/parser/source_parser.rb', line 373 def file @file end |
#globals ⇒ OpenStruct (readonly)
Returns an open struct containing arbitrary global state shared between files and handlers.
382 383 384 |
# File 'lib/yard/parser/source_parser.rb', line 382 def globals @globals end |
#parser_type ⇒ Symbol
Returns the parser type associated with the parser instance. This should be set by the constructor.
377 378 379 |
# File 'lib/yard/parser/source_parser.rb', line 377 def parser_type @parser_type end |
Class Method Details
.after_parse_file {|parser| ... } ⇒ Proc
Registers a callback to be called after an individual file is parsed. The block passed to this method will be called on subsequent parse calls.
To register a callback that is called after the entire list of files is processed, see after_parse_list.
294 295 296 |
# File 'lib/yard/parser/source_parser.rb', line 294 def after_parse_file(&block) after_parse_file_callbacks << block end |
.after_parse_file_callbacks ⇒ Array<Proc>
Returns the list of callbacks to be called after parsing a file. Should only be used for testing.
322 323 324 |
# File 'lib/yard/parser/source_parser.rb', line 322 def after_parse_file_callbacks @after_parse_file_callbacks ||= [] end |
.after_parse_list {|files, globals| ... } ⇒ Proc
Registers a callback to be called after a list of files is parsed via parse. The block passed to this method will be called on subsequent parse calls.
228 229 230 |
# File 'lib/yard/parser/source_parser.rb', line 228 def after_parse_list(&block) after_parse_list_callbacks << block end |
.after_parse_list_callbacks ⇒ Array<Proc>
Returns the list of callbacks to be called after parsing a list of files. Should only be used for testing.
308 309 310 |
# File 'lib/yard/parser/source_parser.rb', line 308 def after_parse_list_callbacks @after_parse_list_callbacks ||= [] end |
.before_parse_file {|parser| ... } ⇒ Proc
Registers a callback to be called before an individual file is parsed. The block passed to this method will be called on subsequent parse calls.
To register a callback that is called before the entire list of files is processed, see before_parse_list.
265 266 267 |
# File 'lib/yard/parser/source_parser.rb', line 265 def before_parse_file(&block) before_parse_file_callbacks << block end |
.before_parse_file_callbacks ⇒ Array<Proc>
Returns the list of callbacks to be called before parsing a file. Should only be used for testing.
315 316 317 |
# File 'lib/yard/parser/source_parser.rb', line 315 def before_parse_file_callbacks @before_parse_file_callbacks ||= [] end |
.before_parse_list {|files, globals| ... } ⇒ Proc
Registers a callback to be called before a list of files is parsed via parse. The block passed to this method will be called on subsequent parse calls.
204 205 206 |
# File 'lib/yard/parser/source_parser.rb', line 204 def before_parse_list(&block) before_parse_list_callbacks << block end |
.before_parse_list_callbacks ⇒ Array<Proc>
Returns the list of callbacks to be called before parsing a list of files. Should only be used for testing.
301 302 303 |
# File 'lib/yard/parser/source_parser.rb', line 301 def before_parse_list_callbacks @before_parse_list_callbacks ||= [] end |
.parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) ⇒ Object
Parses a path or set of paths
66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 |
# File 'lib/yard/parser/source_parser.rb', line 66 def parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) log.debug("Parsing #{paths.inspect} with `#{parser_type}` parser") excluded = excluded.map do |path| case path when Regexp; path else Regexp.new(path.to_s, Regexp::IGNORECASE) end end files = [paths].flatten. map {|p| File.directory?(p) ? "#{p}/**/*.{rb,c}" : p }. map {|p| p.include?("*") ? Dir[p] : p }.flatten. reject {|p| !File.file?(p) || excluded.any? {|re| p =~ re } } log.enter_level(level) do parse_in_order(*files.uniq) end end |
.parse_string(content, ptype = parser_type) ⇒ Object
Parses a string content
89 90 91 |
# File 'lib/yard/parser/source_parser.rb', line 89 def parse_string(content, ptype = parser_type) new(ptype).parse(StringIO.new(content)) end |
.parser_type_for_extension(extension) ⇒ Symbol
Finds a parser type that is registered for the extension. If no type is found, the default Ruby type is returned.
141 142 143 144 145 146 |
# File 'lib/yard/parser/source_parser.rb', line 141 def parser_type_for_extension(extension) type = parser_type_extensions.find do |t, exts| [exts].flatten.any? {|ext| ext === extension } end validated_parser_type(type ? type.first : :ruby) end |
.register_parser_type(type, parser_klass, extensions = nil) ⇒ void
This method returns an undefined value.
Registers a new parser type.
112 113 114 115 116 117 118 |
# File 'lib/yard/parser/source_parser.rb', line 112 def register_parser_type(type, parser_klass, extensions = nil) unless Base > parser_klass raise ArgumentError, "expecting parser_klass to be a subclass of YARD::Parser::Base" end parser_type_extensions[type.to_sym] = extensions if extensions parser_types[type.to_sym] = parser_klass end |
.tokenize(content, ptype = parser_type) ⇒ Array
Tokenizes but does not parse the block of code
98 99 100 |
# File 'lib/yard/parser/source_parser.rb', line 98 def tokenize(content, ptype = parser_type) new(ptype).tokenize(content) end |
Instance Method Details
#parse(content = __FILE__) ⇒ Object?
The main parser method. This should not be called directly. Instead, use the class methods parse and parse_string.
405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 |
# File 'lib/yard/parser/source_parser.rb', line 405 def parse(content = __FILE__) case content when String @file = File.cleanpath(content) content = convert_encoding(File.read_binary(file)) checksum = Registry.checksum_for(content) return if Registry.checksums[file] == checksum if Registry.checksums.has_key?(file) log.info "File '#{file}' was modified, re-processing..." end Registry.checksums[@file] = checksum self.parser_type = parser_type_for_filename(file) else content = content.read if content.respond_to? :read end @contents = content @parser = parser_class.new(content, file) self.class.before_parse_file_callbacks.each do |cb| return @parser if cb.call(self) == false end @parser.parse post_process self.class.after_parse_file_callbacks.each do |cb| cb.call(self) end @parser rescue ArgumentError, NotImplementedError => e log.warn("Cannot parse `#{file}': #{e.}") rescue ParserSyntaxError => e log.warn(e..capitalize) end |
#tokenize(content) ⇒ Array
Tokenizes but does not parse the block of code using the current #parser_type
447 448 449 450 |
# File 'lib/yard/parser/source_parser.rb', line 447 def tokenize(content) @parser = parser_class.new(content, file) @parser.tokenize end |