Class: YARD::Parser::SourceParser

Inherits:
Object
  • Object
show all
Defined in:
lib/yard/parser/source_parser.rb

Overview

Responsible for parsing a source file into the namespace. Parsing also invokes handlers to process the parsed statements and generate any code objects that may be recognized.

Custom Parsers

SourceParser allows custom parsers to be registered and called when a certain filetype is recognized. To register a parser and hook it up to a set of file extensions, call SourceParser.register_parser_type

See Also:

Constant Summary collapse

SHEBANG_LINE = 
/\A\s*#!\S+/
ENCODING_LINE = 
/\A(?:\s*#*!.*\r?\n)?\s*#+.*coding\s*[:=]{1,2}\s*(\S+)/i

Class Attribute Summary collapse

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(parser_type = SourceParser.parser_type, load_order_errors = false) ⇒ SourceParser

Creates a new parser object for code parsing with a specific parser type.

Parameters:

  • parser_type (Symbol) (defaults to: SourceParser.parser_type) —

    the parser type to use

  • load_order_errors (Boolean) (defaults to: false) —

    whether or not to raise the LoadOrderError



190
191
192
193
194
 
# File 'lib/yard/parser/source_parser.rb', line 190

def initialize(parser_type = SourceParser.parser_type, load_order_errors = false)
  @load_order_errors = load_order_errors
  @file = '(stdin)'
  self.parser_type = parser_type
end

Class Attribute Details

.parser_type ⇒ Symbol

Returns the default parser type (defaults to :ruby).

Returns:

  • (Symbol) —

    the default parser type (defaults to :ruby)



40
41
42
 
# File 'lib/yard/parser/source_parser.rb', line 40

def parser_type
  @parser_type
end

.parser_type_extensions=(value) ⇒ Object 



122
 
# File 'lib/yard/parser/source_parser.rb', line 122

def parser_type_extensions=(value) @@parser_type_extensions = value end

.parser_types=(value) ⇒ Object 



114
 
# File 'lib/yard/parser/source_parser.rb', line 114

def parser_types=(value) @@parser_types = value end

Instance Attribute Details

#file ⇒ Object (readonly)

The filename being parsed by the parser.



180
181
182
 
# File 'lib/yard/parser/source_parser.rb', line 180

def file
  @file
end

#parser_type ⇒ Object

The parser type associated with the parser instance. This should be set by the constructor.



184
185
186
 
# File 'lib/yard/parser/source_parser.rb', line 184

def parser_type
  @parser_type
end

Class Method Details

.parse(paths = ["lib/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) ⇒ Object

Parses a path or set of paths

Parameters:

  • paths (String, Array<String>) (defaults to: ["lib/**/*.rb", "ext/**/*.c"]) —

    a path, glob, or list of paths to parse

  • excluded (Array<String, Regexp>) (defaults to: []) —

    a list of excluded path matchers

  • level (Fixnum) (defaults to: log.level) —

    the logger level to use during parsing. See Logger

Returns:

  • the parser object that was used to parse the source.



54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
 
# File 'lib/yard/parser/source_parser.rb', line 54

def parse(paths = ["lib/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level)
  log.debug("Parsing #{paths.inspect} with `#{parser_type}` parser")
  excluded = excluded.map do |path|
    case path
    when Regexp; path
    else Regexp.new(path.to_s, Regexp::IGNORECASE)
    end
  end
  files = [paths].flatten.
    map {|p| File.directory?(p) ? "#{p}/**/*.{rb,c}" : p }.
    map {|p| p.include?("*") ? Dir[p] : p }.flatten.
    reject {|p| !File.file?(p) || excluded.any? {|re| p =~ re } }

  log.enter_level(level) do
    parse_in_order(*files.uniq)
  end
end

.parse_string(content, ptype = parser_type) ⇒ Object

Parses a string content

Parameters:

  • content (String) —

    the block of code to parse

  • ptype (Symbol) (defaults to: parser_type) —

    the parser type to use. See parser_type.

Returns:

  • the parser object that was used to parse content



77
78
79
 
# File 'lib/yard/parser/source_parser.rb', line 77

def parse_string(content, ptype = parser_type)
  new(ptype).parse(StringIO.new(content))
end

.parser_type_for_extension(extension) ⇒ Symbol

Finds a parser type that is registered for the extension. If no type is found, the default Ruby type is returned.

Returns:

  • (Symbol) —

    the parser type to be used for the extension

Since:

  • 0.5.6



129
130
131
132
133
134
 
# File 'lib/yard/parser/source_parser.rb', line 129

def parser_type_for_extension(extension)
  type = parser_type_extensions.find do |t, exts|
    [exts].flatten.any? {|ext| ext === extension }
  end
  validated_parser_type(type ? type.first : :ruby)
end

.register_parser_type(type, parser_klass, extensions = nil) ⇒ void

This method returns an undefined value.

Registers a new parser type.

Examples:

Registering a parser for “java” files

SourceParser.register_parser_type :java, JavaParser, 'java'

Parameters:

  • type (Symbol) —

    a symbolic name for the parser type

  • parser_klass (Base) —

    a class that implements parsing and tokenization

  • extensions (Array<String>, String, Regexp) (defaults to: nil) —

    a list of extensions or a regex to match against the file extension

See Also:



100
101
102
103
104
105
106
 
# File 'lib/yard/parser/source_parser.rb', line 100

def register_parser_type(type, parser_klass, extensions = nil)
  unless Base > parser_klass
    raise ArgumentError, "expecting parser_klass to be a subclass of YARD::Parser::Base"
  end
  parser_type_extensions[type.to_sym] = extensions if extensions
  parser_types[type.to_sym] = parser_klass
end

.tokenize(content, ptype = parser_type) ⇒ Array

Tokenizes but does not parse the block of code

Parameters:

  • content (String) —

    the block of code to tokenize

  • ptype (Symbol) (defaults to: parser_type) —

    the parser type to use. See parser_type.

Returns:

  • (Array) —

    a list of tokens



86
87
88
 
# File 'lib/yard/parser/source_parser.rb', line 86

def tokenize(content, ptype = parser_type)
  new(ptype).tokenize(content)
end

Instance Method Details

#parse(content = __FILE__) ⇒ Object?

The main parser method. This should not be called directly. Instead, use the class methods parse and parse_string.

Parameters:

  • content (String, #read, Object) (defaults to: __FILE__) —

    the source file to parse

Returns:

  • (Object, nil) —

    the parser object used to parse the source



201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
 
# File 'lib/yard/parser/source_parser.rb', line 201

def parse(content = __FILE__)
  case content
  when String
    @file = File.cleanpath(content)
    content = convert_encoding(File.read_binary(file))
    checksum = Registry.checksum_for(content)
    return if Registry.checksums[file] == checksum

    if Registry.checksums.has_key?(file)
      log.info "File '#{file}' was modified, re-processing..."
    end
    Registry.checksums[@file] = checksum
    self.parser_type = parser_type_for_filename(file)
  else
    content = content.read if content.respond_to? :read
  end
  
  @parser = parser_class.new(content, file)
  @parser.parse
  post_process
  @parser
rescue ArgumentError, NotImplementedError => e
  log.warn("Cannot parse `#{file}': #{e.message}")
rescue ParserSyntaxError => e
  log.warn(e.message.capitalize)
end

#tokenize(content) ⇒ Array

Tokenizes but does not parse the block of code using the current #parser_type

Parameters:

  • content (String) —

    the block of code to tokenize

Returns:

  • (Array) —

    a list of tokens



232
233
234
235
 
# File 'lib/yard/parser/source_parser.rb', line 232

def tokenize(content)
  @parser = parser_class.new(content, file)
  @parser.tokenize
end