Class: YARD::Parser::SourceParser

Inherits:
Object
  • Object
show all
Defined in:
lib/yard/parser/source_parser.rb

Overview

Responsible for parsing a source file into the namespace. Parsing also invokes handlers to process the parsed statements and generate any code objects that may be recognized.

Custom Parsers

SourceParser allows custom parsers to be registered and called when a certain filetype is recognized. To register a parser and hook it up to a set of file extensions, call SourceParser.register_parser_type

See Also:

Class Attribute Summary collapse

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(parser_type = SourceParser.parser_type, load_order_errors = false) ⇒ SourceParser

Creates a new parser object for code parsing with a specific parser type.

Parameters:

  • parser_type (Symbol) (defaults to: SourceParser.parser_type) —

    the parser type to use

  • load_order_errors (Boolean) (defaults to: false) —

    whether or not to raise the LoadOrderError



189
190
191
192
193
 
# File 'lib/yard/parser/source_parser.rb', line 189

def initialize(parser_type = SourceParser.parser_type, load_order_errors = false)
  @load_order_errors = load_order_errors
  @file = '(stdin)'
  self.parser_type = parser_type
end

Class Attribute Details

.parser_type ⇒ Symbol

Returns the default parser type (defaults to :ruby).

Returns:

  • (Symbol) —

    the default parser type (defaults to :ruby)



39
40
41
 
# File 'lib/yard/parser/source_parser.rb', line 39

def parser_type
  @parser_type
end

.parser_type_extensions=(value) ⇒ Object 



121
 
# File 'lib/yard/parser/source_parser.rb', line 121

def parser_type_extensions=(value) @@parser_type_extensions = value end

.parser_types=(value) ⇒ Object 



113
 
# File 'lib/yard/parser/source_parser.rb', line 113

def parser_types=(value) @@parser_types = value end

Instance Attribute Details

#file ⇒ Object (readonly)

The filename being parsed by the parser.



179
180
181
 
# File 'lib/yard/parser/source_parser.rb', line 179

def file
  @file
end

#parser_type ⇒ Object

The parser type associated with the parser instance. This should be set by the constructor.



183
184
185
 
# File 'lib/yard/parser/source_parser.rb', line 183

def parser_type
  @parser_type
end

Class Method Details

.parse(paths = ["lib/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level) ⇒ Object

Parses a path or set of paths

Parameters:

  • paths (String, Array<String>) (defaults to: ["lib/**/*.rb", "ext/**/*.c"]) —

    a path, glob, or list of paths to parse

  • excluded (Array<String, Regexp>) (defaults to: []) —

    a list of excluded path matchers

  • level (Fixnum) (defaults to: log.level) —

    the logger level to use during parsing. See Logger

Returns:

  • the parser object that was used to parse the source.



53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
 
# File 'lib/yard/parser/source_parser.rb', line 53

def parse(paths = ["lib/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level)
  log.debug("Parsing #{paths.inspect} with `#{parser_type}` parser")
  excluded = excluded.map do |path|
    case path
    when Regexp; path
    else Regexp.new(path.to_s, Regexp::IGNORECASE)
    end
  end
  files = [paths].flatten.
    map {|p| File.directory?(p) ? "#{p}/**/*.{rb,c}" : p }.
    map {|p| p.include?("*") ? Dir[p] : p }.flatten.
    reject {|p| !File.file?(p) || excluded.any? {|re| p =~ re } }

  log.enter_level(level) do
    parse_in_order(*files.uniq)
  end
end

.parse_string(content, ptype = parser_type) ⇒ Object

Parses a string content

Parameters:

  • content (String) —

    the block of code to parse

  • ptype (Symbol) (defaults to: parser_type) —

    the parser type to use. See parser_type.

Returns:

  • the parser object that was used to parse content



76
77
78
 
# File 'lib/yard/parser/source_parser.rb', line 76

def parse_string(content, ptype = parser_type)
  new(ptype).parse(StringIO.new(content))
end

.parser_type_for_extension(extension) ⇒ Symbol

Finds a parser type that is registered for the extension. If no type is found, the default Ruby type is returned.

Returns:

  • (Symbol) —

    the parser type to be used for the extension

Since:

  • 0.5.6



128
129
130
131
132
133
 
# File 'lib/yard/parser/source_parser.rb', line 128

def parser_type_for_extension(extension)
  type = parser_type_extensions.find do |t, exts|
    [exts].flatten.any? {|ext| ext === extension }
  end
  validated_parser_type(type ? type.first : :ruby)
end

.register_parser_type(type, parser_klass, extensions = nil) ⇒ void

This method returns an undefined value.

Registers a new parser type.

Examples:

Registering a parser for “java” files

SourceParser.register_parser_type :java, JavaParser, 'java'

Parameters:

  • type (Symbol) —

    a symbolic name for the parser type

  • parser_klass (Base) —

    a class that implements parsing and tokenization

  • extensions (Array<String>, String, Regexp) (defaults to: nil) —

    a list of extensions or a regex to match against the file extension

See Also:



99
100
101
102
103
104
105
 
# File 'lib/yard/parser/source_parser.rb', line 99

def register_parser_type(type, parser_klass, extensions = nil)
  unless Base > parser_klass
    raise ArgumentError, "expecting parser_klass to be a subclass of YARD::Parser::Base"
  end
  parser_type_extensions[type.to_sym] = extensions if extensions
  parser_types[type.to_sym] = parser_klass
end

.tokenize(content, ptype = parser_type) ⇒ Array

Tokenizes but does not parse the block of code

Parameters:

  • content (String) —

    the block of code to tokenize

  • ptype (Symbol) (defaults to: parser_type) —

    the parser type to use. See parser_type.

Returns:

  • (Array) —

    a list of tokens



85
86
87
 
# File 'lib/yard/parser/source_parser.rb', line 85

def tokenize(content, ptype = parser_type)
  new(ptype).tokenize(content)
end

Instance Method Details

#parse(content = __FILE__) ⇒ Object?

The main parser method. This should not be called directly. Instead, use the class methods parse and parse_string.

Parameters:

  • content (String, #read, Object) (defaults to: __FILE__) —

    the source file to parse

Returns:

  • (Object, nil) —

    the parser object used to parse the source



200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
 
# File 'lib/yard/parser/source_parser.rb', line 200

def parse(content = __FILE__)
  case content
  when String
    @file = File.cleanpath(content)
    content = convert_encoding(File.read_binary(file))
    checksum = Registry.checksum_for(content)
    return if Registry.checksums[file] == checksum

    if Registry.checksums.has_key?(file)
      log.info "File '#{file}' was modified, re-processing..."
    end
    Registry.checksums[@file] = checksum
    self.parser_type = parser_type_for_filename(file)
  else
    content = content.read if content.respond_to? :read
  end
  
  @parser = parser_class.new(content, file)
  @parser.parse
  post_process
  @parser
rescue ArgumentError, NotImplementedError => e
  log.warn("Cannot parse `#{file}': #{e.message}")
rescue ParserSyntaxError => e
  log.warn(e.message.capitalize)
end

#tokenize(content) ⇒ Array

Tokenizes but does not parse the block of code using the current #parser_type

Parameters:

  • content (String) —

    the block of code to tokenize

Returns:

  • (Array) —

    a list of tokens



231
232
233
234
 
# File 'lib/yard/parser/source_parser.rb', line 231

def tokenize(content)
  @parser = parser_class.new(content, file)
  @parser.tokenize
end