Class: RDoc::Parser

Inherits:
Object
  • Object
show all
Defined in:
lib/rdoc/parser.rb,
lib/rdoc/parser/c.rb,
lib/rdoc/parser/rd.rb,
lib/rdoc/parser/rbs.rb,
lib/rdoc/parser/ruby.rb,
lib/rdoc/parser/text.rb,
lib/rdoc/parser/simple.rb,
lib/rdoc/parser/markdown.rb,
lib/rdoc/parser/changelog.rb,
lib/rdoc/parser/ruby_colorizer.rb

Overview

A parser is simple a class that subclasses RDoc::Parser and implements #scan to fill in an RDoc::TopLevel with parsed data.

The initialize method takes an RDoc::TopLevel to fill with parsed content, the name of the file to be parsed, the content of the file, an RDoc::Options object and an RDoc::Stats object to inform the user of parsed items. The scan method is then called to parse the file and must return the RDoc::TopLevel object. By calling super these items will be set for you.

In order to be used by RDoc the parser needs to register the file extensions it can parse. Use ::parse_files_matching to register extensions.

require 'rdoc'

class RDoc::Parser::Xyz < RDoc::Parser
parse_files_matching /\.xyz$/

def initialize top_level, file_name, content, options, stats
  super

  # extra initialization if needed
end

def scan
  # parse file and fill in @top_level
end
end

Defined Under Namespace

Modules: RubyColorizer, Text Classes: C, ChangeLog, Markdown, RBS, RD, Ruby, Simple

Class Attribute Summary collapse

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(top_level, content, options, stats) ⇒ Parser

Creates a new Parser storing top_level, file_name, content, options and stats in instance variables. In @preprocess an RDoc::Markup::PreProcess object is created which allows processing of directives.



262
263
264
265
266
267
268
269
270
271
272
273
274
 
# File 'lib/rdoc/parser.rb', line 262

def initialize(top_level, content, options, stats)
  @top_level = top_level
  @top_level.parser = self.class
  @store = @top_level.store

  @file_name = top_level.absolute_name
  @content = content
  @options = options
  @stats = stats

  @preprocess = Markup::PreProcess.new @file_name, @options.rdoc_include
  @preprocess.options = @options
end

Class Attribute Details

.parsers ⇒ Object (readonly)

An Array of arrays that maps file extension (or name) regular expressions to parser classes that will parse matching filenames.

Use parse_files_matching to register a parser's file extensions.



46
47
48
 
# File 'lib/rdoc/parser.rb', line 46

def parsers
  @parsers
end

Instance Attribute Details

#file_name ⇒ Object (readonly)

The name of the file being parsed



53
54
55
 
# File 'lib/rdoc/parser.rb', line 53

def file_name
  @file_name
end

Class Method Details

.alias_extension(old_ext, new_ext) ⇒ Object

Alias an extension to another extension. After this call, files ending "new_ext" will be parsed using the same parser as "old_ext"



59
60
61
62
63
64
65
66
67
68
69
 
# File 'lib/rdoc/parser.rb', line 59

def self.alias_extension(old_ext, new_ext)
  old_ext = old_ext.sub(/^\.(.*)/, '\1')
  new_ext = new_ext.sub(/^\.(.*)/, '\1')

  parser = can_parse_by_name "xxx.#{old_ext}"
  return false unless parser

  Parser.parsers.unshift [/\.#{new_ext}$/, parser]

  true
end

.binary?(file) ⇒ Boolean

Determines if the file is a "binary" file which basically means it has content that an RDoc parser shouldn't try to consume.

Returns:

  • (Boolean) 


75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'lib/rdoc/parser.rb', line 75

def self.binary?(file)
  return false if file =~ /\.(rdoc|txt)$/

  s = File.read(file, 1024) or return false

  return true if s[0, 2] == Marshal.dump('')[0, 2] or s.index("\x00")

  mode = 'r:utf-8' # default source encoding has been changed to utf-8
  s.sub!(/\A#!.*\n/, '')     # assume shebang line isn't longer than 1024.
  encoding = s[/^\s*\#\s*(?:-\*-\s*)?(?:en)?coding:\s*([^\s;]+?)(?:-\*-|[\s;])/, 1]
  mode = "rb:#{encoding}" if encoding
  s = File.open(file, mode) {|f| f.gets(nil, 1024)}

  not s.valid_encoding?
end

.can_parse(file_name) ⇒ Object

Return a parser that can handle a particular extension



108
109
110
111
112
113
114
115
 
# File 'lib/rdoc/parser.rb', line 108

def self.can_parse(file_name)
  parser = can_parse_by_name file_name

  # HACK Selenium hides a jar file using a .txt extension
  return if parser == Parser::Simple and zip? file_name

  parser
end

.can_parse_by_name(file_name) ⇒ Object

Returns a parser that can handle the extension for file_name. This does not depend upon the file being readable.



121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
 
# File 'lib/rdoc/parser.rb', line 121

def self.can_parse_by_name(file_name)
  _, parser = Parser.parsers.find { |regexp,| regexp =~ file_name }

  # The default parser must not parse binary files
  ext_name = File.extname file_name
  return parser if ext_name.empty?

  if parser == Parser::Simple and ext_name !~ /txt|rdoc/
    case mode = check_modeline(file_name)
    when nil, 'rdoc' # continue
    else
      Parser.parsers.find { |_, p| return p if mode.casecmp?(p.name[/\w+\z/]) }
      return nil
    end
  end

  parser
rescue Errno::EACCES
end

.check_modeline(file_name) ⇒ Object

Returns the file type from the modeline in file_name



144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
 
# File 'lib/rdoc/parser.rb', line 144

def self.check_modeline(file_name)
  line = File.open file_name do |io|
    io.gets
  end

  /-\*-\s*(.*?\S)\s*-\*-/ =~ line

  return nil unless type = $1

  if /;/ =~ type
    return nil unless /(?:\s|\A)mode:\s*([^\s;]+)/i =~ type
    type = $1
  end

  return nil if /coding:/i =~ type

  type.downcase
rescue ArgumentError
rescue ::Encoding::InvalidByteSequenceError # invalid byte sequence

end

.for(top_level, content, options, stats) ⇒ Object

Finds and instantiates the correct parser for the given file_name and content.



170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
 
# File 'lib/rdoc/parser.rb', line 170

def self.for(top_level, content, options, stats)
  file_name = top_level.absolute_name
  return if binary? file_name

  parser = use_markup content

  unless parser
    parse_name = file_name

    # If no extension, look for shebang
    if file_name !~ /\.\w+$/ && content =~ %r{\A#!(.+)}
      shebang = $1
      case shebang
      when %r{env\s+ruby}, %r{/ruby}
        parse_name = 'dummy.rb'
      end
    end

    parser = can_parse parse_name
  end

  return unless parser

  content = remove_modeline content

  parser.new top_level, content, options, stats
rescue SystemCallError
  nil
end

.parse_files_matching(regexp) ⇒ Object

Record which file types this parser can understand.

It is ok to call this multiple times.



205
206
207
 
# File 'lib/rdoc/parser.rb', line 205

def self.parse_files_matching(regexp)
  Parser.parsers.unshift [regexp, self]
end

.remove_modeline(content) ⇒ Object

Removes an emacs-style modeline from the first line of the document



212
213
214
 
# File 'lib/rdoc/parser.rb', line 212

def self.remove_modeline(content)
  content.sub(/\A.*-\*-\s*(.*?\S)\s*-\*-.*\r?\n/, '')
end

.use_markup(content) ⇒ Object

If there is a markup: parser_name comment at the front of the file, use it to determine the parser. For example:

# markup: rdoc
# Class comment can go here

class C
end

The comment should appear as the first line of the content.

If the content contains a shebang or editor modeline the comment may appear on the second or third line.

Any comment style may be used to hide the markup comment.

The tomdoc and markdown markups name comment formats rather than parsers, so no parser is selected for them and RDoc::Parser.for picks one from the file name instead.



237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
 
# File 'lib/rdoc/parser.rb', line 237

def self.use_markup(content)
  markup = content.lines.first(3).grep(/markup:\s+(\w+)/) { $1 }.first

  return unless markup

  # tomdoc and markdown name a comment format, not a parser.  Skipping them
  # keeps the search below from matching RDoc::Parser::Markdown for a file
  # that is not markdown.
  return if %w[tomdoc markdown].include? markup

  markup = Regexp.escape markup

  _, selected = Parser.parsers.find do |_, parser|
    /^#{markup}$/i =~ parser.name.sub(/.*:/, '')
  end

  selected
end

.zip?(file) ⇒ Boolean

Checks if file is a zip file in disguise. Signatures from http://www.garykessler.net/library/file_sigs.html

Returns:

  • (Boolean) 


95
96
97
98
99
100
101
102
103
 
# File 'lib/rdoc/parser.rb', line 95

def self.zip?(file)
  zip_signature = File.read file, 4

  zip_signature == "PK\x03\x04" or
    zip_signature == "PK\x05\x06" or
    zip_signature == "PK\x07\x08"
rescue
  false
end

Instance Method Details

#handle_tab_width(body) ⇒ Object

Normalizes tabs in body



281
282
283
284
285
286
287
288
289
290
291
292
293
294
 
# File 'lib/rdoc/parser.rb', line 281

def handle_tab_width(body)
  if /\t/ =~ body
    tab_width = @options.tab_width
    body.split(/\n/).map do |line|
      1 while line.gsub!(/\t+/) do
        b, e = $~.offset(0)
        ' ' * (tab_width * (e-b) - b % tab_width)
      end
      line
    end.join "\n"
  else
    body
  end
end