Class: RDoc::Parser
- Inherits:
-
Object
- Object
- RDoc::Parser
- Defined in:
- lib/rdoc/parser.rb
Overview
A parser is simple a class that subclasses RDoc::Parser and implements #scan to fill in an RDoc::TopLevel with parsed data.
The initialize method takes an RDoc::TopLevel to fill with parsed content, the name of the file to be parsed, the content of the file, an RDoc::Options object and an RDoc::Stats object to inform the user of parsed items. The scan method is then called to parse the file and must return the RDoc::TopLevel object. By calling super these items will be set for you.
In order to be used by RDoc the parser needs to register the file extensions it can parse. Use ::parse_files_matching to register extensions.
require 'rdoc'
class RDoc::Parser::Xyz < RDoc::Parser
parse_files_matching /\.xyz$/
def initialize top_level, file_name, content, , stats
super
# extra initialization if needed
end
def scan
# parse file and fill in @top_level
end
end
Defined Under Namespace
Modules: RubyTools, Text Classes: C, ChangeLog, Markdown, RD, RipperStateLex, Ruby, Simple
Class Attribute Summary collapse
-
.parsers ⇒ Object
readonly
An Array of arrays that maps file extension (or name) regular expressions to parser classes that will parse matching filenames.
Instance Attribute Summary collapse
-
#file_name ⇒ Object
readonly
The name of the file being parsed.
Class Method Summary collapse
-
.alias_extension(old_ext, new_ext) ⇒ Object
Alias an extension to another extension.
-
.binary?(file) ⇒ Boolean
Determines if the file is a “binary” file which basically means it has content that an RDoc parser shouldn’t try to consume.
-
.can_parse(file_name) ⇒ Object
Return a parser that can handle a particular extension.
-
.can_parse_by_name(file_name) ⇒ Object
Returns a parser that can handle the extension for
file_name
. -
.check_modeline(file_name) ⇒ Object
Returns the file type from the modeline in
file_name
. -
.for(top_level, file_name, content, options, stats) ⇒ Object
Finds and instantiates the correct parser for the given
file_name
andcontent
. -
.parse_files_matching(regexp) ⇒ Object
Record which file types this parser can understand.
-
.remove_modeline(content) ⇒ Object
Removes an emacs-style modeline from the first line of the document.
-
.use_markup(content) ⇒ Object
If there is a
markup: parser_name
comment at the front of the file, use it to determine the parser. -
.zip?(file) ⇒ Boolean
Checks if
file
is a zip file in disguise.
Instance Method Summary collapse
-
#handle_tab_width(body) ⇒ Object
Normalizes tabs in
body
. -
#initialize(top_level, file_name, content, options, stats) ⇒ Parser
constructor
Creates a new Parser storing
top_level
,file_name
,content
,options
andstats
in instance variables.
Constructor Details
#initialize(top_level, file_name, content, options, stats) ⇒ Parser
Creates a new Parser storing top_level
, file_name
, content
, options
and stats
in instance variables. In @preprocess an RDoc::Markup::PreProcess object is created which allows processing of directives.
254 255 256 257 258 259 260 261 262 263 264 265 266 |
# File 'lib/rdoc/parser.rb', line 254 def initialize top_level, file_name, content, , stats @top_level = top_level @top_level.parser = self.class @store = @top_level.store @file_name = file_name @content = content @options = @stats = stats @preprocess = RDoc::Markup::PreProcess.new @file_name, @options.rdoc_include @preprocess. = @options end |
Class Attribute Details
.parsers ⇒ Object (readonly)
An Array of arrays that maps file extension (or name) regular expressions to parser classes that will parse matching filenames.
Use parse_files_matching to register a parser’s file extensions.
45 46 47 |
# File 'lib/rdoc/parser.rb', line 45 def parsers @parsers end |
Instance Attribute Details
#file_name ⇒ Object (readonly)
The name of the file being parsed
52 53 54 |
# File 'lib/rdoc/parser.rb', line 52 def file_name @file_name end |
Class Method Details
.alias_extension(old_ext, new_ext) ⇒ Object
Alias an extension to another extension. After this call, files ending “new_ext” will be parsed using the same parser as “old_ext”
58 59 60 61 62 63 64 65 66 67 68 |
# File 'lib/rdoc/parser.rb', line 58 def self.alias_extension(old_ext, new_ext) old_ext = old_ext.sub(/^\.(.*)/, '\1') new_ext = new_ext.sub(/^\.(.*)/, '\1') parser = can_parse_by_name "xxx.#{old_ext}" return false unless parser RDoc::Parser.parsers.unshift [/\.#{new_ext}$/, parser] true end |
.binary?(file) ⇒ Boolean
Determines if the file is a “binary” file which basically means it has content that an RDoc parser shouldn’t try to consume.
74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 |
# File 'lib/rdoc/parser.rb', line 74 def self.binary?(file) return false if file =~ /\.(rdoc|txt)$/ s = File.read(file, 1024) or return false return true if s[0, 2] == Marshal.dump('')[0, 2] or s.index("\x00") mode = 'r:utf-8' # default source encoding has been changed to utf-8 s.sub!(/\A#!.*\n/, '') # assume shebang line isn't longer than 1024. encoding = s[/^\s*\#\s*(?:-\*-\s*)?(?:en)?coding:\s*([^\s;]+?)(?:-\*-|[\s;])/, 1] mode = "rb:#{encoding}" if encoding s = File.open(file, mode) {|f| f.gets(nil, 1024)} not s.valid_encoding? end |
.can_parse(file_name) ⇒ Object
Return a parser that can handle a particular extension
107 108 109 110 111 112 113 114 |
# File 'lib/rdoc/parser.rb', line 107 def self.can_parse file_name parser = can_parse_by_name file_name # HACK Selenium hides a jar file using a .txt extension return if parser == RDoc::Parser::Simple and zip? file_name parser end |
.can_parse_by_name(file_name) ⇒ Object
Returns a parser that can handle the extension for file_name
. This does not depend upon the file being readable.
120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 |
# File 'lib/rdoc/parser.rb', line 120 def self.can_parse_by_name file_name _, parser = RDoc::Parser.parsers.find { |regexp,| regexp =~ file_name } # The default parser must not parse binary files ext_name = File.extname file_name return parser if ext_name.empty? if parser == RDoc::Parser::Simple and ext_name !~ /txt|rdoc/ then case mode = check_modeline(file_name) when nil, 'rdoc' then # continue else RDoc::Parser.parsers.find { |_, p| return p if mode.casecmp?(p.name[/\w+\z/]) } return nil end end parser rescue Errno::EACCES end |
.check_modeline(file_name) ⇒ Object
Returns the file type from the modeline in file_name
143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 |
# File 'lib/rdoc/parser.rb', line 143 def self.check_modeline file_name line = File.open file_name do |io| io.gets end /-\*-\s*(.*?\S)\s*-\*-/ =~ line return nil unless type = $1 if /;/ =~ type then return nil unless /(?:\s|\A)mode:\s*([^\s;]+)/i =~ type type = $1 end return nil if /coding:/i =~ type type.downcase rescue ArgumentError rescue Encoding::InvalidByteSequenceError # invalid byte sequence end |
.for(top_level, file_name, content, options, stats) ⇒ Object
Finds and instantiates the correct parser for the given file_name
and content
.
169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 |
# File 'lib/rdoc/parser.rb', line 169 def self.for top_level, file_name, content, , stats return if binary? file_name parser = use_markup content unless parser then parse_name = file_name # If no extension, look for shebang if file_name !~ /\.\w+$/ && content =~ %r{\A#!(.+)} then shebang = $1 case shebang when %r{env\s+ruby}, %r{/ruby} parse_name = 'dummy.rb' end end parser = can_parse parse_name end return unless parser content = remove_modeline content parser.new top_level, file_name, content, , stats rescue SystemCallError nil end |
.parse_files_matching(regexp) ⇒ Object
Record which file types this parser can understand.
It is ok to call this multiple times.
203 204 205 |
# File 'lib/rdoc/parser.rb', line 203 def self.parse_files_matching(regexp) RDoc::Parser.parsers.unshift [regexp, self] end |
.remove_modeline(content) ⇒ Object
Removes an emacs-style modeline from the first line of the document
210 211 212 |
# File 'lib/rdoc/parser.rb', line 210 def self.remove_modeline content content.sub(/\A.*-\*-\s*(.*?\S)\s*-\*-.*\r?\n/, '') end |
.use_markup(content) ⇒ Object
If there is a markup: parser_name
comment at the front of the file, use it to determine the parser. For example:
# markup: rdoc
# Class comment can go here
class C
end
The comment should appear as the first line of the content
.
If the content contains a shebang or editor modeline the comment may appear on the second or third line.
Any comment style may be used to hide the markup comment.
231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 |
# File 'lib/rdoc/parser.rb', line 231 def self.use_markup content markup = content.lines.first(3).grep(/markup:\s+(\w+)/) { $1 }.first return unless markup # TODO Ruby should be returned only when the filename is correct return RDoc::Parser::Ruby if %w[tomdoc markdown].include? markup markup = Regexp.escape markup _, selected = RDoc::Parser.parsers.find do |_, parser| /^#{markup}$/i =~ parser.name.sub(/.*:/, '') end selected end |
.zip?(file) ⇒ Boolean
Checks if file
is a zip file in disguise. Signatures from www.garykessler.net/library/file_sigs.html
94 95 96 97 98 99 100 101 102 |
# File 'lib/rdoc/parser.rb', line 94 def self.zip? file zip_signature = File.read file, 4 zip_signature == "PK\x03\x04" or zip_signature == "PK\x05\x06" or zip_signature == "PK\x07\x08" rescue false end |
Instance Method Details
#handle_tab_width(body) ⇒ Object
Normalizes tabs in body
274 275 276 277 278 279 280 281 282 283 284 285 286 287 |
# File 'lib/rdoc/parser.rb', line 274 def handle_tab_width(body) if /\t/ =~ body tab_width = @options.tab_width body.split(/\n/).map do |line| 1 while line.gsub!(/\t+/) do b, e = $~.offset(0) ' ' * (tab_width * (e-b) - b % tab_width) end line end.join "\n" else body end end |