Class: YARD::DocstringParser

Inherits:
Object
  • Object
show all
Defined in:
lib/yard/docstring_parser.rb

Overview

Parses text and creates a Docstring object to represent documentation for a CodeObjects::Base. To create a new docstring, you should initialize the parser and call #parse followed by #to_docstring.

Subclassing Notes

The DocstringParser can be subclassed and substituted during parsing by setting the YARD::Docstring.default_parser attribute with the name of the subclass. This allows developers to change the way docstrings are parsed, allowing for completely different docstring syntaxes.

Examples:

Creating a Docstring with a DocstringParser

DocstringParser.new.parse("text here").to_docstring

Creating a Custom DocstringParser

# Parses docstrings backwards!
class ReverseDocstringParser
  def parse_content(content)
    super(content.reverse)
  end
end

# Set the parser as default when parsing
YARD::Docstring.default_parser = ReverseDocstringParser

See Also:

Since:

  • 0.8.0

Constant Summary collapse

META_MATCH =

The regular expression to match the tag syntax

Since:

  • 0.8.0

/^@(!)?((?:\w\.?)+)(?:\s+(.*))?$/i

Instance Attribute Summary collapse

Creation and Conversion Methods collapse

Parsing Methods collapse

Parser Callback Methods collapse

Tag Manipulation Methods collapse

Constructor Details

#initialize(library = Tags::Library.instance) ⇒ DocstringParser

Creates a new parser to parse docstring data

Parameters:

  • library (Tags::Library) (defaults to: Tags::Library.instance) —

    a tag library for recognizing tags.

Since:

  • 0.8.0



81
82
83
84
85
86
87
88
89
90
91
 
# File 'lib/yard/docstring_parser.rb', line 81

def initialize(library = Tags::Library.instance)
  @text = ""
  @raw_text = ""
  @tags = []
  @directives = []
  @library = library
  @object = nil
  @reference = nil
  @handler = nil
  @state = OpenStruct.new
end

Instance Attribute Details

#directives ⇒ Array<Tags::Directive>

Returns a list of directives identified by the parser. This list will not be passed on to the Docstring object.

Returns:

  • (Array<Tags::Directive>) —

    a list of directives identified by the parser. This list will not be passed on to the Docstring object.

Since:

  • 0.8.0



45
46
47
 
# File 'lib/yard/docstring_parser.rb', line 45

def directives
  @directives
end

#handler ⇒ Handlers::Base?

Returns the handler parsing this docstring. May be nil if this docstring parser is not initialized through.

Returns:

  • (Handlers::Base, nil) —

    the handler parsing this docstring. May be nil if this docstring parser is not initialized through

Since:

  • 0.8.0



66
67
68
 
# File 'lib/yard/docstring_parser.rb', line 66

def handler
  @handler
end

#library ⇒ Tags::Library

Returns the tag library being used to identify registered tags in the docstring.

Returns:

  • (Tags::Library) —

    the tag library being used to identify registered tags in the docstring.

Since:

  • 0.8.0



70
71
72
 
# File 'lib/yard/docstring_parser.rb', line 70

def library
  @library
end

#object ⇒ CodeObjects::Base?

Returns the object associated with the docstring being parsed. May be nil if the docstring is not attached to any object.

Returns:

  • (CodeObjects::Base, nil) —

    the object associated with the docstring being parsed. May be nil if the docstring is not attached to any object.

Since:

  • 0.8.0



56
57
58
 
# File 'lib/yard/docstring_parser.rb', line 56

def object
  @object
end

#raw_text ⇒ String

Returns the complete input string to the parser.

Returns:

  • (String) —

    the complete input string to the parser.

Since:

  • 0.8.0



36
37
38
 
# File 'lib/yard/docstring_parser.rb', line 36

def raw_text
  @raw_text
end

#reference ⇒ CodeObjects::Base?

Returns the object referenced by the docstring being parsed. May be nil if the docstring doesn’t refer to any object.

Returns:

  • (CodeObjects::Base, nil) —

    the object referenced by the docstring being parsed. May be nil if the docstring doesn’t refer to any object.

Since:

  • 0.8.0



61
62
63
 
# File 'lib/yard/docstring_parser.rb', line 61

def reference
  @reference
end

#state ⇒ OpenStruct

Returns any arbitrary state to be passed between tags during parsing. Mainly used by directives to coordinate behaviour (so that directives can be aware of other directives used in a docstring).

Returns:

  • (OpenStruct) —

    any arbitrary state to be passed between tags during parsing. Mainly used by directives to coordinate behaviour (so that directives can be aware of other directives used in a docstring).

Since:

  • 0.8.0



51
52
53
 
# File 'lib/yard/docstring_parser.rb', line 51

def state
  @state
end

#tags ⇒ Array<Tags::Tag>

Returns the list of meta-data tags identified by the parser.

Returns:

  • (Array<Tags::Tag>) —

    the list of meta-data tags identified by the parser

Since:

  • 0.8.0



40
41
42
 
# File 'lib/yard/docstring_parser.rb', line 40

def tags
  @tags
end

#text ⇒ String

Returns the parsed text portion of the docstring, with tags removed.

Returns:

  • (String) —

    the parsed text portion of the docstring, with tags removed.

Since:

  • 0.8.0



33
34
35
 
# File 'lib/yard/docstring_parser.rb', line 33

def text
  @text
end

Class Method Details

.after_parse {|parser| ... } ⇒ void

This method returns an undefined value.

Creates a callback that is called after a docstring is successfully parsed. Use this method to perform sanity checks on a docstring’s tag data, or add any extra tags automatically to a docstring.

Yields:

  • (parser) —

    a block to be called after a docstring is parsed

Yield Parameters:

  • parser (DocstringParser) —

    the docstring parser object with all directives and tags created.

Yield Returns:

  • (void)

Since:

  • 0.8.0



266
267
268
 
# File 'lib/yard/docstring_parser.rb', line 266

def self.after_parse(&block)
  after_parse_callbacks << block
end

.after_parse_callbacks ⇒ Array<Proc>

Returns the after_parse callback proc objects.

Returns:

Since:

  • 0.8.0



271
272
273
 
# File 'lib/yard/docstring_parser.rb', line 271

def self.after_parse_callbacks
  @after_parse_callbacks ||= []
end

Instance Method Details

#create_directive(tag_name, tag_buf) ⇒ Tags::Directive

Creates a new directive using the registered #library

Returns:

Since:

  • 0.8.0



232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
 
# File 'lib/yard/docstring_parser.rb', line 232

def create_directive(tag_name, tag_buf)
  if library.has_directive?(tag_name)
    dir = library.directive_create(tag_name, tag_buf, self)
    if dir.is_a?(Tags::Directive)
      @directives << dir
      dir
    end
  else
    log.warn "Unknown directive @!#{tag_name}" +
             (object ? " in file `#{object.file}` near line #{object.line}" : "")
    nil
  end
rescue Tags::TagFormatError
  log.warn "Invalid directive format for @!#{tag_name}" +
           (object ? " in file `#{object.file}` near line #{object.line}" : "")
  nil
end

#create_ref_tag(tag_name, name, object_name) ⇒ Object

Creates a Tags::RefTag

Since:

  • 0.8.0



226
227
228
 
# File 'lib/yard/docstring_parser.rb', line 226

def create_ref_tag(tag_name, name, object_name)
  @tags << Tags::RefTagList.new(tag_name, P(object, object_name), name)
end

#create_tag(tag_name, tag_buf = '') ⇒ Tags::Tag, Tags::RefTag

Creates a tag from the tag factory.

To add an already created tag object, append it to #tags.

Parameters:

  • tag_name (String) —

    the tag name

  • tag_buf (String) (defaults to: '') —

    the text attached to the tag with newlines removed.

Returns:

Since:

  • 0.8.0



209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
 
# File 'lib/yard/docstring_parser.rb', line 209

def create_tag(tag_name, tag_buf = '')
  if tag_buf =~ /\A\s*(?:(\S+)\s+)?\(\s*see\s+(\S+)\s*\)\s*\Z/
    return create_ref_tag(tag_name, $1, $2)
  end

  if library.has_tag?(tag_name)
    @tags += [library.tag_create(tag_name, tag_buf)].flatten
  else
    log.warn "Unknown tag @#{tag_name}" +
             (object ? " in file `#{object.file}` near line #{object.line}" : "")
  end
rescue Tags::TagFormatError
  log.warn "Invalid tag format for @#{tag_name}" +
           (object ? " in file `#{object.file}` near line #{object.line}" : "")
end

#parse(content, object = nil, handler = nil) ⇒ self

Parses all content and returns itself.

Parameters:

  • content (String) —

    the docstring text to parse

  • object (CodeObjects::Base) (defaults to: nil) —

    the object that the docstring is attached to. Will be passed to directives to act on this object.

  • handler (Handlers::Base, nil) (defaults to: nil) —

    the handler object that is parsing this object. May be nil if this parser is not being called from a Parser::SourceParser context.

Returns:

  • (self) —

    the parser object. To get the docstring, call #to_docstring.

See Also:

Since:

  • 0.8.0



113
114
115
116
117
118
119
120
121
122
 
# File 'lib/yard/docstring_parser.rb', line 113

def parse(content, object = nil, handler = nil)
  @object = object
  @handler = handler
  @reference, @raw_text = detect_reference(content)
  text = parse_content(@raw_text)
  @text = text.strip
  call_directives_after_parse
  post_process
  self
end

#parse_content(content) ⇒ Object

Note:

Subclasses can override this method to perform custom parsing of content data.

Parses a given block of text.

Parameters:

  • content (String) —

    the content to parse

Since:

  • 0.8.0



129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
 
# File 'lib/yard/docstring_parser.rb', line 129

def parse_content(content)
  content = content.split(/\r?\n/) if content.is_a?(String)
  return '' if !content || content.empty?
  docstring = String.new("")

  indent = content.first[/^\s*/].length
  last_indent = 0
  orig_indent = 0
  directive = false
  last_line = ""
  tag_name = nil
  tag_buf = []

  (content + ['']).each_with_index do |line, index|
    indent = line[/^\s*/].length
    empty = (line =~ /^\s*$/ ? true : false)
    done = content.size == index

    if tag_name && (((indent < orig_indent && !empty) || done ||
        (indent == 0 && !empty)) || (indent <= last_indent && line =~ META_MATCH))
      buf = tag_buf.join("\n")
      if directive || tag_is_directive?(tag_name)
        directive = create_directive(tag_name, buf)
        if directive
          docstring << parse_content(directive.expanded_text).chomp
        end
      else
        create_tag(tag_name, buf)
      end
      tag_name = nil
      tag_buf = []
      directive = false
      orig_indent = 0
    end

    # Found a meta tag
    if line =~ META_MATCH
      directive = $1
      tag_name = $2
      tag_buf = [($3 || '')]
    elsif tag_name && indent >= orig_indent && !empty
      orig_indent = indent if orig_indent == 0
      # Extra data added to the tag on the next line
      last_empty = last_line =~ /^[ \t]*$/ ? true : false

      tag_buf << '' if last_empty
      tag_buf << line.gsub(/^[ \t]{#{orig_indent}}/, '')
    elsif !tag_name
      # Regular docstring text
      docstring << line
      docstring << "\n"
    end

    last_indent = indent
    last_line = line
  end

  docstring
end

#post_process ⇒ void

This method returns an undefined value.

Call post processing callbacks on parser. This is called implicitly by parser. Use this when manually configuring a YARD::Docstring object.

Since:

  • 0.8.0



196
197
198
 
# File 'lib/yard/docstring_parser.rb', line 196

def post_process
  call_after_parse_callbacks
end

#tag_is_directive?(tag_name) ⇒ Boolean

Backward compatibility to detect old tags that should be specified as directives in 0.8 and onward.

Returns:

  • (Boolean)

Since:

  • 0.8.0



252
253
254
255
 
# File 'lib/yard/docstring_parser.rb', line 252

def tag_is_directive?(tag_name)
  list = %w(attribute endgroup group macro method scope visibility)
  list.include?(tag_name)
end

#to_docstring ⇒ Docstring

Returns translates parsed text into a Docstring object.

Returns:

  • (Docstring) —

    translates parsed text into a Docstring object.

Since:

  • 0.8.0



95
96
97
 
# File 'lib/yard/docstring_parser.rb', line 95

def to_docstring
  Docstring.new!(text, tags, object, raw_text, reference)
end