Class: YARD::DocstringParser

Inherits:
Object
  • Object
show all
Defined in:
lib/yard/docstring_parser.rb

Overview

Parses text and creates a Docstring object to represent documentation for a CodeObjects::Base. To create a new docstring, you should initialize the parser and call #parse followed by #to_docstring.

Subclassing Notes

The DocstringParser can be subclassed and subtituted during parsing by setting the YARD::Docstring.default_parser attribute with the name of the subclass. This allows developers to change the way docstrings are parsed, allowing for completely different docstring syntaxes.

Examples:

Creating a Docstring with a DocstringParser

DocstringParser.new.parse("text here").to_docstring

Creating a Custom DocstringParser

# Parses docstrings backwards!
class ReverseDocstringParser
  def parse_content(content)
    super(content.reverse)
  end
end

# Set the parser as default when parsing
YARD::Docstring.default_parser = ReverseDocstringParser

See Also:

Since:

  • 0.8.0

Constant Summary collapse

META_MATCH =

The regular expression to match the tag syntax

Since:

  • 0.8.0

/^@(!)?((?:\w\.?)+)(?:\s+(.*))?$/i

Instance Attribute Summary collapse

Creation and Conversion Methods collapse

Parsing Methods collapse

Tag Manipulation Methods collapse

Parser Callback Methods collapse

Constructor Details

#initialize(library = Tags::Library.instance) ⇒ DocstringParser

Creates a new parser to parse docstring data

Parameters:

  • library (Tags::Library) (defaults to: Tags::Library.instance) —

    a tag library for recognizing tags.

Since:

  • 0.8.0



80
81
82
83
84
85
86
87
88
89
90
 
# File 'lib/yard/docstring_parser.rb', line 80

def initialize(library = Tags::Library.instance)
  @text = ""
  @raw_text = ""
  @tags = []
  @directives = []
  @library = library
  @object = nil
  @reference = nil
  @handler = nil
  @state = OpenStruct.new
end

Instance Attribute Details

#directives ⇒ Array<Directive>

Returns a list of directives identified by the parser. This list will not be passed on to the Docstring object.

Returns:

  • (Array<Directive>) —

    a list of directives identified by the parser. This list will not be passed on to the Docstring object.

Since:

  • 0.8.0



44
45
46
 
# File 'lib/yard/docstring_parser.rb', line 44

def directives
  @directives
end

#handler ⇒ Handlers::Base?

Returns the handler parsing this docstring. May be nil if this docstring parser is not initialized through.

Returns:

  • (Handlers::Base, nil) —

    the handler parsing this docstring. May be nil if this docstring parser is not initialized through

Since:

  • 0.8.0



65
66
67
 
# File 'lib/yard/docstring_parser.rb', line 65

def handler
  @handler
end

#library ⇒ Tags::Library

Returns the tag library being used to identify registered tags in the docstring.

Returns:

  • (Tags::Library) —

    the tag library being used to identify registered tags in the docstring.

Since:

  • 0.8.0



69
70
71
 
# File 'lib/yard/docstring_parser.rb', line 69

def library
  @library
end

#object ⇒ CodeObjects::Base?

Returns the object associated with the docstring being parsed. May be nil if the docstring is not attached to any object.

Returns:

  • (CodeObjects::Base, nil) —

    the object associated with the docstring being parsed. May be nil if the docstring is not attached to any object.

Since:

  • 0.8.0



55
56
57
 
# File 'lib/yard/docstring_parser.rb', line 55

def object
  @object
end

#raw_text ⇒ String

Returns the complete input string to the parser.

Returns:

  • (String) —

    the complete input string to the parser.

Since:

  • 0.8.0



35
36
37
 
# File 'lib/yard/docstring_parser.rb', line 35

def raw_text
  @raw_text
end

#reference ⇒ CodeObjects::Base?

Returns the object referenced by the docstring being parsed. May be nil if the docstring doesn’t refer to any object.

Returns:

  • (CodeObjects::Base, nil) —

    the object referenced by the docstring being parsed. May be nil if the docstring doesn’t refer to any object.

Since:

  • 0.8.0



60
61
62
 
# File 'lib/yard/docstring_parser.rb', line 60

def reference
  @reference
end

#state ⇒ OpenStruct

Returns any arbitrary state to be passed between tags during parsing. Mainly used by directives to coordinate behaviour (so that directives can be aware of other directives used in a docstring).

Returns:

  • (OpenStruct) —

    any arbitrary state to be passed between tags during parsing. Mainly used by directives to coordinate behaviour (so that directives can be aware of other directives used in a docstring).

Since:

  • 0.8.0



50
51
52
 
# File 'lib/yard/docstring_parser.rb', line 50

def state
  @state
end

#tags ⇒ Array<Tag>

Returns the list of meta-data tags identified by the parser.

Returns:

  • (Array<Tag>) —

    the list of meta-data tags identified by the parser

Since:

  • 0.8.0



39
40
41
 
# File 'lib/yard/docstring_parser.rb', line 39

def tags
  @tags
end

#text ⇒ String

Returns the parsed text portion of the docstring, with tags removed.

Returns:

  • (String) —

    the parsed text portion of the docstring, with tags removed.

Since:

  • 0.8.0



32
33
34
 
# File 'lib/yard/docstring_parser.rb', line 32

def text
  @text
end

Class Method Details

.after_parse {|parser| ... } ⇒ void

This method returns an undefined value.

Creates a callback that is called after a docstring is successfully parsed. Use this method to perform sanity checks on a docstring’s tag data, or add any extra tags automatically to a docstring.

Yields:

  • (parser) —

    a block to be called after a docstring is parsed

Yield Parameters:

  • parser (DocstringParser) —

    the docstring parser object with all directives and tags created.

Yield Returns:

  • (void)

Since:

  • 0.8.0



286
287
288
 
# File 'lib/yard/docstring_parser.rb', line 286

def self.after_parse(&block)
  self.after_parse_callbacks << block
end

.after_parse_callbacks ⇒ Array<Proc>

Returns the after_parse callback proc objects.

Returns:

Since:

  • 0.8.0



291
292
293
 
# File 'lib/yard/docstring_parser.rb', line 291

def self.after_parse_callbacks
  @after_parse_callbacks ||= []
end

Instance Method Details

#create_directive(tag_name, tag_buf) ⇒ Directive

Creates a new directive using the registered #library

Returns:

  • (Directive) —

    the directive object that is created

Since:

  • 0.8.0



214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
 
# File 'lib/yard/docstring_parser.rb', line 214

def create_directive(tag_name, tag_buf)
  if library.has_directive?(tag_name)
    dir = library.directive_create(tag_name, tag_buf, self)
    if dir.is_a?(Tags::Directive)
      @directives << dir
      dir
    end
  else
    log.warn "Unknown directive @!#{tag_name}" +
      (object ? " in file `#{object.file}` near line #{object.line}" : "")
    nil
  end
rescue Tags::TagFormatError
  log.warn "Invalid directive format for @!#{tag_name}" +
    (object ? " in file `#{object.file}` near line #{object.line}" : "")
  nil
end

#create_ref_tag(tag_name, name, object_name) ⇒ Object

Creates a Tags::RefTag

Since:

  • 0.8.0



208
209
210
 
# File 'lib/yard/docstring_parser.rb', line 208

def create_ref_tag(tag_name, name, object_name)
  @tags << Tags::RefTagList.new(tag_name, P(object, object_name), name)
end

#create_tag(tag_name, tag_buf = '') ⇒ Tags::Tag, Tags::RefTag

Creates a tag from the tag factory.

To add an already created tag object, append it to #tags.

Parameters:

  • tag_name (String) —

    the tag name

  • tag_buf (String) (defaults to: '') —

    the text attached to the tag with newlines removed.

Returns:

Since:

  • 0.8.0



191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
 
# File 'lib/yard/docstring_parser.rb', line 191

def create_tag(tag_name, tag_buf = '')
  if tag_buf =~ /\A\s*(?:(\S+)\s+)?\(\s*see\s+(\S+)\s*\)\s*\Z/
    return create_ref_tag(tag_name, $1, $2)
  end

  if library.has_tag?(tag_name)
    @tags += [library.tag_create(tag_name, tag_buf)].flatten
  else
    log.warn "Unknown tag @#{tag_name}" +
      (object ? " in file `#{object.file}` near line #{object.line}" : "")
  end
rescue Tags::TagFormatError
  log.warn "Invalid tag format for @#{tag_name}" +
    (object ? " in file `#{object.file}` near line #{object.line}" : "")
end

#parse(content, object = nil, handler = nil) ⇒ self

Parses all content and returns itself.

Parameters:

  • content (String) —

    the docstring text to parse

  • object (CodeObjects::Base) (defaults to: nil) —

    the object that the docstring is attached to. Will be passed to directives to act on this object.

  • handler (Handlers::Base, nil) (defaults to: nil) —

    the handler object that is parsing this object. May be nil if this parser is not being called from a Parser::SourceParser context.

Returns:

  • (self) —

    the parser object. To get the docstring, call #to_docstring.

See Also:

Since:

  • 0.8.0



112
113
114
115
116
117
118
119
120
121
122
 
# File 'lib/yard/docstring_parser.rb', line 112

def parse(content, object = nil, handler = nil)
  @object = object
  @handler = handler
  @reference, @raw_text = detect_reference(content)
  text = parse_content(@raw_text)
  # Remove trailing/leading whitespace / newlines
  @text = text.gsub(/\A[\r\n\s]+|[\r\n\s]+\Z/, '')
  call_directives_after_parse
  call_after_parse_callbacks
  self
end

#parse_content(content) ⇒ Object

Note:

Subclasses can override this method to perform custom parsing of content data.

Parses a given block of text.

Parameters:

  • content (String) —

    the content to parse

Since:

  • 0.8.0



129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
 
# File 'lib/yard/docstring_parser.rb', line 129

def parse_content(content)
  content = content.split(/\r?\n/) if content.is_a?(String)
  return '' if !content || content.empty?
  docstring = ""

  indent, last_indent = content.first[/^\s*/].length, 0
  orig_indent = 0
  directive = false
  last_line = ""
  tag_name, tag_klass, tag_buf = nil, nil, []

  (content+['']).each_with_index do |line, index|
    indent = line[/^\s*/].length
    empty = (line =~ /^\s*$/ ? true : false)
    done = content.size == index

    if tag_name && (((indent < orig_indent && !empty) || done ||
        (indent == 0 && !empty)) || (indent <= last_indent && line =~ META_MATCH))
      buf = tag_buf.join("\n")
      if directive || tag_is_directive?(tag_name)
        directive = create_directive(tag_name, buf)
        if directive
          docstring << parse_content(directive.expanded_text).chomp
        end
      else
        create_tag(tag_name, buf)
      end
      tag_name, tag_buf, directive = nil, [], false
      orig_indent = 0
    end

    # Found a meta tag
    if line =~ META_MATCH
      directive, tag_name, tag_buf = $1, $2, [($3 || '')]
    elsif tag_name && indent >= orig_indent && !empty
      orig_indent = indent if orig_indent == 0
      # Extra data added to the tag on the next line
      last_empty = last_line =~ /^[ \t]*$/ ? true : false

      tag_buf << '' if last_empty
      tag_buf << line.gsub(/^[ \t]{#{orig_indent}}/, '')
    elsif !tag_name
      # Regular docstring text
      docstring << line << "\n"
    end

    last_indent = indent
    last_line = line
  end

  docstring
end

#tag_is_directive?(tag_name) ⇒ Boolean

Backward compatibility to detect old tags that should be specified as directives in 0.8 and onward.

Returns:

  • (Boolean)

Since:

  • 0.8.0



234
235
236
237
 
# File 'lib/yard/docstring_parser.rb', line 234

def tag_is_directive?(tag_name)
  list = %w(attribute endgroup group macro method scope visibility)
  list.include?(tag_name)
end

#to_docstring ⇒ Docstring

Returns translates parsed text into a Docstring object.

Returns:

  • (Docstring) —

    translates parsed text into a Docstring object.

Since:

  • 0.8.0



94
95
96
 
# File 'lib/yard/docstring_parser.rb', line 94

def to_docstring
  Docstring.new!(text, tags, object, raw_text, reference)
end