Class: TomParse::Parser

Inherits:
Object
  • Object
show all
Defined in:
lib/tomparse/parser.rb

Overview

TODO:

Currently uses lazy evaluation, eventually this should be removed and simply parsed all at once.

Encapsulate parsed tomdoc documentation.

Constant Summary collapse

TOMDOC_STATUS =

Recognized description status.

['Internal', 'Public', 'Deprecated']

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(text, parse_options = {}) ⇒ Object

Initialize a TomDoc object.

Parameters:

  • text —

    The raw text of a method or class/module comment.



18
19
20
21
22
23
24
25
26
27
28
29
30
31
 
# File 'lib/tomparse/parser.rb', line 18

def initialize(text, parse_options={})
  @raw = text.to_s.strip

  @arguments        = []
  @options          = []
  @examples         = []
  @returns          = []
  @raises           = []
  @signatures       = []
  @signature_fields = []
  @tags             = []

  #parse unless @raw.empty?
end

Instance Attribute Details

#raw ⇒ Object

Returns the value of attribute raw.



11
12
13
 
# File 'lib/tomparse/parser.rb', line 11

def raw
  @raw
end

Class Method Details

.valid?(text) ⇒ Boolean

Validate given comment text.

Returns:

  • (Boolean) —

    true if comment is valid, otherwise false.



43
44
45
 
# File 'lib/tomparse/parser.rb', line 43

def self.valid?(text)
  new(text).valid?
end

Instance Method Details

#argument_line?(line) ⇒ Boolean (private)

Check if a line of text could be an argument definition. I.e. it has a word followed by a dash.

Returns:

  • (Boolean) —

    Return



430
431
432
 
# File 'lib/tomparse/parser.rb', line 430

def argument_line?(line)
  /^\w+\s+\-/m =~ line.strip
end

#arguments ⇒ Object Also known as: args

Arguments list.

Returns:

  • list of arguments.



137
138
139
140
141
 
# File 'lib/tomparse/parser.rb', line 137

def arguments
  parsed {
    @arguments
  }
end

#clean_example(text) ⇒ Object (private) 



675
676
677
678
679
680
681
682
683
684
685
686
 
# File 'lib/tomparse/parser.rb', line 675

def clean_example(text)
  lines = text.rstrip.lines.to_a
  # remove blank lines from top
  lines.shift while lines.first.strip.empty?
  # determine the indention
  indent = least_indent(lines)
  # remove the indention
  tab = " " * indent
  lines = lines.map{ |line| line.sub(tab, '') }
  # put the lines back together
  lines.join
end

#deprecated? ⇒ Boolean

Check if method is deprecated.

Returns:

  • (Boolean) —

    true if method is deprecated.



247
248
249
250
251
 
# File 'lib/tomparse/parser.rb', line 247

def deprecated?
  parsed {
    @status == 'Deprecated'
  }
end

#description ⇒ Object

Description of method or class/module.

Returns:

  • description String.



128
129
130
131
132
 
# File 'lib/tomparse/parser.rb', line 128

def description
  parsed {
    @description
  }
end

#examples ⇒ String

List of use examples of a method or class/module.

Returns:

  • (String) —

    of examples.



157
158
159
160
161
 
# File 'lib/tomparse/parser.rb', line 157

def examples
  parsed {
    @examples
  }
end

#internal? ⇒ Boolean

Check if method is internal.

Returns:

  • (Boolean) —

    true if method is internal.



238
239
240
241
242
 
# File 'lib/tomparse/parser.rb', line 238

def internal?
  parsed {
    @status == 'Internal'
  }
end

#least_indent(lines) ⇒ Object (private)

Given a multi-line string, determine the minimum indention.



689
690
691
692
693
694
695
696
697
698
 
# File 'lib/tomparse/parser.rb', line 689

def least_indent(lines)
  indents = []
  lines.map do |line|
    next if line.strip.empty?
    if md = /^\ */.match(line)
      indents << md[0].size
    end
  end
  indents.min || 0
end

#options ⇒ Object Also known as: keyword_arguments

Keyword arguments, aka Options.

Returns:

  • list of options.



147
148
149
150
151
 
# File 'lib/tomparse/parser.rb', line 147

def options
  parsed {
    @options
  }
end

#parse ⇒ Object

This method is part of a private API. You should avoid using this method if possible, as it may be removed or be changed in the future.

Parse the Tomdoc formatted comment.

Returns:

  • true if there was a comment to parse.



314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
 
# File 'lib/tomparse/parser.rb', line 314

def parse
  @parsed = true

  sections = smart_split(tomdoc)

  return false if sections.empty?

  # We are assuming that the first section is always description.
  # And it should be, but people aren't always proper, so perhaps
  # this can be made a little smarter in the future.
  parse_description(sections.shift)

  # The second section may be arguments.
  if sections.first && sections.first =~ /^\w+\s+\-/m
    parse_arguments(sections.shift)
  end

  current = sections.shift
  while current
    case type = section_type(current)
    when :arguments
      parse_arguments(current)
    when :options
      parse_options(current)
    when :example
      parse_example(current)
    when :examples
      parse_examples(current)
    when :yields
      parse_yields(current)
    when :returns
      parse_returns(current)
    when :raises
      parse_raises(current)
    when :signature
      parse_signature(current)
    when Symbol
      parse_tag(current)
    end
    current = sections.shift
  end

  return @parsed
end

#parse_arguments(section) ⇒ void (private)

This method returns an undefined value.

Parse arguments section. Arguments occur subsequent to the description.

Parameters:

  • section —

    String containing argument definitions.



487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
 
# File 'lib/tomparse/parser.rb', line 487

def parse_arguments(section)
  args = []
  last_indent = nil

  section.lines.each do |line|
    next if /^Arguments\s*$/i =~ line  # optional header
    next if line.strip.empty?
    indent = line.scan(/^\s*/)[0].to_s.size

    if last_indent && indent >= last_indent
      args.last.description << "\r\n" + line
    else
      param, desc = line.split(" - ")
      args << Argument.new(param.strip, desc.to_s.strip) if param #&& desc
      last_indent = indent + 1
    end
  end

  args.each do |arg|
    arg.parse(arg.description)
  end

  @arguments = args
end

#parse_description(section) ⇒ void (private)

This method returns an undefined value.

Parse description.

Parameters:

  • section —

    String containig description.



468
469
470
471
472
473
474
475
476
477
478
479
 
# File 'lib/tomparse/parser.rb', line 468

def parse_description(section)
  if md = /^([A-Z]\w+\:)/.match(section)
    @status = md[1].chomp(':')
    if TOMDOC_STATUS.include?(@status)
      @description = md.post_match.strip
    else
      @description = section.strip
    end
  else
    @description = section.strip
  end   
end

#parse_example(section) ⇒ void (private)

This method returns an undefined value.

Parse example.

Parameters:

  • section —

    String starting with ‘Example`.



548
549
550
551
552
553
 
# File 'lib/tomparse/parser.rb', line 548

def parse_example(section)
  # remove the initial `Example` line and right strip
  section = section.sub(/.*?\n/, '')
  example = clean_example(section)
  @examples << example unless example.strip.empty?
end

#parse_examples(section) ⇒ void (private)

This method returns an undefined value.

Parse examples.

Parameters:

  • section —

    String starting with ‘Examples`.



560
561
562
563
564
565
566
567
568
 
# File 'lib/tomparse/parser.rb', line 560

def parse_examples(section)
  # remove the initial `Examples` line and right strip
  section = section.sub(/.*?\n/, '')
  section.split("\n\n").each do |ex|
    next if ex.strip.empty?
    example = clean_example(ex)
    @examples << example
  end
end

#parse_options(section) ⇒ void (private)

This method returns an undefined value.

the description.

Parameters:

  • section —

    String containing argument definitions.



517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
 
# File 'lib/tomparse/parser.rb', line 517

def parse_options(section)
  opts = []
  last_indent = nil

  section.lines.each do |line|
    next if /^\s*Options\s*$/i =~ line  # optional header
    next if line.strip.empty?
    indent = line.scan(/^\s*/)[0].to_s.size

    if last_indent && indent > 0 && indent >= last_indent
      opts.last.description << "\r\n" + line
    else
      param, desc = line.split(" - ")
      opts << Option.new(param.strip, desc.strip) if param && desc
    end

    last_indent = indent
  end

  #opts.each do |opt|
  #  opt.parse(arg.description)
  #end

  @options = opts
end

#parse_raises(section) ⇒ void (private)

This method returns an undefined value.

Parse raises section.

Parameters:

  • section —

    String contaning Raises text.



594
595
596
597
 
# File 'lib/tomparse/parser.rb', line 594

def parse_raises(section)
  text = section.gsub(/\s+/, ' ').strip
  @raises << text.strip
end

#parse_returns(section) ⇒ void (private)

This method returns an undefined value.

Parse returns section.

Parameters:

  • section —

    String contaning Returns and/or Raises lines.



584
585
586
587
 
# File 'lib/tomparse/parser.rb', line 584

def parse_returns(section)
  text = section.gsub(/\s+/, ' ').strip
  @returns << text
end

#parse_signature(section) ⇒ void (private)

This method returns an undefined value.

Parse signature section.

IMPORTANT! This is not mojombo TomDoc! Rather signatures are simply a list of alternate ways to call a method, e.g. when *args is used but only specific argument patterns are possible.

Parameters:

  • section —

    String starting with ‘Signature`.



608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
 
# File 'lib/tomparse/parser.rb', line 608

def parse_signature(section)
  signatures = []

  section = section.sub(/^\s*Signature(s)?/, '').strip

  lines = section.lines.to_a

  lines.each do |line|
    next if line.strip.empty?
    signatures << line.strip
  end

  @signatures = signatures

  #if line =~ /^\w+\s*\-/m
  #  parse_signature_fields(sections.shift)
  #end
end

#parse_tag(section) ⇒ void (private)

This method returns an undefined value.

Tags are arbitrary sections designated by a capitalized label and a colon.

Parameters:

  • label —

    String name of the tag.

  • section —

    String of the tag section.



662
663
664
665
666
667
668
669
670
671
 
# File 'lib/tomparse/parser.rb', line 662

def parse_tag(section)
  md = /^([A-Z]\w+)\:\ /m.match(section)
 
  label = md[1]
  desc  = md.post_match

  warn "No label?" unless label

  @tags << [label, desc.strip] if label
end

#parse_yields(section) ⇒ void (private)

This method returns an undefined value.

Parse yields section.

Parameters:

  • section —

    String contaning Yields line.



575
576
577
 
# File 'lib/tomparse/parser.rb', line 575

def parse_yields(section)
  @yields = section.strip
end

#parsed(&block) ⇒ Object (private)

Has the comment been parsed yet?



362
363
364
365
 
# File 'lib/tomparse/parser.rb', line 362

def parsed(&block)
  parse unless @parsed
  block.call
end

#public? ⇒ Boolean

Check if method is public.

Returns:

  • (Boolean) —

    true if method is public.



229
230
231
232
233
 
# File 'lib/tomparse/parser.rb', line 229

def public?
  parsed {
    @status == 'Public'
  }
end

#raises ⇒ Array

A list of errors a method might raise.

Returns:

  • (Array) —

    of method raises descriptions.



184
185
186
187
188
 
# File 'lib/tomparse/parser.rb', line 184

def raises
  parsed {
    @raises
  }
end

#returns ⇒ Array

The list of retrun values a method can return.

Returns:

  • (Array) —

    of method return descriptions.



175
176
177
178
179
 
# File 'lib/tomparse/parser.rb', line 175

def returns
  parsed {
    @returns
  }
end

#section_type(section) ⇒ Object (private)

Determine section type.



435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
 
# File 'lib/tomparse/parser.rb', line 435

def section_type(section)
  case section
  when /\AArguments\s*$/
    :arguments
  when /\AOptions\s*$/
    :options
  when /\AExamples\s*$/
    :examples
  when /\AExample\s*$/
    :example
  when /\ASignature(s)?\s*$/
    :signature
  when /^Yield(s)?/
    :yields
  when /^Return(s)?/
    :returns
  when /^Raise(s)?/
    :raises
  when /\A([A-Z]\w+)\:\ /
    $1.to_sym
  else
    nil
  end
end

#sections ⇒ Array

List of comment sections. These are divided simply on “nn”.

Returns:

  • (Array) —

    of comment sections.



119
120
121
122
123
 
# File 'lib/tomparse/parser.rb', line 119

def sections
  parsed {
    @sections
  }
end

#signature_fields ⇒ Array

A list of signature fields.

Returns:

  • (Array) —

    of field definitions.



202
203
204
205
206
 
# File 'lib/tomparse/parser.rb', line 202

def signature_fields
  parsed {
    @signature_fields
  }
end

#signatures ⇒ Array

A list of alternate method signatures.

Returns:

  • (Array) —

    of signatures.



193
194
195
196
197
 
# File 'lib/tomparse/parser.rb', line 193

def signatures
  parsed {
    @signatures 
  }
end

#smart_split(doc) ⇒ Array<String> (private)

Split the documentation up into proper sections. The method works by building up a list of linenos of where each section begins.

Returns:

  • (Array<String>) —

    Returns an array section strings.



372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
 
# File 'lib/tomparse/parser.rb', line 372

def smart_split(doc)
  splits = []
  index  = -1

  lines = doc.lines.to_a

  # Remove any blank lines off the top.
  lines.shift while lines.first && lines.first.strip.empty?

  # Keep a copy of the lines for later use.
  doc_lines = lines.dup
 

  # The first line may have a `Public`/`Private`/`Deprecated` marker.
  # So we just skip the first line.
  lines.shift
  index += 1

  # The description is always the first section, but it may have
  # multiple paragraphs. And the second section may be an arguments
  # list without a header. This loop handles that.
  while line = lines.shift
    index += 1
    if argument_line?(line)
      splits << index
      break
    elsif section_type(line)
      splits << index
      break
    end
  end
  
  # The rest of the the document should have identifiable section markers.
  while line = lines.shift
    index += 1
    if section_type(line)
      splits << index
    end
  end

  # Now we split the documentation up into sections using
  # the line indexes we collected above.
  sections = []
  b = 0
  splits.shift if splits.first == 0
  splits.each do |i|
    sections << doc_lines[b...i].join
    b = i
  end
  sections << doc_lines[b..-1].join

  return sections
end

#status ⇒ String

Method status, can be ‘Public`, `Internal` or `Deprecated`.

Returns:

  • (String) —

    Returns



220
221
222
223
224
 
# File 'lib/tomparse/parser.rb', line 220

def status
  parsed {
    @status
  }
end

#tags ⇒ Array<Array<String>>

List of tags.

Returns:

  • (Array<Array<String>>) —

    Returns an associatve array of tags.



211
212
213
214
215
 
# File 'lib/tomparse/parser.rb', line 211

def tags
  parsed {
    @tags
  }
end

#to_s ⇒ String

Raw documentation text.

Returns:

  • (String) —

    of raw documentation text.



36
37
38
 
# File 'lib/tomparse/parser.rb', line 36

def to_s
  @raw
end

#tomdoc ⇒ Object

The raw comment text cleaned-up and ready for section parsing.

Returns:

  • cleaned-up comment String.



80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
 
# File 'lib/tomparse/parser.rb', line 80

def tomdoc
  lines = raw.split("\n")

  # remove remark symbol
  if lines.all?{ |line| /^\s*#/ =~ line }   
    lines = lines.map do |line|
      line =~ /^(\s*#)/ ? line.sub($1, '') : nil
    end
  end

  # for some reason the first line is coming in without indention
  # regardless, so we temporary remove it
  first = lines.shift

  # remove indention
  spaces = lines.map do |line|
    next if line.strip.empty?
    md = /^(\s*)/.match(line)
    md ? md[1].size : nil
  end.compact

  space = spaces.min || 0
  lines = lines.map do |line|
    if line.strip.empty?
      line.strip
    else
      line[space..-1]
    end
  end

  # put first line back
  lines.unshift(first.sub(/^\s*/,'')) if first

  lines.compact.join("\n")
end

#valid? ⇒ Boolean

TODO:

This needs improvement.

Validate raw comment.

Returns:

  • (Boolean) —

    true if comment is valid, otherwise false.



52
53
54
55
56
57
58
59
 
# File 'lib/tomparse/parser.rb', line 52

def valid?
  begin
    new(text).validate
    true
  rescue ParseError
    false
  end
end

#validate ⇒ Object

Validate raw comment.

Returns:

  • true if comment is valid.

Raises:

  • ParseError if comment is not valid.



65
66
67
68
69
70
71
72
73
74
75
 
# File 'lib/tomparse/parser.rb', line 65

def validate
  if !raw.include?('Returns')
    raise ParseError.new("No `Returns' statement.")
  end

  if sections.size < 2
    raise ParseError.new("No description section found.")
  end

  true
end

#yields ⇒ String

Description of a methods yield procedure.

Returns:

  • (String) —

    decription of yield procedure.



166
167
168
169
170
 
# File 'lib/tomparse/parser.rb', line 166

def yields
  parsed {
    @yields
  }
end