Class: RDoc::Parser::Ruby

Inherits:
RDoc::Parser show all
Includes:
RubyTools, RubyToken, TokenStream
Defined in:
lib/rdoc/parser/ruby.rb

Overview

Extracts code elements from a source file returning a TopLevel object containing the constituent file elements.

This file is based on rtags

RubyParser understands how to document:

  • classes

  • modules

  • methods

  • constants

  • aliases

  • private, public, protected

  • private_class_function, public_class_function

  • module_function

  • attr, attr_reader, attr_writer, attr_accessor

  • extra accessors given on the command line

  • metaprogrammed methods

  • require

  • include

Method Arguments

– NOTE: I don’t think this works, needs tests, remove the paragraph following this block when known to work

The parser extracts the arguments from the method definition. You can override this with a custom argument definition using the :args: directive:

##
# This method tries over and over until it is tired

def go_go_go(thing_to_try, tries = 10) # :args: thing_to_try
  puts thing_to_try
  go_go_go thing_to_try, tries - 1
end

If you have a more-complex set of overrides you can use the :call-seq: directive: ++

The parser extracts the arguments from the method definition. You can override this with a custom argument definition using the :call-seq: directive:

##
# This method can be called with a range or an offset and length
#
# :call-seq:
#   my_method(Range)
#   my_method(offset, length)

def my_method(*args)
end

The parser extracts yield expressions from method bodies to gather the yielded argument names. If your method manually calls a block instead of yielding or you want to override the discovered argument names use the :yields: directive:

##
# My method is awesome

def my_method(&block) # :yields: happy, times
  block.call 1, 2
end

Metaprogrammed Methods

To pick up a metaprogrammed method, the parser looks for a comment starting with ‘##’ before an identifier:

##
# This is a meta-programmed method!

add_my_method :meta_method, :arg1, :arg2

The parser looks at the token after the identifier to determine the name, in this example, :meta_method. If a name cannot be found, a warning is printed and ‘unknown is used.

You can force the name of a method using the :method: directive:

##
# :method: some_method!

By default, meta-methods are instance methods. To indicate that a method is a singleton method instead use the :singleton-method: directive:

##
# :singleton-method:

You can also use the :singleton-method: directive with a name:

##
# :singleton-method: some_method!

You can define arguments for metaprogrammed methods via either the :call-seq:, :arg: or :args: directives.

Additionally you can mark a method as an attribute by using :attr:, :attr_reader:, :attr_writer: or :attr_accessor:. Just like for :method:, the name is optional.

##
# :attr_reader: my_attr_name

Hidden methods and attributes

You can provide documentation for methods that don’t appear using the :method:, :singleton-method: and :attr: directives:

##
# :attr_writer: ghost_writer
# There is an attribute here, but you can't see it!

##
# :method: ghost_method
# There is a method here, but you can't see it!

##
# this is a comment for a regular method

def regular_method() end

Note that by default, the :method: directive will be ignored if there is a standard rdocable item following it.

Constant Summary collapse

NORMAL =

RDoc::NormalClass type

"::"
SINGLE =

RDoc::SingleClass type

"<<"

Constants included from RubyToken

RubyToken::EXPR_ARG, RubyToken::EXPR_BEG, RubyToken::EXPR_CLASS, RubyToken::EXPR_DOT, RubyToken::EXPR_END, RubyToken::EXPR_FNAME, RubyToken::EXPR_MID, RubyToken::NEWLINE_TOKEN, RubyToken::Symbol, RubyToken::TkReading2Token, RubyToken::TkSymbol2Token, RubyToken::TkToken2Reading, RubyToken::TokenDefinitions

Instance Attribute Summary

Attributes inherited from RDoc::Parser

#file_name

Instance Method Summary collapse

Methods included from RubyTools

#add_token_listener, #get_tk, #get_tk_until, #get_tkread, #peek_read, #peek_tk, #remove_token_listener, #reset, #skip_tkspace, #token_listener, #unget_tk

Methods included from RubyToken

#Token, def_token, #set_token_position

Methods included from TokenStream

#add_tokens, #collect_tokens, #pop_token, to_html, #token_stream, #tokens_to_s

Methods inherited from RDoc::Parser

alias_extension, binary?, can_parse, can_parse_by_name, check_modeline, for, parse_files_matching, process_directive, remove_modeline, use_markup, zip?

Constructor Details

#initialize(top_level, file_name, content, options, stats) ⇒ Ruby

Creates a new Ruby parser.



162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
 
# File 'lib/rdoc/parser/ruby.rb', line 162

def initialize(top_level, file_name, content, options, stats)
  super

  @size = 0
  @token_listeners = nil
  @scanner = RDoc::RubyLex.new content, @options
  @scanner.exception_on_syntax_error = false
  @prev_seek = nil
  @markup = @options.markup
  @track_visibility = :nodoc != @options.visibility

  @encoding = nil
  @encoding = @options.encoding if Object.const_defined? :Encoding

  reset
end

Instance Method Details

#collect_first_comment ⇒ Object

Look for the first comment in a file that isn’t a shebang line.



224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
 
# File 'lib/rdoc/parser/ruby.rb', line 224

def collect_first_comment
  skip_tkspace
  comment = ''
  comment.force_encoding @encoding if @encoding
  first_line = true

  tk = get_tk

  while TkCOMMENT === tk
    if first_line and tk.text =~ /\A#!/ then
      skip_tkspace
      tk = get_tk
    elsif first_line and tk.text =~ /\A#\s*-\*-/ then
      first_line = false
      skip_tkspace
      tk = get_tk
    else
      first_line = false
      comment << tk.text << "\n"
      tk = get_tk

      if TkNL === tk then
        skip_tkspace false
        tk = get_tk
      end
    end
  end

  unget_tk tk

  new_comment comment
end

#consume_trailing_spaces ⇒ Object

Consumes trailing whitespace from the token stream



260
261
262
263
 
# File 'lib/rdoc/parser/ruby.rb', line 260

def consume_trailing_spaces # :nodoc:
  get_tkread
  skip_tkspace false
end

#create_attr(container, single, name, rw, comment) ⇒ Object

Creates a new attribute in container with name.



268
269
270
271
272
273
274
275
276
 
# File 'lib/rdoc/parser/ruby.rb', line 268

def create_attr container, single, name, rw, comment # :nodoc:
  att = RDoc::Attr.new get_tkread, name, rw, comment, single == SINGLE
  record_location att

  container.add_attribute att
  @stats.add_attribute att

  att
end

#create_module_alias(container, constant, rhs_name) ⇒ Object

Creates a module alias in container at rhs_name (or at the top-level for “::”) with the name from constant.



282
283
284
285
286
287
288
289
290
 
# File 'lib/rdoc/parser/ruby.rb', line 282

def create_module_alias container, constant, rhs_name # :nodoc:
  mod = if rhs_name =~ /^::/ then
          @store.find_class_or_module rhs_name
        else
          container.find_module_named rhs_name
        end

  container.add_module_alias mod, constant.name, @top_level if mod
end

#error(msg) ⇒ Object

Aborts with msg



295
296
297
298
299
 
# File 'lib/rdoc/parser/ruby.rb', line 295

def error(msg)
  msg = make_message msg

  abort msg
end

#get_bool ⇒ Object

Looks for a true or false token. Returns false if TkFALSE or TkNIL are found.



305
306
307
308
309
310
311
312
313
314
315
316
317
 
# File 'lib/rdoc/parser/ruby.rb', line 305

def get_bool
  skip_tkspace
  tk = get_tk
  case tk
  when TkTRUE
    true
  when TkFALSE, TkNIL
    false
  else
    unget_tk tk
    true
  end
end

#get_class_or_module(container, ignore_constants = false) ⇒ Object

Look for the name of a class of module (optionally with a leading

or

with

separated named) and return the ultimate name, the associated

container, and the given name (with the ::).



324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
 
# File 'lib/rdoc/parser/ruby.rb', line 324

def get_class_or_module container, ignore_constants = false
  skip_tkspace
  name_t = get_tk
  given_name = ''

  # class ::A -> A is in the top level
  case name_t
  when TkCOLON2, TkCOLON3 then # bug
    name_t = get_tk
    container = @top_level
    given_name << '::'
  end

  skip_tkspace false
  given_name << name_t.name

  while TkCOLON2 === peek_tk do
    prev_container = container
    container = container.find_module_named name_t.name
    container ||=
      if ignore_constants then
        RDoc::Context.new
      else
        c = prev_container.add_module RDoc::NormalModule, name_t.name
        c.ignore unless prev_container.document_children
        @top_level.add_to_classes_or_modules c
        c
      end

    record_location container

    get_tk
    skip_tkspace false
    name_t = get_tk
    given_name << '::' << name_t.name
  end

  skip_tkspace false

  return [container, name_t, given_name]
end

#get_class_specification ⇒ Object

Return a superclass, which can be either a constant of an expression



369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
 
# File 'lib/rdoc/parser/ruby.rb', line 369

def get_class_specification
  case peek_tk
  when TkSELF then return 'self'
  when TkGVAR then return ''
  end

  res = get_constant

  skip_tkspace false

  get_tkread # empty out read buffer

  tk = get_tk

  case tk
  when TkNL, TkCOMMENT, TkSEMICOLON then
    unget_tk(tk)
    return res
  end

  res += parse_call_parameters(tk)
  res
end

#get_constant ⇒ Object

Parse a constant, which might be qualified by one or more class or module names



397
398
399
400
401
402
403
404
405
406
407
408
409
 
# File 'lib/rdoc/parser/ruby.rb', line 397

def get_constant
  res = ""
  skip_tkspace false
  tk = get_tk

  while TkCOLON2 === tk or TkCOLON3 === tk or TkCONSTANT === tk do
    res += tk.name
    tk = get_tk
  end

  unget_tk(tk)
  res
end

#get_constant_with_optional_parens ⇒ Object

Get a constant that may be surrounded by parens



414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
 
# File 'lib/rdoc/parser/ruby.rb', line 414

def get_constant_with_optional_parens
  skip_tkspace false

  nest = 0

  while TkLPAREN === (tk = peek_tk) or TkfLPAREN === tk do
    get_tk
    skip_tkspace
    nest += 1
  end

  name = get_constant

  while nest > 0
    skip_tkspace
    tk = get_tk
    nest -= 1 if TkRPAREN === tk
  end

  name
end

#get_end_token(tk) ⇒ Object

Little hack going on here. In the statement:

f = 2*(1+yield)

We see the RPAREN as the next token, so we need to exit early. This still won’t catch all cases (such as “a = yield + 1”



444
445
446
447
448
449
450
451
452
453
 
# File 'lib/rdoc/parser/ruby.rb', line 444

def get_end_token tk # :nodoc:
  case tk
  when TkLPAREN, TkfLPAREN
    TkRPAREN
  when TkRPAREN
    nil
  else
    TkNL
  end
end

#get_method_container(container, name_t) ⇒ Object

Retrieves the method container for a singleton method.



458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
 
# File 'lib/rdoc/parser/ruby.rb', line 458

def get_method_container container, name_t # :nodoc:
  prev_container = container
  container = container.find_module_named(name_t.name)

  unless container then
    constant = prev_container.constants.find do |const|
      const.name == name_t.name
    end

    if constant then
      parse_method_dummy prev_container
      return
    end
  end

  unless container then
    # TODO seems broken, should starting at Object in @store
    obj = name_t.name.split("::").inject(Object) do |state, item|
      state.const_get(item)
    end rescue nil

    type = obj.class == Class ? RDoc::NormalClass : RDoc::NormalModule

    unless [Class, Module].include?(obj.class) then
      warn("Couldn't find #{name_t.name}. Assuming it's a module")
    end

    if type == RDoc::NormalClass then
      sclass = obj.superclass ? obj.superclass.name : nil
      container = prev_container.add_class type, name_t.name, sclass
    else
      container = prev_container.add_module type, name_t.name
    end

    record_location container
  end

  container
end

#get_symbol_or_name ⇒ Object

Extracts a name or symbol from the token stream.



501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
 
# File 'lib/rdoc/parser/ruby.rb', line 501

def get_symbol_or_name
  tk = get_tk
  case tk
  when TkSYMBOL then
    text = tk.text.sub(/^:/, '')

    if TkASSIGN === peek_tk then
      get_tk
      text << '='
    end

    text
  when TkId, TkOp then
    tk.name
  when TkAMPER,
       TkDSTRING,
       TkSTAR,
       TkSTRING then
    tk.text
  else
    raise RDoc::Error, "Name or symbol expected (got #{tk})"
  end
end

#get_tkread_clean(pattern, replacement) ⇒ Object

Retrieves the read token stream and replaces pattern with replacement using gsub. If the result is only a “;” returns an empty string.



183
184
185
186
187
 
# File 'lib/rdoc/parser/ruby.rb', line 183

def get_tkread_clean pattern, replacement # :nodoc:
  read = get_tkread.gsub(pattern, replacement).strip
  return '' if read == ';'
  read
end

#get_visibility_information(tk) ⇒ Object

Extracts the visibility information for the visibility token tk.

Returns the visibility type (a string), the visibility (a symbol) and singleton if the methods following should be converted to singleton methods.



196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
 
# File 'lib/rdoc/parser/ruby.rb', line 196

def get_visibility_information tk # :nodoc:
  vis_type  = tk.name
  singleton = false

  vis =
    case vis_type
    when 'private'   then :private
    when 'protected' then :protected
    when 'public'    then :public
    when 'private_class_method' then
      singleton = true
      :private
    when 'public_class_method' then
      singleton = true
      :public
    when 'module_function' then
      singleton = true
      :public
    else
      raise RDoc::Error, "Invalid visibility: #{tk.name}"
    end

  return vis_type, vis, singleton
end

#look_for_directives_in(context, comment) ⇒ Object

Look for directives in a normal comment block:

# :stopdoc:
# Don't display comment from this point forward

This routine modifies its comment parameter.



547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
 
# File 'lib/rdoc/parser/ruby.rb', line 547

def look_for_directives_in context, comment
  @preprocess.handle comment, context do |directive, param|
    case directive
    when 'method', 'singleton-method',
         'attr', 'attr_accessor', 'attr_reader', 'attr_writer' then
      false # handled elsewhere
    when 'section' then
      context.set_current_section param, comment.dup
      comment.text = ''
      break
    end
  end

  remove_private_comments comment
end

#make_message(message) ⇒ Object

Adds useful info about the parser to message



566
567
568
569
570
571
572
 
# File 'lib/rdoc/parser/ruby.rb', line 566

def make_message message
  prefix = "#{@file_name}:"

  prefix << "#{@scanner.line_no}:#{@scanner.char_no}:" if @scanner

  "#{prefix} #{message}"
end

#new_comment(comment) ⇒ Object

Creates a comment with the correct format



577
578
579
580
581
 
# File 'lib/rdoc/parser/ruby.rb', line 577

def new_comment comment
  c = RDoc::Comment.new comment, @top_level
  c.format = @markup
  c
end

#parse_alias(context, single, tk, comment) ⇒ Object

Parses an alias in context with comment



649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
 
# File 'lib/rdoc/parser/ruby.rb', line 649

def parse_alias(context, single, tk, comment)
  offset  = tk.seek
  line_no = tk.line_no

  skip_tkspace

  if TkLPAREN === peek_tk then
    get_tk
    skip_tkspace
  end

  new_name = get_symbol_or_name

  @scanner.lex_state = :EXPR_FNAME

  skip_tkspace
  if TkCOMMA === peek_tk then
    get_tk
    skip_tkspace
  end

  begin
    old_name = get_symbol_or_name
  rescue RDoc::Error
    return
  end

  al = RDoc::Alias.new(get_tkread, old_name, new_name, comment,
                       single == SINGLE)
  record_location al
  al.offset = offset
  al.line   = line_no

  read_documentation_modifiers al, RDoc::ATTR_MODIFIERS
  context.add_alias al
  @stats.add_alias al

  al
end

#parse_attr(context, single, tk, comment) ⇒ Object

Creates an RDoc::Attr for the name following tk, setting the comment to comment.



587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
 
# File 'lib/rdoc/parser/ruby.rb', line 587

def parse_attr(context, single, tk, comment)
  offset  = tk.seek
  line_no = tk.line_no

  args = parse_symbol_arg 1
  if args.size > 0 then
    name = args[0]
    rw = "R"
    skip_tkspace false
    tk = get_tk

    if TkCOMMA === tk then
      rw = "RW" if get_bool
    else
      unget_tk tk
    end

    att = create_attr context, single, name, rw, comment
    att.offset = offset
    att.line   = line_no

    read_documentation_modifiers att, RDoc::ATTR_MODIFIERS
  else
    warn "'attr' ignored - looks like a variable"
  end
end

#parse_attr_accessor(context, single, tk, comment) ⇒ Object

Creates an RDoc::Attr for each attribute listed after tk, setting the comment for each to comment.



618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
 
# File 'lib/rdoc/parser/ruby.rb', line 618

def parse_attr_accessor(context, single, tk, comment)
  offset  = tk.seek
  line_no = tk.line_no

  args = parse_symbol_arg
  rw = "?"

  tmp = RDoc::CodeObject.new
  read_documentation_modifiers tmp, RDoc::ATTR_MODIFIERS
  # TODO In most other places we let the context keep track of document_self
  # and add found items appropriately but here we do not.  I'm not sure why.
  return if @track_visibility and not tmp.document_self

  case tk.name
  when "attr_reader"   then rw = "R"
  when "attr_writer"   then rw = "W"
  when "attr_accessor" then rw = "RW"
  else
    rw = '?'
  end

  for name in args
    att = create_attr context, single, name, rw, comment
    att.offset = offset
    att.line   = line_no
  end
end

#parse_call_parameters(tk) ⇒ Object

Extracts call parameters from the token stream.



692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
 
# File 'lib/rdoc/parser/ruby.rb', line 692

def parse_call_parameters(tk)
  end_token = case tk
              when TkLPAREN, TkfLPAREN
                TkRPAREN
              when TkRPAREN
                return ""
              else
                TkNL
              end
  nest = 0

  loop do
    case tk
    when TkSEMICOLON
      break
    when TkLPAREN, TkfLPAREN
      nest += 1
    when end_token
      if end_token == TkRPAREN
        nest -= 1
        break if @scanner.lex_state == :EXPR_END and nest <= 0
      else
        break unless @scanner.continue
      end
    when TkCOMMENT, TkASSIGN, TkOPASGN
      unget_tk(tk)
      break
    when nil then
      break
    end
    tk = get_tk
  end

  get_tkread_clean "\n", " "
end

#parse_class(container, single, tk, comment) ⇒ Object

Parses a class in context with comment



731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
 
# File 'lib/rdoc/parser/ruby.rb', line 731

def parse_class container, single, tk, comment
  offset  = tk.seek
  line_no = tk.line_no

  declaration_context = container
  container, name_t, given_name = get_class_or_module container

  cls =
    case name_t
    when TkCONSTANT
      parse_class_regular container, declaration_context, single,
        name_t, given_name, comment
    when TkLSHFT
      case name = get_class_specification
      when 'self', container.name
        parse_statements container, SINGLE
        return # don't update offset or line
      else
        parse_class_singleton container, name, comment
      end
    else
      warn "Expected class name or '<<'. Got #{name_t.class}: #{name_t.text.inspect}"
      return
    end

  cls.offset = offset
  cls.line   = line_no

  cls
end

#parse_class_regular(container, declaration_context, single, name_t, given_name, comment) ⇒ Object

Parses and creates a regular class



765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
 
# File 'lib/rdoc/parser/ruby.rb', line 765

def parse_class_regular container, declaration_context, single, # :nodoc:
                        name_t, given_name, comment
  superclass = '::Object'

  if given_name =~ /^::/ then
    declaration_context = @top_level
    given_name = $'
  end

  if TkLT === peek_tk then
    get_tk
    skip_tkspace
    superclass = get_class_specification
    superclass = '(unknown)' if superclass.empty?
  end

  cls_type = single == SINGLE ? RDoc::SingleClass : RDoc::NormalClass
  cls = declaration_context.add_class cls_type, given_name, superclass
  cls.ignore unless container.document_children

  read_documentation_modifiers cls, RDoc::CLASS_MODIFIERS
  record_location cls

  cls.add_comment comment, @top_level

  @top_level.add_to_classes_or_modules cls
  @stats.add_class cls

  suppress_parents container, declaration_context unless cls.document_self

  parse_statements cls

  cls
end

#parse_class_singleton(container, name, comment) ⇒ Object

Parses a singleton class in container with the given name and comment.



804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
 
# File 'lib/rdoc/parser/ruby.rb', line 804

def parse_class_singleton container, name, comment # :nodoc:
  other = @store.find_class_named name

  unless other then
    if name =~ /^::/ then
      name = $'
      container = @top_level
    end

    other = container.add_module RDoc::NormalModule, name
    record_location other

    # class << $gvar
    other.ignore if name.empty?

    other.add_comment comment, @top_level
  end

  # notify :nodoc: all if not a constant-named class/module
  # (and remove any comment)
  unless name =~ /\A(::)?[A-Z]/ then
    other.document_self = nil
    other.document_children = false
    other.clear_comment
  end

  @top_level.add_to_classes_or_modules other
  @stats.add_class other

  read_documentation_modifiers other, RDoc::CLASS_MODIFIERS
  parse_statements(other, SINGLE)

  other
end

#parse_comment(container, tk, comment) ⇒ Object

Generates an RDoc::Method or RDoc::Attr from comment by looking for :method: or :attr: directives in comment.



949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
 
# File 'lib/rdoc/parser/ruby.rb', line 949

def parse_comment container, tk, comment
  return parse_comment_tomdoc container, tk, comment if @markup == 'tomdoc'
  column  = tk.char_no
  offset  = tk.seek
  line_no = tk.line_no

  text = comment.text

  singleton = !!text.sub!(/(^# +:?)(singleton-)(method:)/, '\1\3')

  co =
    if text.sub!(/^# +:?method: *(\S*).*?\n/i, '') then
      parse_comment_ghost container, text, $1, column, line_no, comment
    elsif text.sub!(/# +:?(attr(_reader|_writer|_accessor)?): *(\S*).*?\n/i, '') then
      parse_comment_attr container, $1, $3, comment
    end

  if co then
    co.singleton = singleton
    co.offset    = offset
    co.line      = line_no
  end

  true
end

#parse_comment_attr(container, type, name, comment) ⇒ Object

Parse a comment that is describing an attribute in container with the given name and comment.



979
980
981
982
983
984
985
986
987
988
989
 
# File 'lib/rdoc/parser/ruby.rb', line 979

def parse_comment_attr container, type, name, comment # :nodoc:
  return if name.empty?

  rw = case type
       when 'attr_reader' then 'R'
       when 'attr_writer' then 'W'
       else 'RW'
       end

  create_attr container, NORMAL, name, rw, comment
end

#parse_comment_ghost(container, text, name, column, line_no, comment) ⇒ Object

:nodoc:



991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
 
# File 'lib/rdoc/parser/ruby.rb', line 991

def parse_comment_ghost container, text, name, column, line_no, # :nodoc:
                        comment
  name = nil if name.empty?

  meth = RDoc::GhostMethod.new get_tkread, name
  record_location meth

  meth.start_collecting_tokens
  indent = TkSPACE.new 0, 1, 1
  indent.set_text " " * column

  position_comment = TkCOMMENT.new 0, line_no, 1
  position_comment.set_text "# File #{@top_level.relative_name}, line #{line_no}"
  meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]

  meth.params =
    if text.sub!(/^#\s+:?args?:\s*(.*?)\s*$/i, '') then
      $1
    else
      ''
    end

  comment.normalize
  comment.extract_call_seq meth

  return unless meth.name

  container.add_method meth

  meth.comment = comment

  @stats.add_method meth

  meth
end

#parse_comment_tomdoc(container, tk, comment) ⇒ Object

Creates an RDoc::Method on container from comment if there is a Signature section in the comment



1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
 
# File 'lib/rdoc/parser/ruby.rb', line 1031

def parse_comment_tomdoc container, tk, comment
  return unless signature = RDoc::TomDoc.signature(comment)
  offset  = tk.seek
  line_no = tk.line_no

  name, = signature.split %r%[ \(]%, 2

  meth = RDoc::GhostMethod.new get_tkread, name
  record_location meth
  meth.offset    = offset
  meth.line      = line_no

  meth.start_collecting_tokens
  indent = TkSPACE.new 0, 1, 1
  indent.set_text " " * offset

  position_comment = TkCOMMENT.new 0, line_no, 1
  position_comment.set_text "# File #{@top_level.relative_name}, line #{line_no}"
  meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]

  meth.call_seq = signature

  comment.normalize

  return unless meth.name

  container.add_method meth

  meth.comment = comment

  @stats.add_method meth
end

#parse_constant(container, tk, comment, ignore_constants = false) ⇒ Object

Parses a constant in context with comment. If ignore_constants is true, no found constants will be added to RDoc.



843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
 
# File 'lib/rdoc/parser/ruby.rb', line 843

def parse_constant container, tk, comment, ignore_constants = false
  prev_container = container
  offset  = tk.seek
  line_no = tk.line_no

  name = tk.name
  skip_tkspace false

  return unless name =~ /^\w+$/

  eq_tk = get_tk

  if TkCOLON2 === eq_tk then
    unget_tk eq_tk
    unget_tk tk

    container, name_t, = get_class_or_module container, ignore_constants

    name = name_t.name

    eq_tk = get_tk
  end

  unless TkASSIGN === eq_tk then
    suppress_parents container, prev_container

    unget_tk eq_tk
    return false
  end

  if TkGT === peek_tk then
    unget_tk eq_tk
    return
  end

  value = ''
  con = RDoc::Constant.new name, value, comment

  body = parse_constant_body container, con

  return unless body

  value.replace body
  record_location con
  con.offset = offset
  con.line   = line_no
  read_documentation_modifiers con, RDoc::CONSTANT_MODIFIERS

  @stats.add_constant con
  con = container.add_constant con

  true
end

#parse_constant_body(container, constant) ⇒ Object

:nodoc:



897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
 
# File 'lib/rdoc/parser/ruby.rb', line 897

def parse_constant_body container, constant # :nodoc:
  nest     = 0
  rhs_name = ''

  get_tkread

  tk = get_tk

  loop do
    case tk
    when TkSEMICOLON then
      break if nest <= 0
    when TkLPAREN, TkfLPAREN, TkLBRACE, TkfLBRACE, TkLBRACK, TkfLBRACK,
         TkDO, TkIF, TkUNLESS, TkCASE, TkDEF, TkBEGIN then
      nest += 1
    when TkRPAREN, TkRBRACE, TkRBRACK, TkEND then
      nest -= 1
    when TkCOMMENT then
      if nest <= 0 and stop_at_EXPR_END then
        unget_tk tk
        break
      else
        unget_tk tk
        read_documentation_modifiers constant, RDoc::CONSTANT_MODIFIERS
      end
    when TkCONSTANT then
      rhs_name << tk.name

      if nest <= 0 and TkNL === peek_tk then
        create_module_alias container, constant, rhs_name
        break
      end
    when TkNL then
      if nest <= 0 and stop_at_EXPR_END then
        unget_tk tk
        break
      end
    when TkCOLON2, TkCOLON3 then
      rhs_name << '::'
    when nil then
      break
    end
    tk = get_tk
  end

  get_tkread_clean(/^[ \t]+/, '')
end

#parse_extend_or_include(klass, container, comment) ⇒ Object

Parses an include or extend, indicated by the klass and adds it to container # with comment



1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
 
# File 'lib/rdoc/parser/ruby.rb', line 1068

def parse_extend_or_include klass, container, comment # :nodoc:
  loop do
    skip_tkspace_comment

    name = get_constant_with_optional_parens

    unless name.empty? then
      obj = container.add klass, name, comment
      record_location obj
    end

    return unless TkCOMMA === peek_tk

    get_tk
  end
end

#parse_identifier(container, single, tk, comment) ⇒ Object

Parses identifiers that can create new methods or change visibility.

Returns true if the comment was not consumed.



1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
 
# File 'lib/rdoc/parser/ruby.rb', line 1090

def parse_identifier container, single, tk, comment # :nodoc:
  case tk.name
  when 'private', 'protected', 'public', 'private_class_method',
       'public_class_method', 'module_function' then
    parse_visibility container, single, tk
    return true
  when 'attr' then
    parse_attr container, single, tk, comment
  when /^attr_(reader|writer|accessor)$/ then
    parse_attr_accessor container, single, tk, comment
  when 'alias_method' then
    parse_alias container, single, tk, comment
  when 'require', 'include' then
    # ignore
  else
    if comment.text =~ /\A#\#$/ then
      case comment.text
      when /^# +:?attr(_reader|_writer|_accessor)?:/ then
        parse_meta_attr container, single, tk, comment
      else
        method = parse_meta_method container, single, tk, comment
        method.params = container.params if
          container.params
        method.block_params = container.block_params if
          container.block_params
      end
    end
  end

  false
end

#parse_meta_attr(context, single, tk, comment) ⇒ Object

Parses a meta-programmed attribute and creates an RDoc::Attr.

To create foo and bar attributes on class C with comment “My attributes”:

class C

  ##
  # :attr:
  #
  # My attributes

  my_attr :foo, :bar

end

To create a foo attribute on class C with comment “My attribute”:

class C

  ##
  # :attr: foo
  #
  # My attribute

  my_attr :foo, :bar

end



1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
 
# File 'lib/rdoc/parser/ruby.rb', line 1151

def parse_meta_attr(context, single, tk, comment)
  args = parse_symbol_arg
  rw = "?"

  # If nodoc is given, don't document any of them

  tmp = RDoc::CodeObject.new
  read_documentation_modifiers tmp, RDoc::ATTR_MODIFIERS

  if comment.text.sub!(/^# +:?(attr(_reader|_writer|_accessor)?): *(\S*).*?\n/i, '') then
    rw = case $1
         when 'attr_reader' then 'R'
         when 'attr_writer' then 'W'
         else 'RW'
         end
    name = $3 unless $3.empty?
  end

  if name then
    att = create_attr context, single, name, rw, comment
  else
    args.each do |attr_name|
      att = create_attr context, single, attr_name, rw, comment
    end
  end

  att
end

#parse_meta_method(container, single, tk, comment) ⇒ Object

Parses a meta-programmed method



1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
 
# File 'lib/rdoc/parser/ruby.rb', line 1183

def parse_meta_method(container, single, tk, comment)
  column  = tk.char_no
  offset  = tk.seek
  line_no = tk.line_no

  start_collecting_tokens
  add_token tk
  add_token_listener self

  skip_tkspace false

  singleton = !!comment.text.sub!(/(^# +:?)(singleton-)(method:)/, '\1\3')

  name = parse_meta_method_name comment, tk

  return unless name

  meth = RDoc::MetaMethod.new get_tkread, name
  record_location meth
  meth.offset = offset
  meth.line   = line_no
  meth.singleton = singleton

  remove_token_listener self

  meth.start_collecting_tokens
  indent = TkSPACE.new 0, 1, 1
  indent.set_text " " * column

  position_comment = TkCOMMENT.new 0, line_no, 1
  position_comment.value = "# File #{@top_level.relative_name}, line #{line_no}"
  meth.add_tokens [position_comment, NEWLINE_TOKEN, indent]
  meth.add_tokens @token_stream

  parse_meta_method_params container, single, meth, tk, comment

  meth.comment = comment

  @stats.add_method meth

  meth
end

#parse_meta_method_name(comment, tk) ⇒ Object

Parses the name of a metaprogrammed method. comment is used to determine the name while tk is used in an error message if the name cannot be determined.



1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
 
# File 'lib/rdoc/parser/ruby.rb', line 1231

def parse_meta_method_name comment, tk # :nodoc:
  if comment.text.sub!(/^# +:?method: *(\S*).*?\n/i, '') then
    return $1 unless $1.empty?
  end

  name_t = get_tk

  case name_t
  when TkSYMBOL then
    name_t.text[1..-1]
  when TkSTRING then
    name_t.value[1..-2]
  when TkASSIGN then # ignore
    remove_token_listener self

    nil
  else
    warn "unknown name token #{name_t.inspect} for meta-method '#{tk.name}'"
    'unknown'
  end
end

#parse_meta_method_params(container, single, meth, tk, comment) ⇒ Object

Parses the parameters and block for a meta-programmed method.



1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
 
# File 'lib/rdoc/parser/ruby.rb', line 1256

def parse_meta_method_params container, single, meth, tk, comment # :nodoc:
  token_listener meth do
    meth.params = ''

    comment.normalize
    comment.extract_call_seq meth

    container.add_method meth

    last_tk = tk

    while tk = get_tk do
      case tk
      when TkSEMICOLON then
        break
      when TkNL then
        break unless last_tk and TkCOMMA === last_tk
      when TkSPACE then
        # expression continues
      when TkDO then
        parse_statements container, single, meth
        break
      else
        last_tk = tk
      end
    end
  end
end

#parse_method(container, single, tk, comment) ⇒ Object

Parses a normal method defined by def



1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
 
# File 'lib/rdoc/parser/ruby.rb', line 1288

def parse_method(container, single, tk, comment)
  singleton = nil
  added_container = false
  name = nil
  column  = tk.char_no
  offset  = tk.seek
  line_no = tk.line_no

  start_collecting_tokens
  add_token tk

  token_listener self do
    prev_container = container
    name, container, singleton = parse_method_name container
    added_container = container != prev_container
  end

  return unless name

  meth = RDoc::AnyMethod.new get_tkread, name
  meth.singleton = singleton

  record_location meth
  meth.offset = offset
  meth.line   = line_no

  meth.start_collecting_tokens
  indent = TkSPACE.new 0, 1, 1
  indent.set_text " " * column

  token = TkCOMMENT.new 0, line_no, 1
  token.set_text "# File #{@top_level.relative_name}, line #{line_no}"
  meth.add_tokens [token, NEWLINE_TOKEN, indent]
  meth.add_tokens @token_stream

  parse_method_params_and_body container, single, meth, added_container

  comment.normalize
  comment.extract_call_seq meth

  meth.comment = comment

  @stats.add_method meth
end

#parse_method_dummy(container) ⇒ Object

Parses a method that needs to be ignored.



1367
1368
1369
1370
1371
1372
 
# File 'lib/rdoc/parser/ruby.rb', line 1367

def parse_method_dummy container
  dummy = RDoc::Context.new
  dummy.parent = container
  dummy.store  = container.store
  skip_method dummy
end

#parse_method_name(container) ⇒ Object

Parses the name of a method in container.

Returns the method name, the container it is in (for def Foo.name) and if it is a singleton or regular method.



1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
 
# File 'lib/rdoc/parser/ruby.rb', line 1380

def parse_method_name container # :nodoc:
  @scanner.lex_state = :EXPR_FNAME

  skip_tkspace
  name_t = get_tk
  back_tk = skip_tkspace
  singleton = false

  case dot = get_tk
  when TkDOT, TkCOLON2 then
    singleton = true

    name, container = parse_method_name_singleton container, name_t
  else
    unget_tk dot
    back_tk.reverse_each do |token|
      unget_tk token
    end

    name = parse_method_name_regular container, name_t
  end

  return name, container, singleton
end

#parse_method_name_regular(container, name_t) ⇒ Object

For the given container and initial name token name_t the method name is parsed from the token stream for a regular method.



1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
 
# File 'lib/rdoc/parser/ruby.rb', line 1409

def parse_method_name_regular container, name_t # :nodoc:
  case name_t
  when TkSTAR, TkAMPER then
    name_t.text
  else
    unless name_t.respond_to? :name then
      warn "expected method name token, . or ::, got #{name_t.inspect}"
      skip_method container
      return
    end
    name_t.name
  end
end

#parse_method_name_singleton(container, name_t) ⇒ Object

For the given container and initial name token name_t the method name and the new container (if necessary) are parsed from the token stream for a singleton method.



1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
 
# File 'lib/rdoc/parser/ruby.rb', line 1428

def parse_method_name_singleton container, name_t # :nodoc:
  @scanner.lex_state = :EXPR_FNAME
  skip_tkspace
  name_t2 = get_tk

  name =
    case name_t
    when TkSELF, TkMOD then
      case name_t2
        # NOTE: work around '[' being consumed early and not being re-tokenized
        # as a TkAREF
      when TkfLBRACK then
        get_tk
        '[]'
      else
        name_t2.name
      end
    when TkCONSTANT then
      name = name_t2.name

      container = get_method_container container, name_t

      return unless container

      name
    when TkIDENTIFIER, TkIVAR, TkGVAR then
      parse_method_dummy container

      nil
    when TkTRUE, TkFALSE, TkNIL then
      klass_name = "#{name_t.name.capitalize}Class"
      container = @store.find_class_named klass_name
      container ||= @top_level.add_class RDoc::NormalClass, klass_name

      name_t2.name
    else
      warn "unexpected method name token #{name_t.inspect}"
      # break
      skip_method container

      nil
    end

  return name, container
end

#parse_method_or_yield_parameters(method = nil, modifiers = RDoc::METHOD_MODIFIERS) ⇒ Object

Extracts yield parameters from method



1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
 
# File 'lib/rdoc/parser/ruby.rb', line 1477

def parse_method_or_yield_parameters(method = nil,
                                     modifiers = RDoc::METHOD_MODIFIERS)
  skip_tkspace false
  tk = get_tk
  end_token = get_end_token tk
  return '' unless end_token

  nest = 0

  loop do
    case tk
    when TkSEMICOLON then
      break if nest == 0
    when TkLBRACE, TkfLBRACE then
      nest += 1
    when TkRBRACE then
      nest -= 1
      if nest <= 0
        # we might have a.each { |i| yield i }
        unget_tk(tk) if nest < 0
        break
      end
    when TkLPAREN, TkfLPAREN then
      nest += 1
    when end_token then
      if end_token == TkRPAREN
        nest -= 1
        break if nest <= 0
      else
        break unless @scanner.continue
      end
    when TkRPAREN then
      nest -= 1
    when method && method.block_params.nil? && TkCOMMENT then
      unget_tk tk
      read_documentation_modifiers method, modifiers
      @read.pop
    when TkCOMMENT then
      @read.pop
    when nil then
      break
    end
    tk = get_tk
  end

  get_tkread_clean(/\s+/, ' ')
end

#parse_method_parameters(method) ⇒ Object

Capture the method’s parameters. Along the way, look for a comment containing:

# yields: ....

and add this as the block_params for the method



1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
 
# File 'lib/rdoc/parser/ruby.rb', line 1533

def parse_method_parameters method
  res = parse_method_or_yield_parameters method

  res = "(#{res})" unless res =~ /\A\(/
  method.params = res unless method.params

  return if  method.block_params

  skip_tkspace false
  read_documentation_modifiers method, RDoc::METHOD_MODIFIERS
end

#parse_method_params_and_body(container, single, meth, added_container) ⇒ Object

Parses the parameters and body of meth



1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
 
# File 'lib/rdoc/parser/ruby.rb', line 1336

def parse_method_params_and_body container, single, meth, added_container
  token_listener meth do
    @scanner.continue = false
    parse_method_parameters meth

    if meth.document_self or not @track_visibility then
      container.add_method meth
    elsif added_container then
      container.document_self = false
    end

    # Having now read the method parameters and documentation modifiers, we
    # now know whether we have to rename #initialize to ::new

    if meth.name == "initialize" && !meth.singleton then
      if meth.dont_rename_initialize then
        meth.visibility = :protected
      else
        meth.singleton = true
        meth.name = "new"
        meth.visibility = :public
      end
    end

    parse_statements container, single, meth
  end
end

#parse_module(container, single, tk, comment) ⇒ Object

Parses an RDoc::NormalModule in container with comment



1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
 
# File 'lib/rdoc/parser/ruby.rb', line 1548

def parse_module container, single, tk, comment
  container, name_t, = get_class_or_module container

  name = name_t.name

  mod = container.add_module RDoc::NormalModule, name
  mod.ignore unless container.document_children
  record_location mod

  read_documentation_modifiers mod, RDoc::CLASS_MODIFIERS
  mod.add_comment comment, @top_level
  parse_statements mod

  @stats.add_module mod
end

#parse_require(context, comment) ⇒ Object

Parses an RDoc::Require in context containing comment



1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
 
# File 'lib/rdoc/parser/ruby.rb', line 1567

def parse_require(context, comment)
  skip_tkspace_comment
  tk = get_tk

  if TkLPAREN === tk then
    skip_tkspace_comment
    tk = get_tk
  end

  name = tk.text if TkSTRING === tk

  if name then
    @top_level.add_require RDoc::Require.new(name, comment)
  else
    unget_tk tk
  end
end

#parse_rescue ⇒ Object

Parses a rescue



1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
 
# File 'lib/rdoc/parser/ruby.rb', line 1588

def parse_rescue
  skip_tkspace false

  while tk = get_tk
    case tk
    when TkNL, TkSEMICOLON then
      break
    when TkCOMMA then
      skip_tkspace false

      get_tk if TkNL === peek_tk
    end

    skip_tkspace false
  end
end

#parse_statements(container, single = NORMAL, current_method = nil, comment = new_comment('')) ⇒ Object

The core of the Ruby parser.



1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
 
# File 'lib/rdoc/parser/ruby.rb', line 1608

def parse_statements(container, single = NORMAL, current_method = nil,
                     comment = new_comment(''))
  raise 'no' unless RDoc::Comment === comment
  comment.force_encoding @encoding if @encoding

  nest = 1
  save_visibility = container.visibility

  non_comment_seen = true

  while tk = get_tk do
    keep_comment = false
    try_parse_comment = false

    non_comment_seen = true unless TkCOMMENT === tk

    case tk
    when TkNL then
      skip_tkspace
      tk = get_tk

      if TkCOMMENT === tk then
        if non_comment_seen then
          # Look for RDoc in a comment about to be thrown away
          non_comment_seen = parse_comment container, tk, comment unless
            comment.empty?

          comment = ''
          comment.force_encoding @encoding if @encoding
        end

        while TkCOMMENT === tk do
          comment << tk.text << "\n"

          tk = get_tk

          if TkNL === tk then
            skip_tkspace false # leading spaces
            tk = get_tk
          end
        end

        comment = new_comment comment

        unless comment.empty? then
          look_for_directives_in container, comment

          if container.done_documenting then
            throw :eof if RDoc::TopLevel === container
            container.ongoing_visibility = save_visibility
          end
        end

        keep_comment = true
      else
        non_comment_seen = true
      end

      unget_tk tk
      keep_comment = true

    when TkCLASS then
      parse_class container, single, tk, comment

    when TkMODULE then
      parse_module container, single, tk, comment

    when TkDEF then
      parse_method container, single, tk, comment

    when TkCONSTANT then
      unless parse_constant container, tk, comment, current_method then
        try_parse_comment = true
      end

    when TkALIAS then
      parse_alias container, single, tk, comment unless current_method

    when TkYIELD then
      if current_method.nil? then
        warn "Warning: yield outside of method" if container.document_self
      else
        parse_yield container, single, tk, current_method
      end

    # Until and While can have a 'do', which shouldn't increase the nesting.
    # We can't solve the general case, but we can handle most occurrences by
    # ignoring a do at the end of a line.

    when TkUNTIL, TkWHILE then
      nest += 1
      skip_optional_do_after_expression

    # 'for' is trickier
    when TkFOR then
      nest += 1
      skip_for_variable
      skip_optional_do_after_expression

    when TkCASE, TkDO, TkIF, TkUNLESS, TkBEGIN then
      nest += 1

    when TkSUPER then
      current_method.calls_super = true if current_method

    when TkRESCUE then
      parse_rescue

    when TkIDENTIFIER then
      if nest == 1 and current_method.nil? then
        keep_comment = parse_identifier container, single, tk, comment
      end

      case tk.name
      when "require" then
        parse_require container, comment
      when "include" then
        parse_extend_or_include RDoc::Include, container, comment
      when "extend" then
        parse_extend_or_include RDoc::Extend, container, comment
      end

    when TkEND then
      nest -= 1
      if nest == 0 then
        read_documentation_modifiers container, RDoc::CLASS_MODIFIERS
        container.ongoing_visibility = save_visibility

        parse_comment container, tk, comment unless comment.empty?

        return
      end
    else
      try_parse_comment = nest == 1
    end

    if try_parse_comment then
      non_comment_seen = parse_comment container, tk, comment unless
        comment.empty?

      keep_comment = false
    end

    unless keep_comment then
      comment = new_comment ''
      comment.force_encoding @encoding if @encoding
      container.params = nil
      container.block_params = nil
    end

    consume_trailing_spaces
  end

  container.params = nil
  container.block_params = nil
end

#parse_symbol_arg(no = nil) ⇒ Object

Parse up to no symbol arguments



1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
 
# File 'lib/rdoc/parser/ruby.rb', line 1768

def parse_symbol_arg(no = nil)
  skip_tkspace_comment

  case tk = get_tk
  when TkLPAREN
    parse_symbol_arg_paren no
  else
    parse_symbol_arg_space no, tk
  end
end

#parse_symbol_arg_paren(no) ⇒ Object

Parses up to no symbol arguments surrounded by () and places them in args.



1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
 
# File 'lib/rdoc/parser/ruby.rb', line 1783

def parse_symbol_arg_paren no # :nodoc:
  args = []

  loop do
    skip_tkspace_comment
    if tk1 = parse_symbol_in_arg
      args.push tk1
      break if no and args.size >= no
    end

    skip_tkspace_comment
    case tk2 = get_tk
    when TkRPAREN
      break
    when TkCOMMA
    else
      warn("unexpected token: '#{tk2.inspect}'") if $DEBUG_RDOC
      break
    end
  end

  args
end

#parse_symbol_arg_space(no, tk) ⇒ Object

Parses up to no symbol arguments separated by spaces and places them in args.



1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
 
# File 'lib/rdoc/parser/ruby.rb', line 1811

def parse_symbol_arg_space no, tk # :nodoc:
  args = []

  unget_tk tk
  if tk = parse_symbol_in_arg
    args.push tk
    return args if no and args.size >= no
  end

  loop do
    skip_tkspace false

    tk1 = get_tk
    unless TkCOMMA === tk1 then
      unget_tk tk1
      break
    end

    skip_tkspace_comment
    if tk = parse_symbol_in_arg
      args.push tk
      break if no and args.size >= no
    end
  end

  args
end

#parse_symbol_in_arg ⇒ Object

Returns symbol text from the next token



1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
 
# File 'lib/rdoc/parser/ruby.rb', line 1842

def parse_symbol_in_arg
  case tk = get_tk
  when TkSYMBOL
    tk.text.sub(/^:/, '')
  when TkSTRING
    eval @read[-1]
  when TkDSTRING, TkIDENTIFIER then
    nil # ignore
  else
    warn("Expected symbol or string, got #{tk.inspect}") if $DEBUG_RDOC
    nil
  end
end

#parse_top_level_statements(container) ⇒ Object

Parses statements in the top-level container



1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
 
# File 'lib/rdoc/parser/ruby.rb', line 1859

def parse_top_level_statements container
  comment = collect_first_comment

  look_for_directives_in container, comment

  throw :eof if container.done_documenting

  @markup = comment.format

  # HACK move if to RDoc::Context#comment=
  container.comment = comment if container.document_self unless comment.empty?

  parse_statements container, NORMAL, nil, comment
end

#parse_visibility(container, single, tk) ⇒ Object

Determines the visibility in container from tk



1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
 
# File 'lib/rdoc/parser/ruby.rb', line 1877

def parse_visibility(container, single, tk)
  singleton = (single == SINGLE)

  vis_type, vis, singleton = get_visibility_information tk

  skip_tkspace_comment false

  case peek_tk
    # Ryan Davis suggested the extension to ignore modifiers, because he
    # often writes
    #
    #   protected unless $TESTING
    #
  when TkNL, TkUNLESS_MOD, TkIF_MOD, TkSEMICOLON then
    container.ongoing_visibility = vis
  else
    update_visibility container, vis_type, vis, singleton
  end
end

#parse_yield(context, single, tk, method) ⇒ Object

Determines the block parameter for context



1900
1901
1902
1903
1904
1905
1906
 
# File 'lib/rdoc/parser/ruby.rb', line 1900

def parse_yield(context, single, tk, method)
  return if method.block_params

  get_tkread
  @scanner.continue = false
  method.block_params = parse_method_or_yield_parameters
end

#read_directive(allowed) ⇒ Object

Directives are modifier comments that can appear after class, module, or method names. For example:

def fred # :yields: a, b

or:

class MyClass # :nodoc:

We return the directive name and any parameters as a two element array if the name is in allowed. A directive can be found anywhere up to the end of the current line.



1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
 
# File 'lib/rdoc/parser/ruby.rb', line 1922

def read_directive allowed
  tokens = []

  while tk = get_tk do
    tokens << tk

    case tk
    when TkNL, TkDEF then
      return
    when TkCOMMENT then
      return unless tk.text =~ /\s*:?([\w-]+):\s*(.*)/

      directive = $1.downcase

      return [directive, $2] if allowed.include? directive

      return
    end
  end
ensure
  unless tokens.length == 1 and TkCOMMENT === tokens.first then
    tokens.reverse_each do |token|
      unget_tk token
    end
  end
end

#read_documentation_modifiers(context, allowed) ⇒ Object

Handles directives following the definition for context (any RDoc::CodeObject) if the directives are allowed at this point.

See also RDoc::Markup::PreProcess#handle_directive



1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
 
# File 'lib/rdoc/parser/ruby.rb', line 1955

def read_documentation_modifiers context, allowed
  directive, value = read_directive allowed

  return unless directive

  @preprocess.handle_directive '', directive, value, context do |dir, param|
    if %w[notnew not_new not-new].include? dir then
      context.dont_rename_initialize = true

      true
    end
  end
end

#record_location(container) ⇒ Object

Records the location of this container in the file for this parser and adds it to the list of classes and modules in the file.



1973
1974
1975
1976
1977
1978
1979
1980
 
# File 'lib/rdoc/parser/ruby.rb', line 1973

def record_location container # :nodoc:
  case container
  when RDoc::ClassModule then
    @top_level.add_to_classes_or_modules container
  end

  container.record_location @top_level
end

#remove_private_comments(comment) ⇒ Object

Removes private comments from comment – TODO remove



1987
1988
1989
 
# File 'lib/rdoc/parser/ruby.rb', line 1987

def remove_private_comments comment
  comment.remove_private
end

#scan ⇒ Object

Scans this Ruby file for Ruby constructs



1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
 
# File 'lib/rdoc/parser/ruby.rb', line 1994

def scan
  reset

  catch :eof do
    begin
      parse_top_level_statements @top_level

    rescue StandardError => e
      bytes = ''

      20.times do @scanner.ungetc end
      count = 0
      60.times do |i|
        count = i
        byte = @scanner.getc
        break unless byte
        bytes << byte
      end
      count -= 20
      count.times do @scanner.ungetc end

      $stderr.puts <<-EOF

#{self.class} failure around line #{@scanner.line_no} of
#{@file_name}

      EOF

      unless bytes.empty? then
        $stderr.puts
        $stderr.puts bytes.inspect
      end

      raise e
    end
  end

  @top_level
end

#skip_for_variable ⇒ Object

skip the var [in] part of a ‘for’ statement



2079
2080
2081
2082
2083
2084
2085
 
# File 'lib/rdoc/parser/ruby.rb', line 2079

def skip_for_variable
  skip_tkspace false
  tk = get_tk
  skip_tkspace false
  tk = get_tk
  unget_tk(tk) unless TkIN === tk
end

#skip_method(container) ⇒ Object

Skips the next method in container



2090
2091
2092
2093
2094
 
# File 'lib/rdoc/parser/ruby.rb', line 2090

def skip_method container
  meth = RDoc::AnyMethod.new "", "anon"
  parse_method_parameters meth
  parse_statements container, false, meth
end

#skip_optional_do_after_expression ⇒ Object

while, until, and for have an optional do



2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
 
# File 'lib/rdoc/parser/ruby.rb', line 2037

def skip_optional_do_after_expression
  skip_tkspace false
  tk = get_tk
  end_token = get_end_token tk

  b_nest = 0
  nest = 0
  @scanner.continue = false

  loop do
    case tk
    when TkSEMICOLON then
      break if b_nest.zero?
    when TkLPAREN, TkfLPAREN then
      nest += 1
    when TkBEGIN then
      b_nest += 1
    when TkEND then
      b_nest -= 1
    when TkDO
      break if nest.zero?
    when end_token then
      if end_token == TkRPAREN
        nest -= 1
        break if @scanner.lex_state == :EXPR_END and nest.zero?
      else
        break unless @scanner.continue
      end
    when nil then
      break
    end
    tk = get_tk
  end

  skip_tkspace false

  get_tk if TkDO === peek_tk
end

#skip_tkspace_comment(skip_nl = true) ⇒ Object

Skip spaces until a comment is found



2099
2100
2101
2102
2103
2104
2105
 
# File 'lib/rdoc/parser/ruby.rb', line 2099

def skip_tkspace_comment(skip_nl = true)
  loop do
    skip_tkspace skip_nl
    return unless TkCOMMENT === peek_tk
    get_tk
  end
end

#stop_at_EXPR_END ⇒ Object

:nodoc:



525
526
527
 
# File 'lib/rdoc/parser/ruby.rb', line 525

def stop_at_EXPR_END # :nodoc:
  @scanner.lex_state == :EXPR_END || !@scanner.continue
end

#suppress_parents(container, ancestor) ⇒ Object

Marks containers between container and ancestor as ignored



532
533
534
535
536
537
 
# File 'lib/rdoc/parser/ruby.rb', line 532

def suppress_parents container, ancestor # :nodoc:
  while container and container != ancestor do
    container.suppress unless container.documented?
    container = container.parent
  end
end

#update_visibility(container, vis_type, vis, singleton) ⇒ Object

Updates visibility in container from vis_type and vis.



2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
 
# File 'lib/rdoc/parser/ruby.rb', line 2110

def update_visibility container, vis_type, vis, singleton # :nodoc:
  new_methods = []

  case vis_type
  when 'module_function' then
    args = parse_symbol_arg
    container.set_visibility_for args, :private, false

    container.methods_matching args do |m|
      s_m = m.dup
      record_location s_m
      s_m.singleton = true
      new_methods << s_m
    end
  when 'public_class_method', 'private_class_method' then
    args = parse_symbol_arg

    container.methods_matching args, true do |m|
      if m.parent != container then
        m = m.dup
        record_location m
        new_methods << m
      end

      m.visibility = vis
    end
  else
    args = parse_symbol_arg
    container.set_visibility_for args, vis, singleton
  end

  new_methods.each do |method|
    case method
    when RDoc::AnyMethod then
      container.add_method method
    when RDoc::Attr then
      container.add_attribute method
    end
    method.visibility = vis
  end
end

#warn(message) ⇒ Object

Prints message to $stderr unless we’re being quiet



2155
2156
2157
 
# File 'lib/rdoc/parser/ruby.rb', line 2155

def warn message
  @options.warn make_message message
end