Method: YARD::Parser::SourceParser#tokenize

Defined in:
lib/yard/parser/source_parser.rb

#tokenize(content) ⇒ Array

Tokenizes but does not parse the block of code using the current #parser_type

Parameters:

  • content (String) —

    the block of code to tokenize

Returns:

  • (Array) —

    a list of tokens


460
461
462
463
 
# File 'lib/yard/parser/source_parser.rb', line 460

def tokenize(content)
  @parser = parser_class.new(content, file)
  @parser.tokenize
end