Module: Token::Handler

Defined in:
lib/token/handler.rb

Overview

Note:

This module is not built to be used as a mixin! It should be seen as a **global singleton** instead.

The Handler is meant to be the global store for all Token-handlers. Token-handlers are needed to process all tokenlines associated with each comment. The following sample, shows a comment including a token line:

/**
 * @foo this is a default tokenline
 */

This comment will be transformed by the Parser::CommentParser into a tokenline like the following:

puts my_tokenline
#=> <struct Parser::Tokenline token="foo", content="this is a default tokenline">

After creating the right type of CodeObjects (either an Object, Function or a custom type) the converter will trigger the conversion to an appropriate subclass of Token by calling #process_token on the CodeObject.

code_object.process_token(my_tokenline)
code_object.token(:token)
#=> [#<Token::FooToken content="this is a default tokenline">]

Tokens are always stored in an array to make multiple usage in one comment possible. For example one function-comment can contain many ‘@param`s:

/**
 * @function foo
 * @param [Number] bar
 * @param [String] baz
 */

Default Handlers

:text_only

The default Header :text_only cannot parse typelists or tokennames, it only saves the content of the token to an instance of Token Being the default handler we can use it without explicitly specifying it:

Token::Handler.register :token

This Default Handler is enough for tokens like ‘@todo` or `@note`. But for more complex Tokens we need some other handlers.

:typed

Typed tokens look like 

@return [Foo, Bar] This is the description

Additional to their default content they specify the possible types as a commaseperated list.

To register a typed-token, you only need to add the ‘:handler` option

Token::Handler.register :return, :handler => :typed

This line implicitly generates a class ‘Token::ReturnToken` which extends Token, content and types will be filled to access them later:

my_return_token.content #=> "This is the description"
my_return_token.types   #=> ["Foo", "Bar]

:named

Named tokenlines like

@mixin Foo.bar The description of this mixin-usage

interpret the first part (‘Foo.bar`) as name and the rest as content

Token::Handler.register :mixin, :handler => :named

mixin_token.name    #=> "Foo.bar"
mixin_token.content #=> "The description of this mixin-usage"

:named_multiline

Named multiline are similiar to named tokenlines. But instead of taking the first word as name they are using the first line:

@example this is the name
  function() {...}

To register a named-multiline token just add the handler like:

Token::Handler.register :example, :handler => :named_multiline

my_example.name    #=> "this is the name"
my_example.content #=> "function() {...}"

:typed_with_name

The typed_with_name handler is much like the Typed-Token-Handler. It is neccessary for tokenlines, like

@param [String] my_param This is a param

Additional to content and types the generated class will contain a name property.

Token::Handler.register :param, :handler => :typed_with_name   

param_token.name     #=> "my_param"
param_token.types    #=> ["String"]
param_token.content  #=> "This is a param"
param_token.class    #=> Token::ParamToken

:named_nested_shorthand

named_nested_shorthand can be used to parse nested ‘:typed_with_name` tokens.

@param person
  [String] name the name
  [Number] age the age of the person

It is called shorthand, because “‘[String] name the name`” is not a full tokenline. (The `@param` is missing.)

Token::Handler.register :param, :handler => :named_nested_shorthand

The instance of ‘Token::ParamToken` can look like:

param_token.name     #=> "person"
param_token.children # [<Token::ParamToken name="name"><Token::ParamToken name="age">]

It also can parse typed_with_name tokenlines like ‘@param [String] name`. In this case it behaves exaclty like :typed_with_name

:noop

Can be used, if you don’t want to do anything with that token

See Also:

Constant Summary collapse

ALL = 
/./m
NO_BR = 
/((?!\n)\s)/
IDENTIFIER = 
/(?:[^\s])*/
TYPELIST = 
/\[(?<types>#{IDENTIFIER}(?:,#{NO_BR}*#{IDENTIFIER})*)\]/
NAME =

Tokens with name and content

/#{NO_BR}*(?<name>#{IDENTIFIER})#{NO_BR}*(?<content>#{ALL}*)/
TOKEN_W_TYPE =

Token with type and content

/#{NO_BR}*#{TYPELIST}#{NO_BR}*(?<content>#{ALL}*)/
TOKEN_W_TYPE_NAME =

Token with type, name and content

/^#{NO_BR}*
  #{TYPELIST}#{NO_BR}*
  (?<name>#{IDENTIFIER})
  #{NO_BR}*
  (?<content>#{ALL}*)
/x
@@defaults = 
{
   :text_only => ->(tokenklass, content) {
    tokenklass.new(:content => content)
  },
  
  :typed => ->(tokenklass, content) {
    typestring, content = TOKEN_W_TYPE.match(content).captures
    types = typestring.split /,\s*/
    
    tokenklass.new(:types => types, :content => content)
  },
  
  :named => ->(tokenklass, content) {
    name, content = NAME.match(content).captures        
    tokenklass.new(:name => name, :content => content)
  },
   
  :named_multiline => ->(tokenklass, content) { 
    rows = content.split(/\n/)
          
    # use first row as name
    name = rows.shift.strip
    content = rows.join("\n")
          
    tokenklass.new(:name => name, :content => content)
  },
   :typed_with_name => ->(tokenklass, content) {
    typestring, name, content = TOKEN_W_TYPE_NAME.match(content).captures
    types = typestring.split /,\s*/
    
    tokenklass.new(:name => name, :types => types, :content => content)
  },
  
  :named_nested_shorthand => ->(tokenklass, content) {
      
    # First remove linebreaks with 2-times intendation (= continuation)    
    lines         = content.gsub(/\n((?!\n)\s){2}/, ' ').split(/\n/)
    name          = lines.shift.strip
    documentation = []
    children      = []
    
    lines.each do |line|          
      if TOKEN_W_TYPE_NAME.match(line)
        # apply handler :typed_with_name to each child-line
        # @todo maybe we need a special way to select Children's Class?
        children << Handler.apply(:typed_with_name, tokenklass, line)
      else
        documentation << line
      end
    end
  
    tokenklass.new(:name => name, :types => [], :children => children, :content => documentation.join("\n"))   
  },
  
  :noop => ->(tokenklass, content) {}
}
@@handlers = 
{}
@@types =

Hash for all registered CodeObject-Types. For example: 

{ :function => CodeObject::Function }

{}

Class Method Summary collapse

Class Method Details

.add_default_handler(name, &block) ⇒ Object 



264
265
266
 
# File 'lib/token/handler.rb', line 264

def self.add_default_handler(name, &block)
  @@defaults[name] = block;
end

.apply(default_handler, *args) ⇒ Object

Use a default handler



260
261
262
 
# File 'lib/token/handler.rb', line 260

def self.apply(default_handler, *args)
  @@defaults[default_handler].call(*args)
end

.handlers ⇒ Hash<Symbol, Block>

Attribute-Reader for all registered ‘@@handlers`

Returns:

  • (Hash<Symbol, Block>) 


249
250
251
 
# File 'lib/token/handler.rb', line 249

def self.handlers
  @@handlers
end

.register(tokenname, options = {}) {|tokenklass, stringcontent| ... } ⇒ Object

It is possible to register your own Tokenhandlers and therefore extend the capabilities of this documentation-tool (See: CUSTOMIZE).

There are different types of handlers which can be used out of the box. Further documentation about the default handlers can be found in the introduction.

Writing your own custom Token-Handler

By adding the optional block you easily can build your own Tokenhandler:

Token::Handler.register(:my_own) do |token_klass, stringcontent|
  # Do something with token_klass and stringcontent
  # but don't forget to add the token like, if you want to access it from the templates:
  # token_klass will be Token::MyOwnToken, which is dynamically created during registration

  self.add_token(token_klass.new(:content => stringcontent)
end

Because the token processing is done in the **context of the CodeObject** you can easily extend or manipulate the Object.

Parameters:

  • tokenname (String, Symbol)
  • options (Hash) (defaults to: {})

Options Hash (options):

  • :handler (Symbol) — default: :text_only
  • :template (Symbol, String) — default: :default —

    The template for this token-collection

  • :html (Hash) — default: {} —

    Attributes which can be added to the html-representation of this token. For example use 

    :html => { :class => 'big_button' }

    to add the class ‘.big_button` to the html-element. Please note, that your template has to render the html-attributes explicilty. For example by adding 

    <div <%= attributize token.html %>>...</div>
  • :area (Symbol) — default: :body —

    The area the tokens will be rendered in. The default templates make use of (:notification|:body|:sidebar|:footnote), but you can use any symbol here.

  • :description (String) — default: "" —

    The description specified here will appear in the command-line output of ‘docjs tokens`

Yields:

  • (tokenklass, stringcontent) —

    Your custom tokenhandler

See Also:



307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
 
# File 'lib/token/handler.rb', line 307

def self.register(tokenname, options = {}, &handler)
  
  tokenname = tokenname.to_sym   
  
  # search matching handler
  if block_given?
    # handler is already defined
  elsif options[:handler] and @@defaults.include?(options[:handler])
    handler = self.build_handler(options[:handler])
  elsif options[:handler]
    raise Exception, "#{type} has no registered Tokenhandler"
  else
    handler = self.build_handler(:text_only)
  end      
  
  # Dynamically create Class named TokennameToken
  camelcased = tokenname.to_s.capitalize.gsub(/_\w/){|w| w[1].capitalize}
  klass = Token.const_set "#{camelcased}Token", Class.new(Token)
  
  klass.process_options options.merge({
  
    :token    => tokenname,
    :handler  => handler
  
  });  
  
  
  @@types[tokenname] = options[:type] unless options[:type].nil?      
  @@handlers[tokenname] = klass
end

.types ⇒ Hash<Symbol, CodeObject::Base> with symbol beeing the token and CodeObject::Base any subclass

Returns Hash<Symbol, CodeObject::Base> with symbol beeing the token and CodeObject::Base any subclass.

Returns:

  • (Hash<Symbol, CodeObject::Base> with symbol beeing the token and CodeObject::Base any subclass) —

    Hash<Symbol, CodeObject::Base> with symbol beeing the token and CodeObject::Base any subclass



255
256
257
 
# File 'lib/token/handler.rb', line 255

def self.types
  @@types
end

.unregister(tokenname) ⇒ Object

TODO:

remove symbol ‘Token::TokennameToken`

Remove a registered handler from the list.

Examples:

Token::Handler.register :foo
Token::Handler.unregister :foo

Parameters:



347
348
349
 
# File 'lib/token/handler.rb', line 347

def self.unregister(tokenname)
  @@handlers.delete(tokenname.to_sym)
end