Module: Token

Defined in:

lib/token/handler.rb,
lib/token/token.rb,
lib/token/container.rb,
lib/token/exceptions.rb

Overview

This module contains all required mixins and modules to register customized tokens, that will be further processed and added to the CodeObject::Base.

The converter starts the tokenline-processing, by calling the mixed-in function Container#process_token.

There are **two modules** included in Token:

1. {Token::Handler}, which can be used to register new token-handlers.
2. {Token::Container}, which is included in {CodeObject::Base} to add 
    individual token-{Token::Container#add_token container} and 
    {Token::Container#process_token -processing} functionality to all CodeObjects.

Tokenprocessing means analysing the tokenline (plain textual content associated with a token) by applying a tokenhandler to it. The tokenhandler should parse the tokenline into it’s contents and afterwards create an instance of the token-class.

Defined Under Namespace

Modules: Container, Handler Classes: NoTokenHandler, Token, WrongType