Class: StringEater::Tokenizer

Inherits:
CTokenizer show all
Defined in:
lib/string-eater.rb

Method Summary

Methods inherited from CTokenizer

add_field, #ctokenize!, #describe_line, #do_extra_parsing, dup_tokens, #extract_all_fields, #extract_fields, #extract_no_fields, #initialize, look_for, #refresh_tokens, #tokenize!, #tokens, tokens

Constructor Details

This class inherits a constructor from StringEater::CTokenizer