Class: SClust::Util::DocumentTokenizer

Inherits:

TokenizerFilter

• Object
• Filter
• TokenizerFilter
• SClust::Util::DocumentTokenizer

Defined in:

lib/sclust/util/filters.rb

Overview

A tokenizer that applies a few overall document filters.

Instance Method Summary

• #initialize ⇒ DocumentTokenizer constructor

  A new instance of DocumentTokenizer.

Methods inherited from TokenizerFilter

#filter

Methods inherited from Filter

#after, #apply, #before, #filter

Constructor Details

#initialize ⇒ DocumentTokenizer

Returns a new instance of DocumentTokenizer.

# File 'lib/sclust/util/filters.rb', line 176

def initialize()
    super()
    after(HTMLFilter.new())
end