Module: TextRank::TokenFilter

Defined in:

lib/text_rank/token_filter.rb,
lib/text_rank/token_filter/stopwords.rb,
lib/text_rank/token_filter/min_length.rb,
lib/text_rank/token_filter/part_of_speech.rb

Overview

Token filters can be used to pre-process potential tokens prior to creating a graph or executing PageRank. Filters are typically used to throw out tokens which are not good candidates for keywords. However, it is possible for a filter to add new tokens or to modify existing ones.

Token filters are applied as a chain, so care should be taken to use them in the desired order.

Defined Under Namespace

Classes: MinLength, PartOfSpeech, Stopwords