Class: TextRank::TokenFilter::MinLength

Inherits:
Object
  • Object
show all
Defined in:
lib/text_rank/token_filter/min_length.rb

Overview

Token filter to remove "small" tokens

= Example

MinLength.new(min_length: 6).filter!(%w[ and ask each passenger to tell his story and if there is one of them all who has not cursed his existence many times and said to himself over and over again that he was the most miserable of men i give you permission to throw me head-first into the sea ]) => ["passenger", "cursed", "existence", "himself", "miserable", "permission", "head-first"]

Instance Method Summary collapse

Constructor Details

#initialize(min_length: 3, **_) ⇒ MinLength

Returns a new instance of MinLength.

Parameters:

  • min_length (Fixnum) (defaults to: 3) —

    minimum size of token to keep



18
19
20
 
# File 'lib/text_rank/token_filter/min_length.rb', line 18

def initialize(min_length: 3, **_)
  @min_length = min_length
end

Instance Method Details

#filter!(tokens) ⇒ Array<String>

Perform the filter

Parameters:

  • tokens (Array<String>)

Returns:

  • (Array<String>) 


25
26
27
28
29
 
# File 'lib/text_rank/token_filter/min_length.rb', line 25

def filter!(tokens)
  tokens.keep_if do |token|
    token.size >= @min_length
  end
end