Class: Ferret::Analysis::AsciiLetterAnalyzer

Inherits:

Object

Object
Ferret::Analysis::AsciiLetterAnalyzer

show all

Defined in:: ext/r_analysis.c

Overview

Summary

An AsciiLetterAnalyzer creates a TokenStream that splits the input up into maximal strings of ASCII characters. If implemented in Ruby it would look like;

class AsciiLetterAnalyzer
  def initialize(lower = true)
    @lower = lower
  end

  def token_stream(field, str)
    if @lower
      return AsciiLowerCaseFilter.new(AsciiLetterTokenizer.new(str))
    else
      return AsciiLetterTokenizer.new(str)
    end
  end
end

As you can see it makes use of the AsciiLetterTokenizer and AsciiLowerCaseFilter. Note that this tokenizer won’t recognize non-ASCII characters so you should use the LetterAnalyzer is you want to analyze multi-byte data like “UTF-8”.