Class: Ferret::Analysis::AsciiWhiteSpaceAnalyzer

Inherits:

Object

Object
Ferret::Analysis::AsciiWhiteSpaceAnalyzer

show all

Defined in:: ext/r_analysis.c

Overview

Summary

The AsciiWhiteSpaceAnalyzer recognizes tokens as maximal strings of non-whitespace characters. If implemented in Ruby the AsciiWhiteSpaceAnalyzer would look like;

class AsciiWhiteSpaceAnalyzer
def initialize(lower = true)
  @lower = lower
end

def token_stream(field, str)
  if @lower
    return AsciiLowerCaseFilter.new(AsciiWhiteSpaceTokenizer.new(str))
  else
    return AsciiWhiteSpaceTokenizer.new(str)
  end
end
end

As you can see it makes use of the AsciiWhiteSpaceTokenizer. You should use WhiteSpaceAnalyzer if you want to recognize multibyte encodings such as "UTF-8".