Class: Ferret::Analysis::StandardAnalyzer
- Inherits:
-
Object
- Object
- Ferret::Analysis::StandardAnalyzer
- Defined in:
- ext/r_analysis.c
Overview
Summary
The StandardAnalyzer is the most advanced of the available analyzers. If it were implemented in Ruby it would look like this;
class StandardAnalyzer
def initialize(stop_words = FULL_ENGLISH_STOP_WORDS, lower = true)
@lower = lower
@stop_words = stop_words
end
def token_stream(field, str)
ts = StandardTokenizer.new(str)
ts = LowerCaseFilter.new(ts) if @lower
ts = StopFilter.new(ts, @stop_words)
ts = HyphenFilter.new(ts)
end
end
As you can see it makes use of the StandardTokenizer and you can also add your own list of stopwords if you wish.