Class: RMMSeg::Ferret::PunctuationFilter

Inherits:
Ferret::Analysis::TokenStream
  • Object
show all
Defined in:
lib/rmmseg/ferret.rb

Overview

PunctuationFilter filter out the stand alone Chinese punctuation tokens.

Defined Under Namespace

Classes: Dictionary

Instance Method Summary collapse

Constructor Details

#initialize(stream) ⇒ PunctuationFilter

Returns a new instance of PunctuationFilter.



84
85
86
 
# File 'lib/rmmseg/ferret.rb', line 84

def initialize(stream)
  @stream = stream
end

Instance Method Details

#next ⇒ Object

Get next token, skip stand alone Chinese punctuations.



89
90
91
92
93
94
95
96
97
98
 
# File 'lib/rmmseg/ferret.rb', line 89

def next
  token = @stream.next
  dic = Dictionary.instance

  until token.nil? || !(dic.include? token.text)
    token = @stream.next
  end

  token
end

#text ⇒ Object 



100
101
102
 
# File 'lib/rmmseg/ferret.rb', line 100

def text
  @stream.text
end

#text=(str) ⇒ Object 



104
105
106
 
# File 'lib/rmmseg/ferret.rb', line 104

def text=(str)
  @stream.text = str
end