Class: TwitterCldr::Tokenizers::NumberTokenizer
- Inherits:
-
Object
- Object
- TwitterCldr::Tokenizers::NumberTokenizer
- Defined in:
- lib/twitter_cldr/tokenizers/numbers/number_tokenizer.rb
Instance Attribute Summary collapse
-
#data_reader ⇒ Object
readonly
Returns the value of attribute data_reader.
Instance Method Summary collapse
-
#initialize(data_reader) ⇒ NumberTokenizer
constructor
A new instance of NumberTokenizer.
- #tokenize(pattern) ⇒ Object
Constructor Details
#initialize(data_reader) ⇒ NumberTokenizer
Returns a new instance of NumberTokenizer.
12 13 14 |
# File 'lib/twitter_cldr/tokenizers/numbers/number_tokenizer.rb', line 12 def initialize(data_reader) @data_reader = data_reader end |
Instance Attribute Details
#data_reader ⇒ Object (readonly)
Returns the value of attribute data_reader.
10 11 12 |
# File 'lib/twitter_cldr/tokenizers/numbers/number_tokenizer.rb', line 10 def data_reader @data_reader end |
Instance Method Details
#tokenize(pattern) ⇒ Object
16 17 18 19 20 21 22 23 24 |
# File 'lib/twitter_cldr/tokenizers/numbers/number_tokenizer.rb', line 16 def tokenize(pattern) tokens = PatternTokenizer.new(data_reader, tokenizer).tokenize(pattern) if tokens.first.value == "" tokens[1..-1] else tokens end end |