Class: TwitterCldr::Transforms::Tokenizer

Inherits:

Object

Object
TwitterCldr::Transforms::Tokenizer

show all

Defined in:: lib/twitter_cldr/transforms/tokenizer.rb

Instance Method Summary collapse

#tokenize(rule_text) ⇒ Object

Instance Method Details

#tokenize(rule_text) ⇒ `Object`



10
11
12

# File 'lib/twitter_cldr/transforms/tokenizer.rb', line 10

def tokenize(rule_text)
  tokenizer.tokenize(rule_text)
end