Class: TwitterCldr::Tokenizers::UnicodeRegexTokenizer

Inherits:

Object

Object
TwitterCldr::Tokenizers::UnicodeRegexTokenizer

show all

Extended by:: Forwardable

Defined in:: lib/twitter_cldr/tokenizers/unicode_regex/unicode_regex_tokenizer.rb

Instance Method Summary collapse

#tokenize(pattern) ⇒ Object

Instance Method Details

#tokenize(pattern) ⇒ `Object`



13
14
15

# File 'lib/twitter_cldr/tokenizers/unicode_regex/unicode_regex_tokenizer.rb', line 13

def tokenize(pattern)
  tokenizer.tokenize(pattern)
end