Class: Fig::StringTokenizer

Inherits:
Object
  • Object
show all
Defined in:
lib/fig/string_tokenizer.rb

Instance Method Summary collapse

Constructor Details

#initialize(subexpression_matchers = [], metacharacters = '') ⇒ StringTokenizer

subexpression_matchers is an array of hashes. Each hash is expected to contain two keys: :pattern and :action.

The :pattern value needs to be a regular expression for the substring that needs special handling.

The :action value needs to be a block that takes two parameters.

The first parameter is the text that was matched and the second is the error block passed to #tokenize().

On success the block returns either a String containing replacement text or a Fig::TokenizedString::Token representing the special handling of the consumed text. If there was a problem, then the error block should have been invoked and the block should return nil.

metacharacters is a regular expression character class for characters that need to be escaped when un-single quoting a string.



28
29
30
31
32
33
 
# File 'lib/fig/string_tokenizer.rb', line 28

def initialize(subexpression_matchers = [], metacharacters = '')
  @subexpression_matchers = subexpression_matchers
  @metacharacters         = metacharacters

  return
end

Instance Method Details

#tokenize(string, &error_block) ⇒ Object

Takes a block that is invoked when there is an error. Block receives a single parameter of an error message that is the end of a statement describing the problem, with no leading space character. For example, given «‘foo», the block will receive a message like ’has unbalanced single quotes.‘.

Returns the TokenizedString; if there was a parse error, then the return value will be nil (and the block will have been invoked).



43
44
45
46
47
48
49
50
51
52
53
54
 
# File 'lib/fig/string_tokenizer.rb', line 43

def tokenize(string, &error_block)
  @string        = string.dup
  @error_block   = error_block
  @single_quoted = nil
  @segments      = []

  strip_quotes_and_process_escapes

  return if @segments.empty?

  return Fig::TokenizedString.new(@segments, @single_quoted, @metacharacters)
end