Class: Sass::Script::Lexer
- Includes:
- Sass::SCSS::RX
- Defined in:
- lib/sass/script/lexer.rb
Overview
The lexical analyzer for SassScript. It takes a raw string and converts it to individual tokens that are easier to parse.
Direct Known Subclasses
Defined Under Namespace
Classes: Token
Constant Summary collapse
- STRING_REGULAR_EXPRESSIONS =
A hash of regular expressions that are used for tokenizing strings.
The key is a [Symbol, Boolean] pair. The symbol represents which style of quotation to use, while the boolean represents whether or not the string is following an interpolated segment.
{ [:double, false] => string_re('"', '"'), [:single, false] => string_re("'", "'"), [:double, true] => string_re('', '"'), [:single, true] => string_re('', "'"), }
Constants included from Sass::SCSS::RX
Sass::SCSS::RX::CDC, Sass::SCSS::RX::CDO, Sass::SCSS::RX::COMMENT, Sass::SCSS::RX::DASHMATCH, Sass::SCSS::RX::DEFAULT, Sass::SCSS::RX::ESCAPE, Sass::SCSS::RX::FUNCTION, Sass::SCSS::RX::GREATER, Sass::SCSS::RX::H, Sass::SCSS::RX::HASH, Sass::SCSS::RX::HEXCOLOR, Sass::SCSS::RX::IDENT, Sass::SCSS::RX::IMPORTANT, Sass::SCSS::RX::INCLUDES, Sass::SCSS::RX::INTERP_START, Sass::SCSS::RX::NAME, Sass::SCSS::RX::NL, Sass::SCSS::RX::NMCHAR, Sass::SCSS::RX::NMSTART, Sass::SCSS::RX::NONASCII, Sass::SCSS::RX::NOT, Sass::SCSS::RX::NUM, Sass::SCSS::RX::NUMBER, Sass::SCSS::RX::PLUS, Sass::SCSS::RX::PREFIXMATCH, Sass::SCSS::RX::RANGE, Sass::SCSS::RX::S, Sass::SCSS::RX::SINGLE_LINE_COMMENT, Sass::SCSS::RX::STRING, Sass::SCSS::RX::STRING1, Sass::SCSS::RX::STRING2, Sass::SCSS::RX::SUBSTRINGMATCH, Sass::SCSS::RX::SUFFIXMATCH, Sass::SCSS::RX::TILDE, Sass::SCSS::RX::UNICODE, Sass::SCSS::RX::UNICODERANGE, Sass::SCSS::RX::URI, Sass::SCSS::RX::URL, Sass::SCSS::RX::W
Instance Attribute Summary collapse
-
#line ⇒ Fixnum
readonly
The line number of the lexer's current position.
-
#offset ⇒ Fixnum
readonly
The number of bytes into the current line of the lexer's current position.
Instance Method Summary collapse
-
#done? ⇒ Boolean
Whether or not there's more source text to lex.
- #expected!(name)
-
#initialize(str, line, offset, options) ⇒ Lexer
constructor
A new instance of Lexer.
-
#next ⇒ Token
Moves the lexer forward one token.
-
#peek ⇒ Token
Returns the next token without moving the lexer forward.
- #str
-
#unpeek!
Rewinds the underlying StringScanner to before the token returned by #peek.
-
#whitespace?(tok = @tok) ⇒ Boolean
Returns whether or not there's whitespace before the next token.
Methods included from Sass::SCSS::RX
Constructor Details
#initialize(str, line, offset, options) ⇒ Lexer
Returns a new instance of Lexer.
132 133 134 135 136 137 138 139 |
# File 'lib/sass/script/lexer.rb', line 132
def initialize(str, line, offset, options)
@scanner = str.is_a?(StringScanner) ? str : StringScanner.new(str)
@line = line
@offset = offset
@options = options
@interpolation_stack = []
@prev = nil
end
|
Instance Attribute Details
#line ⇒ Fixnum (readonly)
The line number of the lexer's current position.
34 35 36 |
# File 'lib/sass/script/lexer.rb', line 34
def line
@line
end
|
#offset ⇒ Fixnum (readonly)
The number of bytes into the current line of the lexer's current position.
40 41 42 |
# File 'lib/sass/script/lexer.rb', line 40
def offset
@offset
end
|
Instance Method Details
#done? ⇒ Boolean
Returns Whether or not there's more source text to lex.
177 178 179 180 |
# File 'lib/sass/script/lexer.rb', line 177
def done?
whitespace unless after_interpolation? && @interpolation_stack.last
@scanner.eos? && @tok.nil?
end
|
#expected!(name)
182 183 184 185 |
# File 'lib/sass/script/lexer.rb', line 182
def expected!(name)
unpeek!
Sass::SCSS::Parser.expected(@scanner, name, @line)
end
|
#next ⇒ Token
Moves the lexer forward one token.
144 145 146 147 148 149 |
# File 'lib/sass/script/lexer.rb', line 144
def next
@tok ||= read_token
@tok, tok = nil, @tok
@prev = tok
return tok
end
|
#peek ⇒ Token
Returns the next token without moving the lexer forward.
166 167 168 |
# File 'lib/sass/script/lexer.rb', line 166
def peek
@tok ||= read_token
end
|
#str
187 188 189 190 191 192 |
# File 'lib/sass/script/lexer.rb', line 187
def str
old_pos = @tok ? @tok.pos : @scanner.pos
yield
new_pos = @tok ? @tok.pos : @scanner.pos
@scanner.string[old_pos...new_pos]
end
|
#unpeek!
Rewinds the underlying StringScanner to before the token returned by #peek.
172 173 174 |
# File 'lib/sass/script/lexer.rb', line 172
def unpeek!
@scanner.pos = @tok.pos if @tok
end
|
#whitespace?(tok = @tok) ⇒ Boolean
Returns whether or not there's whitespace before the next token.
154 155 156 157 158 159 160 161 |
# File 'lib/sass/script/lexer.rb', line 154
def whitespace?(tok = @tok)
if tok
@scanner.string[0...tok.pos] =~ /\s$/
else
@scanner.string[@scanner.pos, 1] =~ /^\s/ ||
@scanner.string[@scanner.pos - 1, 1] =~ /\s$/
end
end
|