Class: HeadMusic::Notation::LilyPond::Lexer

Inherits:
Object
  • Object
show all
Defined in:
lib/head_music/notation/lily_pond/lexer.rb

Overview

Splits a LilyPond document into tokens.

LilyPond is brace-structured rather than line-structured, so one scanner walks the whole document, skipping whitespace and comments as it goes (never pre-stripped, so a % inside a string survives and line numbers stay true). Notes, rests, and chord closers carry their duration; every other construct is a single-purpose token. Constructs outside the supported subset lex as :unsupported so the reader can name them where it meets them, or skip them with the block they sit in.

Constant Summary collapse

DURATION_PATTERN = 
/(?:\d+|\\breve|\\longa|\\maxima)\.*/
MULTIPLIER_PATTERN = 
%r{\*(\d+(?:/\d+)?)}
NOTE_PATTERN =

The Dutch contractions as/es/ases/eses come first so "es" is E-flat rather than the letter e followed by a stray s; the lookahead keeps words that start with a note letter (bass, alto, composer) whole.

/(?:(a|e)(ses|s)|([a-g])(isis|eses|is|es)?)(?![A-Za-z])('+|,+)?(#{DURATION_PATTERN})?(?:#{MULTIPLIER_PATTERN})?/o
REST_PATTERN = 
/([rR])(?![A-Za-z])(#{DURATION_PATTERN})?(?:#{MULTIPLIER_PATTERN})?/o
SPACER_PATTERN = 
/s(?![A-Za-z])#{DURATION_PATTERN}?/o
CLOSE_CHORD_PATTERN = 
/>(#{DURATION_PATTERN})?(?:#{MULTIPLIER_PATTERN})?/o
STRING_PATTERN = 
/"((?:[^"\\]|\\.)*)"/m
COMMAND_PATTERN = 
/\\([A-Za-z]+)/
WORD_PATTERN = 
/[A-Za-z_][A-Za-z0-9_]*/
NUMBER_PATTERN = 
%r{\d+(?:/\d+)?}
QUARTER_TONE_PATTERN =

Quarter-tone names (cih, ceh, cisih, ceseh) are real LilyPond pitches the model cannot hold, so they are named as unsupported rather than falling through as stray words.

/[a-g](?:isih|eseh|ih|eh)(?![A-Za-z])(?:'+|,+)?#{DURATION_PATTERN}?/o
MARK_PATTERN = 
/\\\\|#\S*|[\[\]()]|[-^_][.>^_+!-]?|[:!?]/
UNSUPPORTED_PATTERN =

A spacer rest starts with a letter no note or rest starts with, so it can be tried alongside the other unsupported constructs.

Regexp.union(MARK_PATTERN, QUARTER_TONE_PATTERN, SPACER_PATTERN)
ALIAS_SUFFIXES = 
{"s" => "es", "ses" => "eses"}.freeze
SIMPLE_TOKENS = 
{
  "<<" => :open_parallel, ">>" => :close_parallel, "<" => :open_chord,
  "{" => :open_brace, "}" => :close_brace,
  "|" => :bar_check, "~" => :tie, "=" => :equals
}.freeze
SIMPLE_PATTERN = 
Regexp.union(SIMPLE_TOKENS.keys.sort_by { |key| -key.length })

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(source) ⇒ Lexer

Returns a new instance of Lexer.



44
45
46
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 44

def initialize(source)
  @source = source
end

Instance Attribute Details

#scanner ⇒ Object (readonly, private)

Returns the value of attribute scanner.



54
55
56
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 54

def scanner
  @scanner
end

Instance Method Details

#close_chord_token ⇒ Object (private) 



97
98
99
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 97

def close_chord_token
  consume(CLOSE_CHORD_PATTERN) && timed_token(:close_chord, 1)
end

#command_token ⇒ Object (private) 



101
102
103
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 101

def command_token
  consume(COMMAND_PATTERN) && token(:command, lexeme: scanner[1])
end

#lexeme_token(pattern, type) ⇒ Object (private) 



126
127
128
129
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 126

def lexeme_token(pattern, type)
  lexeme = consume(pattern)
  lexeme && token(type, lexeme: lexeme)
end

#next_token ⇒ Object (private)

A token is stamped with where it began, remembered here because consuming its lexeme moves the scanner past it.



71
72
73
74
75
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 71

def next_token
  @token_line = scanner.line
  @token_column = scanner.column
  read_token || raise(scanner.unexpected_character_error)
end

#note_token ⇒ Object (private) 



105
106
107
108
109
110
111
112
113
114
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 105

def note_token
  return unless consume(NOTE_PATTERN)

  alias_letter, alias_suffix, letter, suffix, octave_marks, duration, multiplier = scanner.captures
  token(
    :note, lexeme: scanner.matched,
    letter: alias_letter || letter, suffix: alias_letter ? ALIAS_SUFFIXES.fetch(alias_suffix) : suffix,
    octave_marks: octave_marks, duration: duration, multiplier: multiplier
  )
end

#read_token ⇒ Object (private) 



77
78
79
80
81
82
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 77

def read_token
  string_token || simple_token || close_chord_token ||
    lexeme_token(UNSUPPORTED_PATTERN, :unsupported) || command_token ||
    note_token || rest_token ||
    lexeme_token(WORD_PATTERN, :word) || lexeme_token(NUMBER_PATTERN, :number)
end

#rest_token ⇒ Object (private) 



116
117
118
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 116

def rest_token
  consume(REST_PATTERN) && timed_token((scanner[1] == "R") ? :whole_bar_rest : :rest, 2)
end

#scan ⇒ Object (private) 



58
59
60
61
62
63
64
65
66
67
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 58

def scan
  @scanner = SourceScanner.new(@source)
  tokens = []
  until scanner.eos?
    next if scanner.skip_insignificant

    tokens << next_token
  end
  tokens
end

#simple_token ⇒ Object (private) 



92
93
94
95
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 92

def simple_token
  lexeme = consume(SIMPLE_PATTERN)
  lexeme && token(SIMPLE_TOKENS.fetch(lexeme), lexeme: lexeme)
end

#string_token ⇒ Object (private) 



84
85
86
87
88
89
90
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 84

def string_token
  if consume(STRING_PATTERN)
    token(:string, lexeme: StringText.unescape(scanner[1]))
  elsif scanner.match?(/"/)
    raise scanner.error("Unterminated string")
  end
end

#timed_token(type, duration_group) ⇒ Object (private)

A token whose lexeme ends in an optional duration and multiplier, the capture groups from the given one onward.



122
123
124
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 122

def timed_token(type, duration_group)
  token(type, lexeme: scanner.matched, duration: scanner[duration_group], multiplier: scanner[duration_group + 1])
end

#token(type, **fields) ⇒ Object (private) 



131
132
133
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 131

def token(type, **fields)
  Token.new(type: type, line: @token_line, column: @token_column, **fields)
end

#tokens ⇒ Object 



48
49
50
 
# File 'lib/head_music/notation/lily_pond/lexer.rb', line 48

def tokens
  @tokens ||= scan
end