rattler

DESCRIPTION:

Rattler - Ruby Tool for Language Recognition

Parsing for Ruby that’s so easy it feels like cheating.

Rattler is a parser generator for Ruby based on parsing expression grammars. Rattler’s purpose is to make language design and recognition as simple and fun as possible. To that end, the normal parsing expression grammar syntax is extended with simple features to overcome some of the limitations of pure PEG grammars.

A language syntax is specified in a grammar using the Rattler syntax. Parser classes and modules can be generated statically using the “rtlr” command or dynamically from strings.

The cucumber features are currently the best documentation.

RDoc

FEATURES:

• Uses readable PEG-like grammars

• Supports directly and indirectly left-recursive grammars

• Whitespace can be specified in one place and skipped automatically

• Automatic and custom parse error reporting

• Optimizing parser generator generates efficient pure-ruby parsers

• Compatible with Ruby 1.8.7 and 1.9.2, tested with MRI and JRuby on Linux and Windows and with Rubinius on Linux

PROBLEMS/LIMITATIONS:

• Only strings can be parsed, so files have to be read completely before parsing

• There are many holes in the tests so there are undoubtedly many bugs

• Support for composable grammars is not yet implemented

• Semantics of automatic whitespace skipping could be refined

• Incomplete documentation

EXAMPLES:

Example 1: Statically generated parser class

json.rtlr:

# JSON parser based on the grammar at http://www.json.org

require File.expand_path('json_helper', File.dirname(__FILE__))

parser JsonParser < Rattler::Runtime::PackratParser

include JsonHelper

%whitespace (SPACE+ / comment)* {

  json_text <-  (object / array) EOF

  object    <-  ~'{' members ~'}'                                   { object _ }

  members   <-  pair *, ','

  pair      <-  string ~':' value

  array     <-  ~'[' elements ~']'

  elements  <-  value *, ','

  value     <-  string
              / number
              / object
              / array
              / `true`                                              { :true }
              / `false`                                             { :false }
              / `null`                                              { :null }
              / fail "value expected"

  string    <-  @('"' char* '"')                                    { string _ }

  number    <-  @(int frac? exp?)                                   { number _ }
}

%inline {

  char      <-  !('"' / '\\' / CNTRL) .
              / '\\' (["\\/bfnrt] / 'u' XDIGIT XDIGIT XDIGIT XDIGIT)

  int       <-  '-'? ('0' !DIGIT / [1-9] DIGIT*)

  frac      <-  '.' DIGIT+

  exp       <-  [eE] [+-]? DIGIT+

  comment   <-  '/*' (! '*/' .)* '*/'
              / '//' [^\n]*
}

json_helper.rb:

module JsonHelper

  def object(members)
    Hash[*members.flatten(1)]
  end

  def string(expr)
    eval "%q#{expr}", TOPLEVEL_BINDING
  end

  def number(expr)
    eval expr, TOPLEVEL_BINDING
  end

end

$ rtlr json.rtlr

json.rtlr -> json_parser.rb

parse_json.rb:

require 'rubygems'
require 'rattler'
require 'json_parser'

begin
  p JsonParser.parse!(open(ARGV[0]) {|io| io.read })
rescue Rattler::Runtime::SyntaxError => e
  puts e
end

Example 2: Statically generated grammar module

json.rtlr:

# JSON parser based on the grammar at http://www.json.org

grammar JsonGrammar

%whitespace (SPACE+ / comment)* {

...

$ rtlr json.rtlr

json.rtlr -> json_grammar.rb

json_parser.rb:

require 'rubygems'
require 'rattler'
require 'json_grammar'

class JsonParser < Rattler::Runtime::PackratParser

  include JsonGrammar

  def object(members)
    Hash[*members.flatten(1)]
  end

  ...

Example 3: Dynamically generated parser class

require 'rubygems'
require 'rattler'

Calculator = Rattler::compile_parser %{
  %whitespace SPACE*

  start   <-  expr EOF

  expr    <-  expr ~'+' term                  {|a, b| a + b }
            / expr ~'-' term                  {|a, b| a - b }
            / term

  term    <-  term ~'*' primary               {|a, b| a * b }
            / term ~'/' primary               {|a, b| a / b }
            / primary

  primary <-  ~'(' expr ~')'
            / @('-'? DIGIT+ ('.' DIGIT+)?)    { _.to_f }
},
:type => :extended_packrat

begin
  puts Calculator.parse!(expr)
rescue Rattler::Runtime::SyntaxError => e
  puts e
end

REQUIREMENTS:

• Ruby 1.8.7 or greater

• RSpec recommended

INSTALL:

sudo gem install rattler

Contributing to rattler

• Check out the latest master to make sure the feature hasn’t been implemented or the bug hasn’t been fixed yet

• Check out the issue tracker to make sure someone already hasn’t requested it and/or contributed it

• Fork the project

• Start a feature/bugfix branch

• Commit and push until you are happy with your contribution

• Make sure to add tests for it. This is important so I don’t break it in a future version unintentionally.

• Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.

Copyright

Copyright © 2011 Jason Arhart. See LICENSE.txt for further details.