Class: Ripper
- Inherits:
-
Object
- Object
- Ripper
- Defined in:
- lib/ripper.rb,
lib/ripper/core.rb,
lib/ripper/sexp.rb,
lib/ripper/lexer.rb,
lib/ripper/filter.rb,
ripper.c
Overview
Ripper is a Ruby script parser.
You can get information from the parser with event-based style. Information such as abstract syntax trees or simple lexical analysis of the Ruby program.
Usage
Ripper provides an easy interface for parsing your program into a symbolic expression tree (or S-expression).
Understanding the output of the parser may come as a challenge, it’s recommended you use PP to format the output for legibility.
require 'ripper'
require 'pp'
pp Ripper.sexp('def hello(world) "Hello, #{world}!"; end')
#=> [:program,
[[:def,
[:@ident, "hello", [1, 4]],
[:paren,
[:params, [[:@ident, "world", [1, 10]]], nil, nil, nil, nil, nil, nil]],
[:bodystmt,
[[:string_literal,
[:string_content,
[:@tstring_content, "Hello, ", [1, 18]],
[:string_embexpr, [[:var_ref, [:@ident, "world", [1, 27]]]]],
[:@tstring_content, "!", [1, 33]]]]],
nil,
nil,
nil]]]]
You can see in the example above, the expression starts with :program
.
From here, a method definition at :def
, followed by the method’s identifier :@ident
. After the method’s identifier comes the parentheses :paren
and the method parameters under :params
.
Next is the method body, starting at :bodystmt
(stmt
meaning statement), which contains the full definition of the method.
In our case, we’re simply returning a String, so next we have the :string_literal
expression.
Within our :string_literal
you’ll notice two @tstring_content
, this is the literal part for Hello,
and !
. Between the two @tstring_content
statements is a :string_embexpr
, where embexpr is an embedded expression. Our expression consists of a local variable, or var_ref
, with the identifier (@ident
) of world
.
Resources
Requirements
-
ruby 1.9 (support CVS HEAD only)
-
bison 1.28 or later (Other yaccs do not work)
License
Ruby License.
Minero Aoki
aamine@loveruby.net
http://i.loveruby.net
Direct Known Subclasses
Defined Under Namespace
Classes: Filter, Lexer, SexpBuilder, SexpBuilderPP, TokenPattern
Constant Summary collapse
- PARSER_EVENTS =
This array contains name of parser events.
PARSER_EVENT_TABLE.keys
- SCANNER_EVENTS =
This array contains name of scanner events.
SCANNER_EVENT_TABLE.keys
- EVENTS =
This array contains name of all ripper events.
PARSER_EVENTS + SCANNER_EVENTS
- Version =
version of Ripper
rb_usascii_str_new2(RIPPER_VERSION)
Class Method Summary collapse
-
.lex(src, filename = '-', lineno = 1) ⇒ Object
Tokenizes the Ruby program and returns an array of an array, which is formatted like
[[lineno, column], type, token]
. -
.parse(src, filename = '(ripper)', lineno = 1) ⇒ Object
Parses the given Ruby program read from
src
. -
.sexp(src, filename = '-', lineno = 1) ⇒ Object
- EXPERIMENTAL
-
Parses
src
and create S-exp tree.
-
.sexp_raw(src, filename = '-', lineno = 1) ⇒ Object
- EXPERIMENTAL
-
Parses
src
and create S-exp tree.
-
.slice(src, pattern, n = 0) ⇒ Object
- EXPERIMENTAL
-
Parses
src
and return a string which was matched topattern
.
-
.token_match(src, pattern) ⇒ Object
:nodoc:.
-
.tokenize(src, filename = '-', lineno = 1) ⇒ Object
Tokenizes the Ruby program and returns an array of strings.
Instance Method Summary collapse
-
#column ⇒ Integer
Return column number of current parsing line.
-
#encoding ⇒ Encoding
Return encoding of the source.
-
#end_seen? ⇒ Boolean
Return true if parsed source ended by _END_.
-
#filename ⇒ String
Return current parsing filename.
-
#new(src, filename = "(ripper)", lineno = 1) ⇒ Object
constructor
Create a new Ripper object.
-
#lineno ⇒ Integer
Return line number of current parsing line.
-
#parse ⇒ Object
Start parsing and returns the value of the root action.
-
#yydebug ⇒ Boolean
Get yydebug.
-
#yydebug=(flag) ⇒ Object
Set yydebug.
Constructor Details
#new(src, filename = "(ripper)", lineno = 1) ⇒ Object
Create a new Ripper object. src must be a String, an IO, or an Object which has #gets method.
This method does not starts parsing. See also Ripper#parse and Ripper.parse.
18029 18030 18031 18032 18033 18034 18035 18036 18037 18038 18039 18040 18041 18042 18043 18044 18045 18046 18047 18048 18049 18050 18051 18052 18053 18054 18055 18056 18057 18058 18059 |
# File 'ripper.c', line 18029
static VALUE
ripper_initialize(int argc, VALUE *argv, VALUE self)
{
struct parser_params *parser;
VALUE src, fname, lineno;
TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser);
rb_scan_args(argc, argv, "12", &src, &fname, &lineno);
if (RB_TYPE_P(src, T_FILE)) {
parser->parser_lex_gets = ripper_lex_get_generic;
}
else {
StringValue(src);
parser->parser_lex_gets = lex_get_str;
}
parser->parser_lex_input = src;
parser->eofp = Qfalse;
if (NIL_P(fname)) {
fname = STR_NEW2("(ripper)");
}
else {
StringValue(fname);
}
parser_initialize(parser);
parser->parser_ruby_sourcefile_string = fname;
parser->parser_ruby_sourcefile = RSTRING_PTR(fname);
parser->parser_ruby_sourceline = NIL_P(lineno) ? 0 : NUM2INT(lineno) - 1;
return Qnil;
}
|
Class Method Details
.lex(src, filename = '-', lineno = 1) ⇒ Object
Tokenizes the Ruby program and returns an array of an array, which is formatted like [[lineno, column], type, token]
.
require 'ripper'
require 'pp'
pp Ripper.lex("def m(a) nil end")
#=> [[[1, 0], :on_kw, "def"],
[[1, 3], :on_sp, " " ],
[[1, 4], :on_ident, "m" ],
[[1, 5], :on_lparen, "(" ],
[[1, 6], :on_ident, "a" ],
[[1, 7], :on_rparen, ")" ],
[[1, 8], :on_sp, " " ],
[[1, 9], :on_kw, "nil"],
[[1, 12], :on_sp, " " ],
[[1, 13], :on_kw, "end"]]
42 43 44 |
# File 'lib/ripper/lexer.rb', line 42 def Ripper.lex(src, filename = '-', lineno = 1) Lexer.new(src, filename, lineno).lex end |
.parse(src, filename = '(ripper)', lineno = 1) ⇒ Object
Parses the given Ruby program read from src
. src
must be a String or an IO or a object with a #gets method.
17 18 19 |
# File 'lib/ripper/core.rb', line 17 def Ripper.parse(src, filename = '(ripper)', lineno = 1) new(src, filename, lineno).parse end |
.sexp(src, filename = '-', lineno = 1) ⇒ Object
- EXPERIMENTAL
-
Parses
src
and create S-exp tree. Returns more readable tree rather than Ripper.sexp_raw. This method is mainly for developer use.require 'ripper' require 'pp' pp Ripper.sexp("def m(a) nil end") #=> [:program, [[:def, [:@ident, "m", [1, 4]], [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil, nil]], [:bodystmt, [[:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]
30 31 32 |
# File 'lib/ripper/sexp.rb', line 30 def Ripper.sexp(src, filename = '-', lineno = 1) SexpBuilderPP.new(src, filename, lineno).parse end |
.sexp_raw(src, filename = '-', lineno = 1) ⇒ Object
- EXPERIMENTAL
-
Parses
src
and create S-exp tree. This method is mainly for developer use.require 'ripper' require 'pp' pp Ripper.sexp_raw("def m(a) nil end") #=> [:program, [:stmts_add, [:stmts_new], [:def, [:@ident, "m", [1, 4]], [:paren, [:params, [[:@ident, "a", [1, 6]]], nil, nil, nil]], [:bodystmt, [:stmts_add, [:stmts_new], [:var_ref, [:@kw, "nil", [1, 9]]]], nil, nil, nil]]]]
54 55 56 |
# File 'lib/ripper/sexp.rb', line 54 def Ripper.sexp_raw(src, filename = '-', lineno = 1) SexpBuilder.new(src, filename, lineno).parse end |
.slice(src, pattern, n = 0) ⇒ Object
- EXPERIMENTAL
-
Parses
src
and return a string which was matched topattern
.pattern
should be described as Regexp.require 'ripper' p Ripper.slice('def m(a) nil end', 'ident') #=> "m" p Ripper.slice('def m(a) nil end', '[ident lparen rparen]+') #=> "m(a)" p Ripper.slice("<<EOS\nstring\nEOS", 'heredoc_beg nl $(tstring_content*) heredoc_end', 1) #=> "string\n"
84 85 86 87 88 89 |
# File 'lib/ripper/lexer.rb', line 84 def Ripper.slice(src, pattern, n = 0) if m = token_match(src, pattern) then m.string(n) else nil end end |
.token_match(src, pattern) ⇒ Object
:nodoc:
91 92 93 |
# File 'lib/ripper/lexer.rb', line 91 def Ripper.token_match(src, pattern) #:nodoc: TokenPattern.compile(pattern).match(src) end |
.tokenize(src, filename = '-', lineno = 1) ⇒ Object
Tokenizes the Ruby program and returns an array of strings.
p Ripper.tokenize("def m(a) nil end")
# => ["def", " ", "m", "(", "a", ")", " ", "nil", " ", "end"]
20 21 22 |
# File 'lib/ripper/lexer.rb', line 20 def Ripper.tokenize(src, filename = '-', lineno = 1) Lexer.new(src, filename, lineno).tokenize end |
Instance Method Details
#column ⇒ Integer
Return column number of current parsing line. This number starts from 0.
18122 18123 18124 18125 18126 18127 18128 18129 18130 18131 18132 18133 18134 18135 |
# File 'ripper.c', line 18122
static VALUE
ripper_column(VALUE self)
{
struct parser_params *parser;
long col;
TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser);
if (!ripper_initialized_p(parser)) {
rb_raise(rb_eArgError, "method called for uninitialized object");
}
if (NIL_P(parser->parsing_thread)) return Qnil;
col = parser->tokp - parser->parser_lex_pbeg;
return LONG2NUM(col);
}
|
#encoding ⇒ Encoding
Return encoding of the source.
17624 17625 17626 17627 17628 17629 17630 17631 |
# File 'ripper.c', line 17624
VALUE
rb_parser_encoding(VALUE vparser)
{
struct parser_params *parser;
TypedData_Get_Struct(vparser, struct parser_params, &parser_data_type, parser);
return rb_enc_from_encoding(current_enc);
}
|
#end_seen? ⇒ Boolean
Return true if parsed source ended by _END_.
17609 17610 17611 17612 17613 17614 17615 17616 |
# File 'ripper.c', line 17609
VALUE
rb_parser_end_seen_p(VALUE vparser)
{
struct parser_params *parser;
TypedData_Get_Struct(vparser, struct parser_params, &parser_data_type, parser);
return ruby__end__seen ? Qtrue : Qfalse;
}
|
#filename ⇒ String
Return current parsing filename.
18143 18144 18145 18146 18147 18148 18149 18150 18151 18152 18153 |
# File 'ripper.c', line 18143
static VALUE
ripper_filename(VALUE self)
{
struct parser_params *parser;
TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser);
if (!ripper_initialized_p(parser)) {
rb_raise(rb_eArgError, "method called for uninitialized object");
}
return parser->parser_ruby_sourcefile_string;
}
|
#lineno ⇒ Integer
Return line number of current parsing line. This number starts from 1.
18162 18163 18164 18165 18166 18167 18168 18169 18170 18171 18172 18173 |
# File 'ripper.c', line 18162
static VALUE
ripper_lineno(VALUE self)
{
struct parser_params *parser;
TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser);
if (!ripper_initialized_p(parser)) {
rb_raise(rb_eArgError, "method called for uninitialized object");
}
if (NIL_P(parser->parsing_thread)) return Qnil;
return INT2NUM(parser->parser_ruby_sourceline);
}
|
#parse ⇒ Object
Start parsing and returns the value of the root action.
18094 18095 18096 18097 18098 18099 18100 18101 18102 18103 18104 18105 18106 18107 18108 18109 18110 18111 18112 18113 |
# File 'ripper.c', line 18094
static VALUE
ripper_parse(VALUE self)
{
struct parser_params *parser;
TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser);
if (!ripper_initialized_p(parser)) {
rb_raise(rb_eArgError, "method called for uninitialized object");
}
if (!NIL_P(parser->parsing_thread)) {
if (parser->parsing_thread == rb_thread_current())
rb_raise(rb_eArgError, "Ripper#parse is not reentrant");
else
rb_raise(rb_eArgError, "Ripper#parse is not multithread-safe");
}
parser->parsing_thread = rb_thread_current();
rb_ensure(ripper_parse0, self, ripper_ensure, self);
return parser->result;
}
|
#yydebug ⇒ Boolean
Get yydebug.
17639 17640 17641 17642 17643 17644 17645 17646 |
# File 'ripper.c', line 17639
VALUE
rb_parser_get_yydebug(VALUE self)
{
struct parser_params *parser;
TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser);
return yydebug ? Qtrue : Qfalse;
}
|
#yydebug=(flag) ⇒ Object
Set yydebug.
17654 17655 17656 17657 17658 17659 17660 17661 17662 |
# File 'ripper.c', line 17654
VALUE
rb_parser_set_yydebug(VALUE self, VALUE flag)
{
struct parser_params *parser;
TypedData_Get_Struct(self, struct parser_params, &parser_data_type, parser);
yydebug = RTEST(flag);
return flag;
}
|