memprof
(c) Joe Damato @joedamato http://timetobleed.com
Memprof is a Ruby level memory profiler that can help you find reference leaks in your application.
Memprof can also do very lightweight function call tracing to figure out which system and library calls are happening in your code.
Installation
gem install memprof
API
Memprof.stats
Memprof.start
12.times{ "abc" }
Memprof.stats
Memprof.stop
Start tracking file/line information for objects created after calling
Memprof.start, and print out a summary of file:line/class pairs
created.
12 file.rb:2:String
Note: Call Memprof.stats again after GC.start to see which objects
are cleaned up by the garbage collector:
Memprof.start
10.times{ $last_str = "abc" }
puts '=== Before GC'
Memprof.stats
puts '=== After GC'
GC.start
Memprof.stats
Memprof.stop
After GC.start, only the very last instance of "abc" will still
exist:
=== Before GC
10 file.rb:2:String
=== After GC
1 file.rb:2:String
Note: Use Memprof.stats("/path/to/file") to write results to a file.
Note: Use Memprof.stats! to clear out tracking data after printing
out results.
Memprof.track
Simple wrapper for Memprof.stats that will start/stop memprof around a
given block of ruby code.
Memprof.track{
100.times{ "abc" }
100.times{ 1.23 + 1 }
100.times{ Module.new }
}
For the block of ruby code, print out file:line/class pairs for ruby objects created.
100 file.rb:2:String
100 file.rb:3:Float
100 file.rb:4:Module
Note: You can call GC.start at the end of the block to print out only objects that are 'leaking' (i.e. objects that still have inbound references).
Note: Use Memprof.track("/path/to/file") to write the results to a
file instead of stdout.
Memprof.dump
Memprof.dump{
"hello" + "world"
}
Dump out all objects created in a given ruby block as detailed json objects.
{
"_id": "0x15e5018",
"file": "file.rb",
"line": 2,
"type": "string",
"class_name": "String",
"length": 10,
"data": "helloworld"
}
Note: Use Memprof.dump("/path/to/filename") to write the json output
to a file, one per line.
Memprof.dump_all
Memprof.dump_all("myapp_heap.json")
Dump out all live objects inside the Ruby VM to myapp_heap.json, one
per line.
memprof.com heap visualizer
# load memprof before requiring rubygems, so objects created by
# rubygems itself are tracked by memprof too
require `gem which memprof/signal`.strip
require 'rubygems'
require 'myapp'
Installs a URG signal handler and starts tracking file/line
information for newly created ruby objects. When the process receives
SIGURG, it will fork and call Memprof.dump_all to write out the
entire heap to a json file.
Use the memprof command to send the signal and upload the heap to
memprof.com:
memprof --pid <PID> --name my_leaky_app --key <API_KEY>
Memprof.trace
require 'open-uri'
require 'mysql'
require 'memcached'
Memprof.trace{
10.times{ Module.new }
10.times{ GC.start }
10.times{ open('http://google.com/') }
10.times{ Mysql.connect.query("select 1+2") }
10.times{ Memcached.new.get('memprof') }
}
For a given block of ruby code, count:
- number of objects created per type
- number of calls to and time spent in GC
- number of calls to and time spent in connect/read/write/select
- number of calls to and time spent in mysql queries
- number of calls to and responses to memcached commands
- number of calls to and bytes through malloc/realloc/free
The resulting json report looks like:
{
"objects": {
"created": 10,
"types": {
"module": 10, # Module.new
}
},
"gc": {
"calls": 10, # GC.start
"time": 0.17198
},
"fd": {
"connect": {
"calls": 10, # open('http://google.com')
"time": 0.0110
}
},
"mysql": {
"queries": 10, # Mysql.connect.query("select 1+2")
"time": 0.0006
},
"memcache": {
"get": {
"calls": 10, # Memcached.new.get('memprof')
"responses": {
"notfound": 10
}
}
}
}
Note: To write json to a file instead, set Memprof.trace_filename =
"/path/to/file.json"
Memprof.trace_request
Memprof.trace_request(env){ @app.call(env) }
Like Memprof.trace, but assume an incoming Rack request and include
information about the request itself.
{
"start" : 1272424769750716,
"tracers" : {
/* ... */
},
"rails" : {
"controller" : "home",
"action" : "index"
},
"request" : {
"REQUEST_URI" : "/home",
"REQUEST_METHOD" : "GET",
"REMOTE_ADDR" : "127.0.0.1",
"QUERY_STRING" : null
},
"time" : 1.3442
}
Middlewares
Memprof::Middleware
require 'memprof/middleware'
config.middlewares.use(Memprof::Middleware)
Wrap each request in a Memprof.track to print out all object
location/type pairs created during that request.
Note: It is preferable to run this in staging or production mode with Rails applications, since development mode creates a lot of unnecessary objects during each request.
Note: To force a GC run before printing out a report, pass in
:force_gc => true to the middleware.
Memprof::Tracer
require 'memprof/tracer'
config.middleware.insert(0, Memprof::Tracer)
Wrap each request in a Memprof.trace_request and write results to
/tmp/memprof_tracer-PID.json
Memprof::Filter
Similar to Memprof::Tracer, but for legacy Rails 2.2 applications.
class ApplicationController < ActionController::Base
require 'memprof/tracer'
around_filter(Memprof::Filter)
end
Compatibility
Memprof supports all 1.8.x (MRI and REE) VMs, as long as they are 64-bit and contain debugging symbols. For best results, use RVM to compile ruby and make sure you are on a 64-bit machine.
The following ruby builds are not supported:
- Ruby on small/medium EC2 instances (32-bit machines)
- OSX's default system ruby (no debugging symbols, fat 32/64-bit binary)
Note: Many linux distributions do not package debugging symbols by
default. You can usually install these separately, for example using
apt-get install libruby1.8-dbg
Coming soon
- support for Ruby 1.9
- support for i386/i686 ruby builds
Credits
- Jake Douglas for the Mach-O Snow Leopard support
- Aman Gupta for various bug fixes and other cleanup
- Rob Benson for initial 1.9 support and cleanup
- Paul Barry for force_gc support in
Memprof::Middleware