DeepTest
DeepTest enables tests to run in parallel using multiple processes. Processes may spawned locally to take advantage of multiple processors on a single machine or distributed across many machines to take advantage of distributed processing.
Usage
In your Rakefile:
require "rubygems"
require "deep_test/rake_tasks"
# sample DeepTest task
DeepTest::TestTask.new "task_name" do |t|
t.number_of_workers = 2 # optional, defaults to 2
t.timeout_in_seconds = 30 # optional, defaults to 30
t.server_port = 6969 # optional, defaults to 6969
t.pattern = "test/**/*_test.rb"
t.libs << "test" # may be necessary for Rails >= 2.1.x
end
# sample SpecTask using DeepTest
Spec::Rake::SpecTask.new(:deep_spec) do |t|
t.spec_files = FileList['spec/**/*_spec.rb']
t.deep_test :number_of_workers => 2, # optional, defaults to 2
:timeout_in_seconds => 30, # optional, defaults to 30
:server_port => 6969 # optional, defaults to 6969
end
Specifying a Listener to be Notified of Events
In my_worker_listener.rb
class MyWorkerListener
def before_sync
end
def before_starting_workers
end
def starting(worker)
end
def starting_work(worker, work_unit)
end
def finished_work(worker, work_unit, result)
end
end
In your Rakefile
DeepTest::TestTask.new "task_name" do |t|
...
t.worker_listener = "MyWorkerListener"
end
An instance of MyWorkerListener will be created for each worker that is started, and will receive events from that worker. However, you must ensure that the MyWorkerListener class is loaded at the time that the test files are loaded, so it is available to be instantiated. You can specify multiple listener classes by separating them with a comma. For more information about when events are triggered, see the documentation at DeepTest::NullWorkerListener.
Setting Up A New Database For Each Worker
By default, DeepTest will reinitialize ActiveRecord connections if ActiveRecord is loaded when the workers are started. This means all workers are running against the same database. You may want each worker to use a database decidicated to it. To facilitate this, DeepTest provides a worker listener to help you. If you’re using Rails with Mysql, simply configure DeepTest as follows in your Rakefile:
DeepTest::TestTask.new "task_name" do |t|
...
t.worker_listener = "DeepTest::Database::MysqlSetupListener"
end
Before spawning workers, DeepTest will dump the schema of the database for the current Rails environment (usually test). As each worker starts up, the listener will create a database dedicated to that worker and load the schema into it. The database will be dropped when the worker process exits.
If you’re using Mysql but not using Rails, you’ll have to create a subclass of MysqlSetupListener and override master_database_config and dump_file_name, as the default implementations of these methods are Rails specific.
If you’re using a database other than Mysql, read the documentation for DeepTest::Database::SetupListener and create a new subclass for your database type. If you do this, please consider contributing your subclass back to the project so that it can be included in later releases of DeepTest.
Distributing Tests Across Multiple Machines
In addition to running your tests in parallel, DeepTest can also distribute them across multiple machines. It does this by first mirroring the local working copy that launched the tests on each machine that will be running tests. Then workers are launched on each of the machines and consume tests in the same fashion as when DeepTest is running locally.
Requirements
Before you can distribute tests, you must ensure that all the machines involved (including the machine with the local working copy) have rsync installed for mirroring working copies. You must also have either SSH or an RSync daemon exposing your local working copy running on the local development machine. For more information about rsync, visit the rsync webpage. Currently only passwordless access is supported, so you must either setup your RSync daemon to be accessible without a password or enable passwordless SSH access from the test machines to the local development machine. DeepTest must also be installed as a gem on each test machine and available either as a gem or in your project on the local machine.
Starting a Test Server
On each test machine, execute the following:
> deep_test test_server
This will launch a server providing mirroring and worker services. By default, 2 workers will be launched for each set of tests run. If you wish to change the number of workers, simply specify the –number_of_workers option. For information about what options are available, use -h. The test_server will print out the uri it is bound to, druby://<hostname>:4022 by default.
Starting a Master Test Server
On a single machine (probably one of the test machines), execute this:
> deep_test master_test_server <uris of test servers>
The master_test_server will also print it’s binding uri, druby://<hostname>:4021 by default. It will also provide a webserver bound to port 4020 that provides a status page summarizing the state of all the test servers.
Overriding the DRb uri for the Test Server or Master Test Server
The DRb url that the Test Server or Master Test Server should bind to can be specified when the process is started. Use the –uri option followed by the DRb url that you would like DRb to bind to and return to clients. By default, DRb does not provide any capability to bind to all addresses but return a specific IP or hostname as part of the DRb url for clients. DeepTest provides a special DRb protocol to enable this, drubyall. If you wish your test server to listen on all addresses (which you probably do), start the server with a command like this (where deeptest1.local is the hostname of the machine):
> deep_test test_server --uri drubyall://deeptest1.local/
The server will report that it is listening on druby://deeptest1.local as the public DRb url, but it will actually accept connections on all addresses to server requests.
Configuring Your Project
If you’re using rsync over ssh, create a DeepTest test task similar to that below in your Rakefile.
DeepTest::TestTask.new "deep_test_distributed" do |t|
t.pattern = "test/**/*_test.rb" # whatever is appropriate for your project
t.distributed_server = <drb uri of master_test_server>
t.sync_options = {
:source => <absolute path of project root on machine>,
:username => "username"
}
end
The :source entry in sync_options can be easily calculated based on the value of _FILE_ when defining the task. :username will be used by rsync to ssh back to the local machine and mirror the working copy.
If you have an rsync daemon running in your local machine, configure the rake task as follows.
DeepTest::TestTask.new "deep_test_distributed" do |t|
t.pattern = "test/**/*_test.rb" # whatever is appropriate for your project
t.distributed_server = <drb uri of master_test_server>
t.sync_options = {
:source => <name of rsync module from daemon configuration>,
:daemon => true,
:username => "username"
}
end
Username is optional in both cases. You’ll need to either setup passwordless ssh access or run an rsync daemon that doesn’t require passwords.
There may be other options you’d like to pass to rsync in your particular scenario. This can be done by adding an :rsync_options entry to sync_options. For example, if you’re working on a Rails project you’ll probably want to at least have something like this:
DeepTest::TestTask.new "deep_test_distributed" do |t|
...
excludes = %w[.svn tmp/** log/**]
t. = {
...
:rsync_options => excludes.map {|s| "'--exclude=#{s}'"}.join(' ')
}
end
That way you can avoid spending any time mirroring tmp and log files that don’t have any effect on the tests. If you are running distributed tests against a database, consult the section above about creating a new database for each worker to see how to configure DeepTest for your project.
Any number of projects can be run using the same test servers, as long as they’re all using the same version of Ruby and DeepTest.
Contributors
-
anonymous z
-
Alex Chaffee
-
Scott Taylor