testify

Testify is a test framework framework. Think “Rack for testing.”

Overview

Like Rack, Testify is typically going to be run as a stack of applications that implement a call method. In typical usage, the stack will be assembled by a Runner and consist of a Framework and optionally some middleware, but this isn’t a requirement.

The source and detailed documentation can be found at github.com/djspinmonkey/testify and rdoc.info/projects/djspinmonkey/testify, respectively.

Testify Applications

A Testify Application is any object that implements a call method accepting a single argument. The one argument will be a hash (canonically called env), and the return value should be an array of TestResult objects.

The `env` Hash

The env hash should be an instance of Hash with one of the following keys defined:

:files: an array of filenames containing the tests
:path: a path containing the files containing the tests

In addition, all of the following keys should be defined:

:testify_version: An array of integers representing this version of Testify
:hooks: A hash of callbacks - see below.

Given that Testify is still at a fairly early stage of development, it is quite possible that more required keys will be added later.

Middleware

Middleware will generally either modify the env hash before the framework gets it (eg, to execute only a subset of tests), or modify the TestResult objects on the way back out (eg, to colorize the result text). If you want to respond to a particular event as it happens (eg, send a Growl notification when a test fails), it is probably best to do this by adding a hook rather than looking at the returned TestResult objects, since call() will not return until all the tests have finished running.

Frameworks

A Framework object is responsible for actually running the tests and generating TestResult objects. Frameworks are required to respect all of the hooks described below. Generally, Frameworks should inherit from Testify::Framework::Base and should have at least one alias in order to benefit from all of Testify’s built-in functionality, but neither of these is a requirement. (See the Aliasable module in the Classy gem for a more thorough description of aliases. github.com/djspinmonkey/classy)

At least initially, most Framework classes will likely be adaptors for existing test frameworks (RSpec, Test::Unit, etc).

Hooks

env is a hash containing arrays of callable objects (usually Procs, but anything responding to call will work) which will be called in specific circumstances. The following keys should supported:

:before_all: Will be called before running any tests. An array of Test objects will be passed in. A middleware app might use this hook instead of simply running immediately if it needed access to individual tests instead of just files, or if it needed to run after all other middleware. For example, if some other middleware later in the stack were excluding some subset of tests from running, you might need to be sure you were operating only on the tests that would actually be run.
:after_all: Will be called after running all tests. An array of TestResult objects will be passed in. Middleware setting this hook should probably just work with the array of TestResults returned from calling the next Testify app on the stack instead, unless it needs to run before the results pass through any other middleware. Note that there is no guarantee another hook may not be added prior to yours, however.
:before_each: Will be called before running each individual test. A single Test object will be passed in.
:after_each: Will be called after running each individual test. A single TestResult object will be passed in.
:after_status: Points to a hash containing symbols as the keys, and arrays of callable objects to be called whenever a TestResult is generated with the status corresponding to that symbol. The TestResult will be passed in.

If using the values provided by Testify.env_defaults, values for the first four keys will be initialized to empty arrays, and :after_status will be initialized to an empty hash.

Like the env hash, this is highly subject to change at this point.

TODO

Write adaptors for common test frameworks (rspec and minitest first, probably).
Write an autotest-like Runner
Write some useful middleware (growl notifications, colorized output, etc)
World domination!

Note on Patches/Pull Requests

Fork the project.
Make your feature addition or bug fix.
Add tests for it. This is important so I don’t break it in a future version unintentionally.
Commit, do not mess with rakefile, version, or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
Send me a pull request. Bonus points for topic branches.