TTK - TestToolKit

TestToolKit, aka TTK, is designed to ease the test stage of the development of your projects.

Testing is as important as design and implementation of a project. But it’s very cumbersome to write tests and tester scripts. That’s why a project such as TTK can help you. It is written in Ruby, which is a script language one hundred percents dynamic and object oriented. It’s inherited from Smalltalk, Python and Perl. TestToolKit allows two ways of test writing: a configuration file way (using YAML: an XML plain text format) and a class extension way where you can extend an already made class to specialize it for your own project case.

TTK’s philosophy follows the principle that you can always extend already made test strategy to specialize it for your own case. Initially, TTK comes with a few test strategies and provides specially abstract test strategies. Thus, TTK will become bigger only by contributions from its users who publish their test strategies. At the end, it will provide enough test strategies that almost everybody will find the test strategy he needs.

TTK comes with a set of classic test strategies, statistics computation, a test suite manager, a pluggable loader/dumper architecture, and a distributed mode.

Roadmap

  • If you want to use TTK to test your command line based program, see the TTK::Strategies::Cmd strategy documentation.

  • If you want to use TTK to wrap your unit test suite, see the TTK::Strategies::RUnit strategy documentation.

  • If you want to try writing your own strategy, see the TTK::Strategies module documentation.

  • If you want to try writing your own loader, see the TTK::Loaders module documentation.

  • If you want to try writing your own dumper, see the TTK::Dumpers module documentation.

  • If you want to try writing your own filter, see the TTK::Filters module documentation.

  • If you want to try writing your own filter, see the TTK::Weights module documentation.

  • If you want to use TTK as a library and to write your own driver, see the TTK module documentation.

Usage

TTK is invoked from the command line using:

% ttk <options> [files...]

Strategies described in the files are instantiated and run. The files format must follow the one the chosen loader understand. By default, TTK uses the YAML loader. Use the -P option to use another loader. If no file are specified, TTK will read on the standard input. You can also use the -i option to change that.

You can have a description of all TTK’s options using this command:

% ttk -h

TTK can show you a list of all dynamic strategies available, this way:

% ttk --strategy-list

If you want to have a description of all the available attributes of a given strategy, use this command:

% ttk -h <the-given-strategy>

If you want to see the list of all the available loader, type this command:

% ttk -L

For dumpers, do that:

% ttk -P

For filter:

% ttk -F

And, for weight classes:

% ttk -W

To set the level of severity of the TTK’s logger use the -d option. Without argument, it will show you all the available levels:

% ttk -d

Otherwise, give a correct level as argument to the -d option:

% ttk -d debug

By default, the severity level is fatal.

Examples

Some examples are available in the test/examples directory. These examples are written for the default loader of TTK (the YAML loader). YAML is very easy to read and write. If you want to learn it, see the YAML cookbook at yaml4r.sourceforge.net/cookbook/.

A detailed description of these examples is available in the test/examples/README file.

Stable and unstable parts

TTK is still under development. Thus, there are some part of it which are still unstable. Of course, even some stable part are not free from bugs. Perfection doesn’t exist!

  • Stable parts:

    • The loader, dumper, strategy and logger architecture.

    • The YAML loader.

    • The YAML and the XML dumper.

    • All strategies, expect those mentioned in the _unstable strategies_ section, are considered stable.

  • Unstable parts:

    • The filter architecture is very unstable and not yet well developed. This part will be fully functional in the next release. We discourage its usage.

    • The distributed mode already provides some basic features but it is not considered as really usable. We discourage its usage.

    • The timeout features provided by the abstract strategy Strategy tend to be stable but it may block some times. You can use it, since it works most of the time.

    • The pre,post_assertion features provided by the abstract strategy Strategy doesn’t work yet.

    • Some old YAML loader are still in the repository, for backward compatibility reasons, but we discourage their usage.

Unstable strategies

RUnit

It works fine most a time, but doesn’t not handle yet the abort features. If you try to abort it, using the timeout features (or whatever) for instance, some processes may stayed alive. You can use it, otherwise.

Test

It works fine, but it will be redesign in the next release, because it needs some upgrades. You can use it carefully.

JUnit

It has been stable a long time ago, but hasn’t been adapted to the recent strategy architecture changes yet. We discourage its usage.

Distribute

It is still under development and doesn’t give correct results yet. We discourage its usage.

DistDuplicate

It needs to be adapted to the recent strategy architecture changes. We discourage its usage.

Repository tree description

NEWS

What’s new in each release of TTK.

AUTHORS

Present all authors and what they have done for TTK.

bin

Contains all drivers provided by TTK.

ChangeLog

Keep track of the changes of TTK for all revisions.

COPYING

Describe the license.

DESCRIPTION

A short and a long description of TTK (used for rubygem).

doc

Documentation directory.

doc/html

RDoc output directory.

lib

Contains the library.

misc

Some useful miscellaneous scripts when developing TTK.

NORM

Everybody who write code for TTK must follow the rules written in this file.

Rakefile

For rubygem, doc generation and more generaly packaging operations.

README

This file.

ruby_ex

Some useful Ruby’s extensions. The end user is not supposed to look at this directory. Those files will be bundle in one or several separated packages in the futur.

template

Some ERB templates to ease writting new strategies.

template/README

Explains how to use these templates.

test

Contains TTK’s tests.

TODO

What maintainers have to do.

VERSION

The current version of TTK.

REVISION

The SubVersion repository revision.

DATE

The date of the SubVersion repository last update.

Bugs report and features request

If you want to send us a bug report or a feature request, have a look at the TODO file.

Known bugs

An exhaustive list of known bugs is available at www.feydakins.org/projects/ttk/tracker/base_view.

License

TTK is copyright © 2004-2005 TTK’s maintainers. It is free software, and may be redistributed under the terms specified in the COPYING file of the TTK distribution. TTK’s maintainers refers to the people mentioned in the AUTHORS file.

– TTK’s maintainers