TourBus

Flexible and scalable website testing tool.

Authors

David Brady – [email protected]
Tim Harper – [email protected]
James Britt – [email protected]
JT Zemp – [email protected]
Joe Tanner – [email protected]

General Info

TourBus is an intelligent website load testing tool. Allows for complicated testing scenarios including filling out forms, following redirects, handling cookies, and following links–all of the things you’d normally associate with a regression suite or integration testing tool. The difference is that TourBus also scales concurrently, and you can perform hundreds of complicated regression tests simultaneously in order to thoroughly load test your website.

It uses Webrat::Mechanize to run the browsing session, so you get the load testing you want, and all the sweetness of Webrat to write your tests in.

Motivation

I started writing TourBus because I needed flexibility and scalability in a website testing tool, and the extant tools all provided one but not the other. Selenium is ultraflexible but limited to the number of browsers you can have open at once, while Apache Bench is powerful and fast but limited to simple tests.

TourBus lets you define complicated paths through your website, then execute those paths concurrently for stress testing.

Example

To see TourBus in action, you need to write scripts. For lack of a better name, these are called Tours.

Example Tour

Make a folder called tours and put a file in it called simple.rb. In it write:

class Simple < Tourist
  def tour_homepage
    visit "http://#{@host}/"
    assert_contain "My Home Page"
  end
end

Files in ./tours should have classes that match their names. E.g. “class BigHairyTest < Tourist” belongs in ./tours/big_hairy_test.rb
Like how Test::Unit finds test_* methods automagically, TourBus will make each tourist run any tour_* methods. setup() and teardown() methods will be executed at the appropriate times.
Two more magic methods: before_tours and after_tours will be run before any tours and after all tours, respectively.

Example TourBus Run

You want to invoke tourbus from the parent directory of the @tours/@ folder.

For example, if you have this project tree …

`-- contact_app
    |-- README.rdoc
    |-- contact_app.rb
    `-- tours
        |-- simple.rb
        `-- tourbus.yml

… then you execute tourbus from the contact_app/ directory.

tourbus -c 2 -n 3 simple

That will run the simple.rb tour file.

It will create 2 concurrent Tour runners, each of which will run all of the methods in Simple three times.

You can specify multiple tours.

tourbus -c 2 -n 3 simple1 simple2 simple3

If you don’t specify a tour, all tours in ./tours will be run.
tourbus –help will give you more information.

You can run tours and filter given tests.

tourbus -c 2 -n 3 simple -t test_login,test_logout

Note that if you specify multiple tours and filter tests, the filtered tests will be run on all tours specified. If you do not specify a tour, the filtered tests will be run on all tours found in the ./tours folder.

Example TourWatch Run

On the webserver, you can type

tourwatch -c 4

To begin running tourwatch. It’s basically a stripped-down version of top with cheesy text graphs. (TourWatch’s development cycles were included in the 2 days for TourBus.)

The -c option is for the total number of cores on the server. The top app will cheerfully report a process as taking 392% CPU if it is using 98% of four cores. This option is only necessary for making the little text graphs scale correctly.
You can choose which processes to watch by passing a csv to -p:
```
tourwatch -p ruby,mongrel
```
Each process name is a partial regexp, so the above would match mongrel AND mongrel_rails, etc.
tourwatch –help will give you more information.

History and Status

TourBus began life as a 2-day throwaway app. It is definitely an app whose development provides many opportunities for open-source contributors to make improvements. It is chock-full of brutal hacks, duplications, oversights, and kludges.

Hacks, Kludges, Known Issues, and Piles of Steaming Poo

If you give a tour a name that is pluralized, it won’t work. This is probably a bug worth fixing. The reason for it is that we take file names and “classify” them, and e.g. “ranking_reports” becomes “RankingReport”, not “RankingReports”. This is an artifact of borrowing from Rails’ activesupport libs and should probably be fixed.
Mechanize 0.8 doesn’t always play well together with TourBus. If you get “connection refused” socket errors, try upgrading to Mechanize 0.9.
JRuby doesn’t play well with Nokogiri. I have set the html_parser to use hpricot, which should work around the issue for now.
There are no specs. Yikes! This is to my eternal shame because I’m sort of a testing freak. Because TourBus WAS a testing tool, I didn’t put tests on it. I haven’t put tests on it yet because I’m not sure how to go about it. Instead of exercising a web app with a test browser, we need to exercise a test browser with… um… a web app? (dbrady notes: Now that we have a contact_app, we could try writing some specs and features to run tourbus against it.)
Web-Sickle is another internal app, written by Tim Harper, that works “well enough”. Until I open-sourced this project, it was a submodule in the app. We wanted to keep TourBus extensions separate from WebSickle itself, so there’s a lot of code in Runner that really belongs in WebSickle.
Documentation is <strike>horrible</strike> merely quite bad.
There’s not much in the way of examples, either. When I removed all the LMP-specific code, all of the examples went with it. Sorry about that. Coming soon.

Feature Requests (How You Can Help!)

I’d like to beef up the example contact app to show more of TourBus than the simplest possible path. Adding in another page or two, and then adding an additional tour or two would make it more apparent to new users that you can do things like run multiple tours at once. Also, having more than one test_ method in simple.rb would let us demonstrate test filtering as well. (Be aware that at present [2009-04-17], webrat still does not play well with Sinatra sessions so there would be complications. dbrady’s fork of webrat combines jferris’ fix with the latest webrat, and will be maintained until main webrat includes the feature. That fork is at github.com/dbrady/webrat)
I’d like to remove WebSickle and replace it with Webrat. There is a webrat branch on the main fork (github.com/dbrady/tourbus) that is 90% complete. Once that’s done we can start massaging the API to be a little more friendly. [done (but now that it is, it needs a refactoring–Tour should probably inherit from Webrat::Mechanize, not delegate to it.)]

Credits

Tim Harper camped at my place for a day fixing bugs in WebSickle as I exercised more and more new bits of it. Thanks, dude.
Lead Media Partners paid me to write TourBus, then let me open source it. How much do they rock? All the way to 11, that’s how much they rock.
James Britt jumped on this and revived it as it was gathering dust. Thanks!
JT Zemp added before_tour, after_tour. Thanks!

License

MIT. See the license file.