Sisyphus
Sisyphus provides a really simple way of starting and stopping multiple parallel worker processes that are meant to run repeatedly in an efficient way.
It requires no frameworks, databases or anything. It just does what you want it to - until you tell it to stop doing it.
How it works
The Master takes a job as argument in the initializer, that responds to setup
and perform
.
When sending the start_worker
message to the master, it forks a child process where it hands
the job to a new worker, which then sends the setup
message to the job during initialization.
The purpose of the setup
method is to load and initialize anything necessary to perform the job.
The master then starts the worker, which enters a run loop, that forks yet another child process where
the job receives the perform
message.
Getting started
- Add
gem 'sisyphus'
to your Gemfile and runbundle
- Subclass
Sisyphus::Job
and implement theperform
method - Instantiate the
Sisyphus::Master
, giving it an instance of your job and optionally anoptions
hash. Theoptions
hash supports the following keys::workers
which specifies the number (FixNum
) of workers you need.:logger
which is used to log any exceptions the master or workers encounter. The logger should quack like aLogger
instance from the Ruby stdlib. Exceptions are logged with theLogger::WARN
level.:execution_strategy
which can either beSisyphus::SimpleExecutionStrategy
orSisyphus::ForkingExecutionStrategy
. This is the strategy used by workers when performing the job. The default isSisyphus::ForkingExecutionStrategy
.
- You can start workers by doing one of the following things:
- Send the
start
message to the master, if theoptions
hash was provided. This starts a run loop which monitors workers and restarts them if they encounters uncaught exceptions - Otherwise send the
start_worker
message to the master as many times as the number of workers needed. This doesn't start any run loop, and there is no monitoring of workers.
- Send the
If you need to stop workers, you can do it one at the time by sending
the stop_worker
message to the master or stop them all by sending the
stop_all
message.
When stopping workers, they are allowed to finish what they are doing,
before they stop. Which means you're screwed right now, if the perform
method in your job never returns.
The master registers signal handlers when you send it the start
message and enters a run loop.
The signals the master responds to are:
SIGINT
tells the master to gracefully stop all workers and shut downSIGTTIN
tells the master to spawn a new workerSIGTTOU
tells the master to stop a worker
Things missing
Sisyphus is still very much in its infancy, though the ambition isn't to build a Resque clone, but instead build as small a tool with as few features as possible.
There are, however, still features that are missing:
- Force killing workers
- Daemonization of the master
- The master should have a shutdown immediately signal
- Communication with the master through signals (something like Unicorn)
- Basic logging
- Some sort of error handling
- Some sort of reaping of worker processes
- Documentation
Contributing
Feel free to report issues or fork, fix and submit pull requests.
License
Copyright 2013 Rasmus Bang Grouleff
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.