grifork

Fast propagative task runner for systems which consist of a lot of servers.

Concept

Grifork runs defined tasks on the system in a way like tree's branching.
Give grifork a list of hosts, then it creates a tree graph internally, and runs tasks in a top-down way.

Grifork has two modes to work:

Standalone mode. This requires grifork program only on the root server in the tree graph: i.e. the server which invokes tasks.
Grifork mode. On the other hand, this requires grifork program to work on every node in the graph.

Take a look at each mechanism.

Standalone Mode

The image below illustrates a 3-depth tree of 13 nodes including root node.

standalone mode

We are running a task to copy a file tree to every host.
1st stage is completed: to copy them to nodes in 1st generation.

Now at 2nd stage, root node logins each of its children by ssh and kicks rsync program there, in order to copy the file tree from 1st to 2nd generation.

NOTE:

Max concurrency of running task in standalone mode is the number of nodes at the generation which holds max. This is the last genaration or the genartion before the last.

Grifork Mode

The image below is similar to previous situation except that this is in grifork mode.

grifork mode

In this mode, parent nodes in the graph invokes grifork command on every child node via ssh, giving the graph tree which descends from each child node.

System Requirements

Ruby v2
ssh, rsync

Installation

git clone https://github.com/key-amb/grifork.git
cd grifork
bundle

Usage

edit Griforkfile
./bin/grifork [--[f]ile path/to/Griforkfile] [-n|--dry-run]

Griforkfile

Griforkfile is DSL file for grifork which configures and defines the tasks to be executed by grifork.

Here is a small example:

branches 4
log file: 'grifork.log'
hosts ['web1.internal', 'web2.internal', 'db1.internal', 'db2.internal', ...]

local do
  rsync '/path/to/myapp/'
end

remote do
  rsync_remote '/path/to/myapp/'
end

If you run grifork with this Griforkfile, it just syncs /path/to/myapp/ in localhost to target hosts by rsync command.

See example directory for more examples of Griforkfile.

And refer to Grifork::DSL as API document of Griforkfile.

Authors

IKEDA Kiyoshi [email protected]

License

The MIT License (MIT)