octo

A lightweight tool for running commands on multiple hosts at the same time.

I found myself getting increasingly frustrated while working with clusters of remote servers. Things like tailing a log on every server, or wanted to grep for something on the entire cluster was very tedious to do.

I tried using pssh (and family) but it was difficult to manage multiple host groups, and small annoyances like leaving commands running on the remote server when pssh was interrupted/killed made them not so much fun to use. So, I created this tool as a quick way for me to be able to manage clusters of similar servers.

Features

• Support for Parallel SSH commands [currently only support public key auth, username:password coming)
• Support for Parallel MySQL commands [working, but needs more testing]
• Profile/Server cluster management
• Option for output to files based on remote server [Future]
• Graceful error handling [Future]
• Support for timeouts [Future]

Requirements

Since octo currently only support public key auth, it relies on your user having his ssh keys properly configured and loaded.

Installation

gem install octo

Configuration

octo looks for a .octorc file in your home directory. It's a simple yaml file and the best way to explain it would be to give an example

---
ssh:
  test:
  - root@test.server.com
  stage:
  - root@server1.stage.server.com
  - root@server2.stage.server.com
  prod:
  - root@server1.server.com
  - root@server2.server.com
  - root@server3.server.com
  prod_db:
  - root@db1.server.com
  - root@db2.server.com
mysql:
  test:
  - test:password@test.server.com/db_test
  staging:
  - stage:password@db1.staging.server.com/db_stage
  prod_central:
  - prod_user:superpassword@central.server.com/db_central
  prod_shards:
  - prod_user:superpassword@shard1.server.com/db_shard_1
  - prod_user:superpassword@shard2.server.com/db_shard_2

This file can also be managed on the command line

$ octo profile <command>

NAME
    profile - Manage profiles

SYNOPSIS
    octo [global options] profile  add profile server
    octo [global options] profile  list [profile]
    octo [global options] profile  rm profile server

DESCRIPTION
    Manage profile that will be used to run commands. Each profile consists of a set of servers.

COMMANDS
    add  - Add a host to a profile
    list - List all profiles
    rm   - Remove a host from a profile

Using

Running a parallel command in octo is pretty simple, once you have your profiles configured

octo run <profile> <command>

For example

$ octo run prod 'tail -n 3 /var/log/messages'
[root@server1.server.com] Oct 22 03:49:01 server8 auditd[2519]: Audit daemon rotating log files
[root@server1.server.com] Oct 22 14:57:09 server8 auditd[2519]: Audit daemon rotating log files
[root@server1.server.com] Oct 23 03:29:01 server8 auditd[2519]: Audit daemon rotating log files
[root@server2.server.com] Oct 22 07:48:01 server11 auditd[2400]: Audit daemon rotating log files
[root@server2.server.com] Oct 22 21:10:01 server11 auditd[2400]: Audit daemon rotating log files
[root@server2.server.com] Oct 23 12:50:01 server11 auditd[2400]: Audit daemon rotating log files
[root@server3.server.com] Oct 20 11:40:11 server12 kernel: possible SYN flooding on port 9000. Sending cookies.
[root@server3.server.com] Oct 21 11:55:49 server12 auditd[2645]: Audit daemon rotating log files
[root@server3.server.com] Oct 22 12:53:01 server12 auditd[2645]: Audit daemon rotating log files

Of course you can also do things like

$ octo run prod 'tail -f /var/log/nginx/error.log | grep search' # piping on the remote boxes

$ octo run prod 'tail -f /var/log/nginx/error.log' | grep string # also works locally, but obviously slower

$ octo run prod 'ruby script.rb' > stdout.log 2> stderr.log # stdout and stderr are preserved

Octo also has experimental support for running queries in parallel

$ octo -m mysql run prod_shards 'select username, email, first_name, last_name from users where username like "nan.%"'

Running query on prod_user:superpassword@shard1.server.com/db_shard_1... 1 results
+-------------+-------------------------+------------+--------------+
| username    | email                   | first_name | last_name    |
+-------------+-------------------------+------------+--------------+
| nan.test    | nan.test@mailinator.com | New        | Test User    |
+-------------+-------------------------+------------+--------------+
Running query on prod_user:superpassword@shard2.server.com/db_shard_2... 3 results
+------------+---------------------------+------------+-----------+
| username   | email                     | first_name | last_name |
+------------+---------------------------+------------+-----------+
| nan.814    | nan.814@mailinator.com    | bla        | bla       |
| nan.leigh  |                           |            |           |
| nan.manley | nan.manley@mailinator.com | bla        | bla       |
+------------+---------------------------+------------+-----------+

Contribute

Feel free to submit issues and I will try my best to get to them. If you are interested, pull requests are highly welcome. :)

License

The MIT License (MIT)

Copyright (c) 2013 Nan Zhong

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.