FiverrCopy

TODO: Write a gem description

Installation

Add this line to your application's Gemfile:

gem 'fiverr_copy'

And then execute:

$ bundle

Or install it yourself as:

$ gem install fiverr_copy

Usage

FiverrCopy automates the process of distributing files to multiple servers in a sync column method. You'll need to write a "recipe" and boot up the server with that recipe.

What it actually does is to create a linked list of servers in which ever client is connected to the next client starting with the server. the server then distributes the file to the first client in queue, and this client passes every chunk of data it gets to the next client and so on.

the main benefit is obviously getting a more efficient way to distribute files to multiple machines than the normal scp, nfs mount, etc.

Recipes

To create a FiverrCopy recipe, just create a ruby file that looks something like this

FiverrCopy::Server.recipe = FiverrCopy::Recipe.new("My Awesome Script", "a void description") do |recipe|
  recipe.filter = "*.jpg"
  recipe.clients = ["server1", "server2"]
  recipe.username = "serveruser"
  recipe.port = 1555
  recipe.chunk_size = 1024
  recipe.manual = true
end
  • filter: this is the file selector that the FiverrCopy::Server will grab and tar before distribution, you can either specify a wildcard, a specific file or join both by spaces.
  • clients: a list of ips or hosts to distribute the file to. protip: use host names that all the servers in the process are familiar with (server and clients).
  • username: FiverrCopy uses Net::SSH to fire up some events, it counts on you that your current user can login to any of the machines using a username only and an ssh key.
  • port: which port to run on.
  • chunk_size: buffer size when sending file chunks.
  • manuak: FiverrCopy will try to fire up clients when it runs, if you want to run them by yourself - toggle this option and the FiverrCopy::Server will assume the clients are ready.

Binary

FiverrCopy contains only one binary which is fiverr_copy:

Usage: fiverr_copy [options]
    -c, --client                     Run as client
    -s, --server                     Run as server
    -p, --port PORT                  Select a port. (default: 1234)
    -r, --recipe FILE                Specify a recipe file
    -n, --nexthop IP/HOST            Specify the next hop for distribution
    -f, --filename FILENAME          Specify filename
    -w, --chunk CHUNK_SIZE           specify chunk size
    -h, --help                       Display this screen

Examples

running a server:

fiverr_copy --server --recipe /etc/distribute_mysql_dump.rb

the server will only need the --server and the --recipe options.

running a client manually:

fiverr_copy --client --port 1555 --filename mysql_dump_compressed_1333364100.tgz --chunk 1024

this will run a FiverrCopy::Client instance on port 1555, reading 1024 bytes at a time from stream to the mysql_dump_compressed_* file destination.

TODOs

  1. Testing, how.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Added some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request