:title: README
= flare-tools
https://github.com/gree/flare/wiki
Flare-tools is a collection of command line tools to maintain a flare cluster.
* flare-stats
* flare-stats is a command used for aquring statistics of nodes in a flare cluster.
* flare-admin
* flare-admin is a command used for maintaining your flare clusters.
== DESCRIPTION:
Management Tools for Flare
Authors:: Kiyoshi Ikehara <[email protected]>
Copyright:: Copyright (C) GREE, Inc. 2011.
License:: MIT-style
== SYNOPSIS:
=== flare-stats
$ flare-stats [options] [hostname:port] ...
=== flare-admin
$ flare-admin [subcommand] [options] [arguments]
== REQUIREMENTS:
* Flare >= 1.0.14
== INSTALL:
* Install Flare
Please see the web pages about Flare[https://github.com/gree/flare/wiki].
* Install flare-tools
# gem install flare-tools
== AUTHORS:
* Kiyoshi Ikehara
== USAGE:
=== SPECIFYING YOUR INDEX SERVER
All the commands in flare-tools have common ways to specify an index server.
If your flare index server doesn't listen to the defalut port (12120), you have to specify
its hostname (or IP address) and port of your index node specifically.
This is done by setting options, --index-server and --index-server-port.
(other tools can also accept these two options.)
$ flare-stats --index-server=your.index.server --index-server-port=13130
You can also set or FLARE_INDEX_SERVER and FLARE_INDEX_SERVER_PORT environment variables.
These variables dosen't override the index node specified by options.
$ export FLARE_INDEX_SERVER=your.index.server
$ export FLARE_INDEX_SERVER_PORT=13130
$ flare-stats
or
$ export FLARE_INDEX_SERVER=your.index.server:13130
$ flare-stats
=== flare-stats
Flare-stats is a dedicated command line tool used for aquiring statistics of flare nodes.
This tool shows a list of nodes in a flare cluster and their summarized information.
$ flare-stats --index-server=flare1.example.com
hostname:port state role partition balance items conn behind hit size uptime version
flare1.example.com:12121 active master 0 1 10000 111 0 100 10 12d 1.0.10
flare2.example.com:12121 active slave 0 1 10000 111 0 100 10 12d 1.0.10
flare3.example.com:12121 active master 1 1 10001 111 0 100 10 12d 1.0.10
flare4.example.com:12121 active slave 1 1 10001 111 0 100 10 12d 1.0.10
Flare-stats is a short-cut version of stats subcommand of flare-admin.
Please see the stats subcommand section of flare-admin for further detail.
==== flare-stats's options
Usage: flare-stats [options]
-h, --help shows this message
-d, --debug enables debug mode
-w, --warn turns on warnings
-i, --index-server=[HOSTNAME] index server hostname(default:127.0.0.1)
-p, --index-server-port=[PORT] index server port(default:12120)
-q, --qps show qps
--wait=[SECOND] wait time for repeat(second)
-c, --count=[REPEATTIME] repeat count
--delimiter=[CHAR] delimiter
=== flare-admin
Flare-admin consists of a battery of subcommands used for maintaining a flare cluster.
You must specify the hostname and port number of the index node by telling them as options
or environment variables.
==== MASTER subcommand
'master' subcommand is used for creating a new partition in a cluster.
You must specify a hostname, a port number, an intitial balance and a partition number
for each partition in "hostname:port:balance:partition" format. The specified node will
be changed from 'proxy' state to 'master' state and given the specified balance after
the construction.
If the node isn't a proxy or the partition already exists, this subcommand fails.
$ flare-admin master newmaster1:12131:1:1
(confirmation)
(construction)
This subcommand asks you before committing the change to make sure that you know the
result. If you're convinced of the effect of this subcommand and doesn't want to be
bothered, you can specify --force option to skip the confirmation steps.
$ flare-admin master --force newmaster:12131:1:1
==== RECONSTRUCT subcommand
'reconstruct' subcommand flushes the existing data and reconstruct new database from
another node. This subcommand can be applied to both slaves and masters.
If you reconstruct a master, the node will be turned down and one of slave nodes will
take over master's role.
$ flare-admin reconstruct node1:12121
(confirmation)
(construction)
If you want to reconstruct all nodes in a cluster, you can specify --all option.
$ flare-admin reconstruct --all
(confirmation)
(construction)
...
This subcommand asks you before committing the change to make sure that you know the
result. If you're convinced of the effect of this subcommand and doesn't want to be
bothered, you can specify --force option to skip the confirmation steps.
$ flare-admin reconstruct --force node1:12121
(construction)
==== PING subcommand
'ping' subcommand checks if the nodes specified as arguments are alive or not by
connecting each node and sending a ping request. If the response is positive (OK),
flare-admin exits with a status code 0 (otherwise 1). You can check the soundness
of a node by this subcommand in scripts.
$ flare-admin ping
alive
If you want to wait for a node to start up, you should specify --wait option and
the command repeatedly sends ping requests until the node is available.
$ flare-admin ping --wait
==== DEPLOY subcommand (experimental)
'deploy' subcommand generates node settings with control scripts.
This subcommand is mainly used for experiments.
==== SLAVE subcommand
'slave' subcommand makes a node a slave state and construct the slave database.
Before changing the state, this confirms the current state and the resulted state.
After that, duplication of data starts and the progress is put on your console.
$ flare-admin slave newslave1:12132:1:0 newslave2:12132:1:0
(confirmation)
(construction)
==== BALANCE subcommand
'balance' subcommand sets balance parameters of nodes.
$ flare-admin balance node1:12131:3 node2:12131:2
(confirmation)
(setting balance)
==== LIST subcommand
'list' subcommand shows a node list in a cluster.
$ flare-admin list
node partition role state balance
server1:12121 - proxy active 0
This subcommand just refers the cluster information the index node in a cluster.
If 'stats' subcommand fails, please check the node status by this subcommand.
==== THREADS subcommand (experimental)
'threads' subcommand shows the thread status of the specified node.
$ flare-admin threads node1:12121
==== STATS subcommand
'stats' subcommand shows the status of nodes in a cluster.
$ flare-admin stats
hostname:port state role partition balance items conn behind hit size uptime version
server1:12121 active master 0 1 30 11 0 70 0 1d 1.0.14
server2:12121 active slave 0 1 35 19 0 68 0 1h 1.0.14
This subcommand also display qps (query per second) statistics of each node.
$ flare-admin stats --qps --count=100
...
hostname:port state role partition balance items conn behind hit size uptime version qps qps-r qps-w
server1:12121 active master 0 1 30 11 0 70 0 1d 1.0.14 100.3 50.1 50.2
server2:12121 active slave 0 1 35 19 0 68 0 1h 1.0.14 52.1 0 52.1
...
==== INDEX subcommand (experimental)
'index' subcommand generates the index file of a cluster.
$ flare-admin index
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="4">
<node_map class_id='0' tracking_level='0' version='0'> ... </node_map>
==== DOWN subcommand
'down' subcommand turn nodes down.
$ flare-admin down node1:12121 node2:12121
==== flare-admin's options
Usage: flare-admin [subcommand] [options] [arguments]
-h, --help show this message
-d, --debug enable debug mode
-w, --warn turn on warnings
-n, --dry-run dry run
-i, --index-server=[HOSTNAME] index server hostname(default:)
-p, --index-server-port=[PORT] index server port(default:)
--log-file=[LOGFILE] output log to LOGFILE
subcommands:
[dumpkey] dump key from nodes.
Usage: flare-admin dumpkey [hostname:port] ...
-o, --output=[FILE] output to file
-f, --format=[FORMAT] output format [csv]
-p, --partition=[NUMBER] partition number
-s, --partition-size=[SIZE] partition size
--bwlimit=[BANDWIDTH] bandwidth limit (bps)
--all dump form all partitions
[master] construct a partition with a proxy node for master role.
Usage: flare-admin master [hostname:port:balance:partition] ...
--force commit changes without confirmation
--retry=[COUNT] specify retry count (default:10)
--activate change node's state from ready to active
--without-clean don't clear datastore before construction
[balance] set the balance values of nodes.
Usage: flare-admin balance [hostname:port:balance] ...
--force commit changes without confirmation
[down] turn down nodes and move them to proxy state.
Usage: flare-admin down [hostname:port] ...
--force commit changes without confirmation
[restore] restore data to nodes. (experimental)
Usage: flare-admin restore [hostname:port]
-i, --input=[FILE] input from file
-f, --format=[FORMAT] input format [tch]
--bwlimit=[BANDWIDTH] bandwidth limit (bps)
--include=[PATTERN] include pattern
--prefix-include=[STRING] prefix string
--exclude=[PATTERN] exclude pattern
--print-keys enables key dump to console
[stats] show the statistics of a flare cluster.
Usage: flare-admin stats [hostname:port] ...
-q, --qps show qps
-w, --wait=[SECOND] specify wait time for repeat(second)
-c, --count=[REPEATTIME] specify repeat count
-d, --delimiter=[CHAR] spedify delimiter
[verify] verify the cluster. (experimental)
Usage: flare-admin verify
--key-hash-algorithm=[TYPE] key hash algorithm
--use-test-data store test data
--debug use debug mode
--64bit (experimental) 64bit mode
--verbose use verbose mode
--meta use meta command
--quiet use quiet mode
[remove] remove a node. (experimental)
Usage: flare-admin remove [hostname:port] ...
--force commit changes without confirmation
--retry=COUNT retry count(default:0)
[dump] dump data from nodes. (experimental)
Usage: flare-admin dump [hostname:port] ...
-o, --output=[FILE] output to file
-f, --format=[FORMAT] specify output format [default,csv,tch]
--bwlimit=[BANDWIDTH] specify bandwidth limit (bps)
--all dump from all master nodes
--raw raw dump mode (for debugging)
[threads] show the list of threads in a flare cluster.
Usage: flare-admin threads [hostname:port]
[slave] construct slaves from proxy nodes.
Usage: flare-admin slave [hostname:port:balance:partition] ...
--force commit changes without confirmation
--retry=[COUNT] specify retry count(default:10)
--without-clean don't clear datastore before construction
--clean [obsolete] now slave command clear datastore before construction by default.
[list] show the list of nodes in a flare cluster.
Usage: flare-admin list
--numeric-hosts show numerical host addresses
[activate] activate
Usage: flare-admin down [hostname:port] ...
--force commit changes without confirmation
[ping] ping
Usage: flare-admin ping [hostname:port] ...
--wait wait for OK responses from nodes
[index] print the index XML document from a cluster information.
Usage: flare-admin index
--output=[FILE] output index to a file
--increment=[VERSION] increment node_map_version
[reconstruct] reconstruct the database of nodes by copying.
Usage: flare-admin reconstruct [hostname:port] ...
--force commit changes without confirmation
--unsafe reconstruct a node safely
--safe [obsolete] now reconstruct a node safely by default
--retry=COUNT specify retry count (default:10)
--all reconstruct all nodes
== THANKS:
* Masaki FUJIMOTO (the author of Flare)
* kgws[http://d.hatena.ne.jp/kgws] (the author of old flare-tools)
= flare-tools
https://github.com/gree/flare/wiki
Flare-tools is a collection of command line tools to maintain a flare cluster.
* flare-stats
* flare-stats is a command used for aquring statistics of nodes in a flare cluster.
* flare-admin
* flare-admin is a command used for maintaining your flare clusters.
== DESCRIPTION:
Management Tools for Flare
Authors:: Kiyoshi Ikehara <[email protected]>
Copyright:: Copyright (C) GREE, Inc. 2011.
License:: MIT-style
== SYNOPSIS:
=== flare-stats
$ flare-stats [options] [hostname:port] ...
=== flare-admin
$ flare-admin [subcommand] [options] [arguments]
== REQUIREMENTS:
* Flare >= 1.0.14
== INSTALL:
* Install Flare
Please see the web pages about Flare[https://github.com/gree/flare/wiki].
* Install flare-tools
# gem install flare-tools
== AUTHORS:
* Kiyoshi Ikehara
== USAGE:
=== SPECIFYING YOUR INDEX SERVER
All the commands in flare-tools have common ways to specify an index server.
If your flare index server doesn't listen to the defalut port (12120), you have to specify
its hostname (or IP address) and port of your index node specifically.
This is done by setting options, --index-server and --index-server-port.
(other tools can also accept these two options.)
$ flare-stats --index-server=your.index.server --index-server-port=13130
You can also set or FLARE_INDEX_SERVER and FLARE_INDEX_SERVER_PORT environment variables.
These variables dosen't override the index node specified by options.
$ export FLARE_INDEX_SERVER=your.index.server
$ export FLARE_INDEX_SERVER_PORT=13130
$ flare-stats
or
$ export FLARE_INDEX_SERVER=your.index.server:13130
$ flare-stats
=== flare-stats
Flare-stats is a dedicated command line tool used for aquiring statistics of flare nodes.
This tool shows a list of nodes in a flare cluster and their summarized information.
$ flare-stats --index-server=flare1.example.com
hostname:port state role partition balance items conn behind hit size uptime version
flare1.example.com:12121 active master 0 1 10000 111 0 100 10 12d 1.0.10
flare2.example.com:12121 active slave 0 1 10000 111 0 100 10 12d 1.0.10
flare3.example.com:12121 active master 1 1 10001 111 0 100 10 12d 1.0.10
flare4.example.com:12121 active slave 1 1 10001 111 0 100 10 12d 1.0.10
Flare-stats is a short-cut version of stats subcommand of flare-admin.
Please see the stats subcommand section of flare-admin for further detail.
==== flare-stats's options
Usage: flare-stats [options]
-h, --help shows this message
-d, --debug enables debug mode
-w, --warn turns on warnings
-i, --index-server=[HOSTNAME] index server hostname(default:127.0.0.1)
-p, --index-server-port=[PORT] index server port(default:12120)
-q, --qps show qps
--wait=[SECOND] wait time for repeat(second)
-c, --count=[REPEATTIME] repeat count
--delimiter=[CHAR] delimiter
=== flare-admin
Flare-admin consists of a battery of subcommands used for maintaining a flare cluster.
You must specify the hostname and port number of the index node by telling them as options
or environment variables.
==== MASTER subcommand
'master' subcommand is used for creating a new partition in a cluster.
You must specify a hostname, a port number, an intitial balance and a partition number
for each partition in "hostname:port:balance:partition" format. The specified node will
be changed from 'proxy' state to 'master' state and given the specified balance after
the construction.
If the node isn't a proxy or the partition already exists, this subcommand fails.
$ flare-admin master newmaster1:12131:1:1
(confirmation)
(construction)
This subcommand asks you before committing the change to make sure that you know the
result. If you're convinced of the effect of this subcommand and doesn't want to be
bothered, you can specify --force option to skip the confirmation steps.
$ flare-admin master --force newmaster:12131:1:1
==== RECONSTRUCT subcommand
'reconstruct' subcommand flushes the existing data and reconstruct new database from
another node. This subcommand can be applied to both slaves and masters.
If you reconstruct a master, the node will be turned down and one of slave nodes will
take over master's role.
$ flare-admin reconstruct node1:12121
(confirmation)
(construction)
If you want to reconstruct all nodes in a cluster, you can specify --all option.
$ flare-admin reconstruct --all
(confirmation)
(construction)
...
This subcommand asks you before committing the change to make sure that you know the
result. If you're convinced of the effect of this subcommand and doesn't want to be
bothered, you can specify --force option to skip the confirmation steps.
$ flare-admin reconstruct --force node1:12121
(construction)
==== PING subcommand
'ping' subcommand checks if the nodes specified as arguments are alive or not by
connecting each node and sending a ping request. If the response is positive (OK),
flare-admin exits with a status code 0 (otherwise 1). You can check the soundness
of a node by this subcommand in scripts.
$ flare-admin ping
alive
If you want to wait for a node to start up, you should specify --wait option and
the command repeatedly sends ping requests until the node is available.
$ flare-admin ping --wait
==== DEPLOY subcommand (experimental)
'deploy' subcommand generates node settings with control scripts.
This subcommand is mainly used for experiments.
==== SLAVE subcommand
'slave' subcommand makes a node a slave state and construct the slave database.
Before changing the state, this confirms the current state and the resulted state.
After that, duplication of data starts and the progress is put on your console.
$ flare-admin slave newslave1:12132:1:0 newslave2:12132:1:0
(confirmation)
(construction)
==== BALANCE subcommand
'balance' subcommand sets balance parameters of nodes.
$ flare-admin balance node1:12131:3 node2:12131:2
(confirmation)
(setting balance)
==== LIST subcommand
'list' subcommand shows a node list in a cluster.
$ flare-admin list
node partition role state balance
server1:12121 - proxy active 0
This subcommand just refers the cluster information the index node in a cluster.
If 'stats' subcommand fails, please check the node status by this subcommand.
==== THREADS subcommand (experimental)
'threads' subcommand shows the thread status of the specified node.
$ flare-admin threads node1:12121
==== STATS subcommand
'stats' subcommand shows the status of nodes in a cluster.
$ flare-admin stats
hostname:port state role partition balance items conn behind hit size uptime version
server1:12121 active master 0 1 30 11 0 70 0 1d 1.0.14
server2:12121 active slave 0 1 35 19 0 68 0 1h 1.0.14
This subcommand also display qps (query per second) statistics of each node.
$ flare-admin stats --qps --count=100
...
hostname:port state role partition balance items conn behind hit size uptime version qps qps-r qps-w
server1:12121 active master 0 1 30 11 0 70 0 1d 1.0.14 100.3 50.1 50.2
server2:12121 active slave 0 1 35 19 0 68 0 1h 1.0.14 52.1 0 52.1
...
==== INDEX subcommand (experimental)
'index' subcommand generates the index file of a cluster.
$ flare-admin index
<?xml version="1.0" encoding="UTF-8" standalone="yes" ?>
<!DOCTYPE boost_serialization>
<boost_serialization signature="serialization::archive" version="4">
<node_map class_id='0' tracking_level='0' version='0'> ... </node_map>
==== DOWN subcommand
'down' subcommand turn nodes down.
$ flare-admin down node1:12121 node2:12121
==== flare-admin's options
Usage: flare-admin [subcommand] [options] [arguments]
-h, --help show this message
-d, --debug enable debug mode
-w, --warn turn on warnings
-n, --dry-run dry run
-i, --index-server=[HOSTNAME] index server hostname(default:)
-p, --index-server-port=[PORT] index server port(default:)
--log-file=[LOGFILE] output log to LOGFILE
subcommands:
[dumpkey] dump key from nodes.
Usage: flare-admin dumpkey [hostname:port] ...
-o, --output=[FILE] output to file
-f, --format=[FORMAT] output format [csv]
-p, --partition=[NUMBER] partition number
-s, --partition-size=[SIZE] partition size
--bwlimit=[BANDWIDTH] bandwidth limit (bps)
--all dump form all partitions
[master] construct a partition with a proxy node for master role.
Usage: flare-admin master [hostname:port:balance:partition] ...
--force commit changes without confirmation
--retry=[COUNT] specify retry count (default:10)
--activate change node's state from ready to active
--without-clean don't clear datastore before construction
[balance] set the balance values of nodes.
Usage: flare-admin balance [hostname:port:balance] ...
--force commit changes without confirmation
[down] turn down nodes and move them to proxy state.
Usage: flare-admin down [hostname:port] ...
--force commit changes without confirmation
[restore] restore data to nodes. (experimental)
Usage: flare-admin restore [hostname:port]
-i, --input=[FILE] input from file
-f, --format=[FORMAT] input format [tch]
--bwlimit=[BANDWIDTH] bandwidth limit (bps)
--include=[PATTERN] include pattern
--prefix-include=[STRING] prefix string
--exclude=[PATTERN] exclude pattern
--print-keys enables key dump to console
[stats] show the statistics of a flare cluster.
Usage: flare-admin stats [hostname:port] ...
-q, --qps show qps
-w, --wait=[SECOND] specify wait time for repeat(second)
-c, --count=[REPEATTIME] specify repeat count
-d, --delimiter=[CHAR] spedify delimiter
[verify] verify the cluster. (experimental)
Usage: flare-admin verify
--key-hash-algorithm=[TYPE] key hash algorithm
--use-test-data store test data
--debug use debug mode
--64bit (experimental) 64bit mode
--verbose use verbose mode
--meta use meta command
--quiet use quiet mode
[remove] remove a node. (experimental)
Usage: flare-admin remove [hostname:port] ...
--force commit changes without confirmation
--retry=COUNT retry count(default:0)
[dump] dump data from nodes. (experimental)
Usage: flare-admin dump [hostname:port] ...
-o, --output=[FILE] output to file
-f, --format=[FORMAT] specify output format [default,csv,tch]
--bwlimit=[BANDWIDTH] specify bandwidth limit (bps)
--all dump from all master nodes
--raw raw dump mode (for debugging)
[threads] show the list of threads in a flare cluster.
Usage: flare-admin threads [hostname:port]
[slave] construct slaves from proxy nodes.
Usage: flare-admin slave [hostname:port:balance:partition] ...
--force commit changes without confirmation
--retry=[COUNT] specify retry count(default:10)
--without-clean don't clear datastore before construction
--clean [obsolete] now slave command clear datastore before construction by default.
[list] show the list of nodes in a flare cluster.
Usage: flare-admin list
--numeric-hosts show numerical host addresses
[activate] activate
Usage: flare-admin down [hostname:port] ...
--force commit changes without confirmation
[ping] ping
Usage: flare-admin ping [hostname:port] ...
--wait wait for OK responses from nodes
[index] print the index XML document from a cluster information.
Usage: flare-admin index
--output=[FILE] output index to a file
--increment=[VERSION] increment node_map_version
[reconstruct] reconstruct the database of nodes by copying.
Usage: flare-admin reconstruct [hostname:port] ...
--force commit changes without confirmation
--unsafe reconstruct a node safely
--safe [obsolete] now reconstruct a node safely by default
--retry=COUNT specify retry count (default:10)
--all reconstruct all nodes
== THANKS:
* Masaki FUJIMOTO (the author of Flare)
* kgws[http://d.hatena.ne.jp/kgws] (the author of old flare-tools)