Class: Riak::Node

Inherits:
Object show all
Includes:
Util::Translation
Defined in:
lib/riak/node.rb,
lib/riak/node/log.rb,
lib/riak/node/console.rb,
lib/riak/node/control.rb,
lib/riak/node/version.rb,
lib/riak/node/defaults.rb,
lib/riak/node/generation.rb,
lib/riak/node/configuration.rb

Overview

A Node encapsulates the generation and management of standalone Riak nodes. It is used by the TestServer to start and manage an in-memory node for supporting integration test suites.

Direct Known Subclasses

TestServer

Defined Under Namespace

Classes: Console, Tuple

Constant Summary collapse

LAGER_LEVELS = 
[
 :debug,
 :info,
 :notice,
 :warning,
 :error,
 :critical,
 :alert,
 :emergency
]
STATS_REGEXP =

Regexp for parsing riak-admin status output. Takes into account the minor bug fixed by https://github.com/basho/riak_kv/pull/134 and multiline output used when lists of things grow long.

/^([^:\n]+)\s:\s((?:.*)(?:\n\s+[^\n]+)*)/
ENV_DEFAULTS =

Settings based on Riak 1.1.

{
  :riak_core => {
    :ring_creation_size => 64
  },
  :riak_kv => {
    :storage_backend => :riak_kv_bitcask_backend,
    :map_js_vm_count => 8,
    :reduce_js_vm_count => 6,
    :hook_js_vm_count => 2,
    :mapper_batch_size => 5,
    :js_max_vm_mem => 8,
    :js_thread_stack => 16,
    :riak_kv_stat => true,
    :legacy_stats => true,
    :vnode_vclocks => true,
    :http_url_encoding => :on,
    :legacy_keylisting => false,
    :mapred_system => :pipe,
    :mapred_2i_pipe => true,
    :listkeys_backpressure => true,
    :add_paths => []
  },
  :riak_search => {
    :enabled => true
  },
  :luwak => {
    :enabled => true
  },
  :merge_index => {
    :buffer_rollover_size => 1048576,
    :max_compact_segments => 20
  },
  :eleveldb => {},
  :bitcask => {},
  :lager => {
    :crash_log_size => 10485760,
    :crash_log_msg_size => 65536,
    :crash_log_date => "$D0",
    :crash_log_count => 5,
    :error_logger_redirect => true
  },
  :riak_sysmon => {
    :process_limit => 30,
    :port_limit => 30,
    :gc_ms_limit => 100,
    :heap_word_limit => 40111000,
    :busy_port => true,
    :busy_dist_port => true
  },
  :sasl => {
    :sasl_error_logger => false
  },
  :riak_control => {
    :enabled => false,
    :auth => :userlist,
    :userlist => {"user" => "pass"},
    :admin => true
  }
}.freeze
VM_DEFAULTS =

Based on Riak 1.1.

{
  "+K" => true,
  "+A" => 64,
  "-smp" => "enable",
  "+W" => "w",
  "-env ERL_MAX_PORTS" => 4096,
  "-env ERL_FULLSWEEP_AFTER" => 0
}.freeze
NODE_DIRECTORIES =

The directories (and accessor methods) that will be created under the generated node.

[:bin, :etc, :log, :data, :ring, :pipe]

Instance Attribute Summary collapse

Class Method Summary collapse

Instance Method Summary collapse

Methods included from Util::Translation

#i18n_scope, #t

Constructor Details

#initialize(configuration = {}) ⇒ Node

Creates the template for a Riak node. To generate the node after initialization, use #create.



28
29
30
31
 
# File 'lib/riak/node.rb', line 28

def initialize(configuration={})
  set_defaults
  configure configuration
end

Instance Attribute Details

#configuration ⇒ Hash (readonly)

Returns the configuration that was passed to the Node when initialized.

Returns:

  • (Hash) —

    the configuration that was passed to the Node when initialized



30
31
32
 
# File 'lib/riak/node/configuration.rb', line 30

def configuration
  @configuration
end

#env ⇒ Hash (readonly)

Returns the contents of the Erlang environment, which will be created into the app.config file.

Returns:

  • (Hash) —

    the contents of the Erlang environment, which will be created into the app.config file.



22
23
24
 
# File 'lib/riak/node/configuration.rb', line 22

def env
  @env
end

#root ⇒ Pathname (readonly)

The root directory of the Riak::Node, where all files are placed after generation.

Returns:

  • (Pathname) —

    the root directory of the node



104
105
106
 
# File 'lib/riak/node/configuration.rb', line 104

def root
  @root
end

#source ⇒ Pathname (readonly)

The source of the Riak installation from where the Riak::Node will be generated. This should point to the directory that contains the ‘riak’ and ‘riak-admin’ scripts.

Returns:

  • (Pathname) —

    the source Riak installation



99
100
101
 
# File 'lib/riak/node/configuration.rb', line 99

def source
  @source
end

#vm ⇒ Hash (readonly)

Returns the command-line switches for the Erlang virtual machine, which will be created into the vm.args file.

Returns:

  • (Hash) —

    the command-line switches for the Erlang virtual machine, which will be created into the vm.args file



26
27
28
 
# File 'lib/riak/node/configuration.rb', line 26

def vm
  @vm
end

Class Method Details

.create(configuration = {}) ⇒ Object

Creates a new Riak node. Unlike #new, this will also generate the node if it does not exist on disk. Equivalent to new followed by #create.

See Also:

  • #new


20
21
22
23
24
 
# File 'lib/riak/node.rb', line 20

def self.create(configuration={})
  new(configuration).tap do |node|
    node.create
  end
end

Instance Method Details

#admin_script ⇒ Pathname

The script for controlling non-lifecycle features of Riak like joining, leaving, status, ringready, etc.

Returns:

  • (Pathname) —

    the path to the administrative script



121
122
123
 
# File 'lib/riak/node/configuration.rb', line 121

def admin_script
  @admin_script ||= root + 'bin' + "#{control_script_name}-admin"
end

#attach ⇒ Riak::Node::Console

Attach to the node’s console via the pipe.

Returns:

See Also:



70
71
72
 
# File 'lib/riak/node/control.rb', line 70

def attach
  Console.open self
end

#base_dir ⇒ Pathname

Returns the location of Riak installation, aka RUNNER_BASE_DIR.

Returns:

  • (Pathname) —

    the location of Riak installation, aka RUNNER_BASE_DIR



11
12
13
 
# File 'lib/riak/node/version.rb', line 11

def base_dir
  @base_dir ||= configure_base_dir
end

#control_script ⇒ Pathname

The script for starting, stopping and pinging the Node.

Returns:

  • (Pathname) —

    the path to the control script



108
109
110
 
# File 'lib/riak/node/configuration.rb', line 108

def control_script
  @control_script ||= root + 'bin' + control_script_name
end

#control_script_name ⇒ String

The name of the ‘riak’ or ‘riaksearch’ control script.

Returns:

  • (String) —

    ‘riak’ or ‘riaksearch’



114
115
116
 
# File 'lib/riak/node/configuration.rb', line 114

def control_script_name
  @control_script_name ||= (source + 'riaksearch').exist? ? 'riaksearch' : 'riak'
end

#cookie ⇒ String

Returns the cookie/shared secret used for connecting a cluster.

Returns:

  • (String) —

    the cookie/shared secret used for connecting a cluster



91
92
93
 
# File 'lib/riak/node/configuration.rb', line 91

def cookie
  vm['-setcookie']
end

#create ⇒ Object

Generates the node.



17
18
19
20
21
22
23
24
25
 
# File 'lib/riak/node/generation.rb', line 17

def create
  unless exist?
    create_directories
    write_scripts
    write_vm_args
    write_app_config
    write_manifest
  end
end

#destroy ⇒ Object

Removes the node from disk and freezes the object.



43
44
45
46
 
# File 'lib/riak/node/generation.rb', line 43

def destroy
  delete
  freeze
end

#drop ⇒ Object

Clears data from known data directories. Stops the node if it is running.



29
30
31
32
33
34
35
36
37
38
39
40
 
# File 'lib/riak/node/generation.rb', line 29

def drop
  was_started = started?
  stop if was_started
  data.children.each do |item|
    if item.directory?
      item.children.each {|c| c.rmtree }
    else
      item.delete
    end
  end
  start if was_started
end

#erlang_sources ⇒ Array<Pathname>

Returns where user Erlang code will be loaded from.

Returns:

  • (Array<Pathname>) —

    where user Erlang code will be loaded from



33
34
35
 
# File 'lib/riak/node/configuration.rb', line 33

def erlang_sources
  env[:riak_kv][:add_paths].map {|p| Pathname.new(p) }
end

#exist? ⇒ Boolean

Does the node exist on disk?

Returns:

  • (Boolean) 


6
7
8
 
# File 'lib/riak/node/generation.rb', line 6

def exist?
  manifest.exist?
end

#expand_log_level(level) ⇒ Object 



23
24
25
26
27
28
29
30
31
32
 
# File 'lib/riak/node/log.rb', line 23

def expand_log_level(level)
  case level
  when Range
    first = LAGER_LEVELS.index(level.begin.to_sym) || 0
    last = LAGER_LEVELS.index(level.end.to_sym) || -1
    LAGER_LEVELS[first..last]
  when Symbol
    level
  end
end

#handoff_port ⇒ Fixnum

Returns The port used for handing off data to other nodes.

Returns:

  • (Fixnum) —

    The port used for handing off data to other nodes.



48
49
50
 
# File 'lib/riak/node/configuration.rb', line 48

def handoff_port
  env[:riak_core][:handoff_port]
end

#http_ip ⇒ String

Returns the interface to which the HTTP API is connected.

Returns:

  • (String) —

    the interface to which the HTTP API is connected



64
65
66
 
# File 'lib/riak/node/configuration.rb', line 64

def http_ip
  env[:riak_core][:http][0][0]
end

#http_port ⇒ Fixnum

Returns The port to which the HTTP API is connected.

Returns:

  • (Fixnum) —

    The port to which the HTTP API is connected.



53
54
55
56
 
# File 'lib/riak/node/configuration.rb', line 53

def http_port
  # We'll only support 0.14 and later, which uses http rather than web_ip/web_port
  env[:riak_core][:http][0][1]
end

#javascript_source ⇒ Pathname

Returns where user Javascript code will be loaded from.

Returns:

  • (Pathname) —

    where user Javascript code will be loaded from



38
39
40
 
# File 'lib/riak/node/configuration.rb', line 38

def javascript_source
  Pathname.new(env[:riak_kv][:js_source_dir])
end

#join(node) ⇒ String

Joins the node to another node to create a cluster.

Returns:

  • (String) —

    the output of the ‘riak-admin join’ command



89
90
91
92
 
# File 'lib/riak/node/control.rb', line 89

def join(node)
  node = node.name if Node === node
  riak_admin 'join', node
end

#js_reload ⇒ Object

Forces the node to restart/reload its JavaScript VMs, effectively reloading any user-provided code.



140
141
142
 
# File 'lib/riak/node/control.rb', line 140

def js_reload
  riak_admin 'js_reload'
end

#kv_backend ⇒ Symbol

Returns the storage backend for Riak KV.

Returns:

  • (Symbol) —

    the storage backend for Riak KV.



79
80
81
 
# File 'lib/riak/node/configuration.rb', line 79

def kv_backend
  env[:riak_kv][:storage_backend]
end

#leave ⇒ String

Removes this node from its current cluster, handing off all data.

Returns:

  • (String) —

    the output of the ‘riak-admin leave’ command



96
97
98
 
# File 'lib/riak/node/control.rb', line 96

def leave
  riak_admin 'leave'
end

#manifest ⇒ Pathname

The “manifest” file where the node configuration will be written.

Returns:

  • (Pathname) —

    the path to the manifest



128
129
130
 
# File 'lib/riak/node/configuration.rb', line 128

def manifest
  root + '.node.yml'
end

#member_status ⇒ Hash

Provides the status of members of the ring.

Returns:

  • (Hash) —

    a collection of stats about ring members



146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
 
# File 'lib/riak/node/control.rb', line 146

def member_status
  output = riak_admin 'member_status'
  result = {}
  if $?.success?
    output.each_line do |line|
      next if line =~ /^(?:[=-]|Status)+/  # Skip the pretty headers
      if line =~ %r{^Valid:(\d+) / Leaving:(\d+) / Exiting:(\d+) / Joining:(\d+) / Down:(\d+)}
        result.merge!(:valid =>   $1.to_i,
                      :leaving => $2.to_i,
                      :exiting => $3.to_i,
                      :joining => $4.to_i,
                      :down =>    $5.to_i)
      else
        result[:members] ||= {}
        status, ring, pending, node = line.split(/\s+/)
        node = $1 if node =~ /^'(.*)'$/
        ring = $1.to_f if ring =~ /(\d+\.\d+)%/
        result[:members][node] = {
          :status => status,
          :ring => ring,
          :pending => (pending == '--') ? 0 : pending.to_i
        }
      end
    end
  end
  result
end

#name ⇒ String

Returns the name of the Riak node as seen by distributed Erlang communication. AKA “-name” in vm.args.

Returns:

  • (String) —

    the name of the Riak node as seen by distributed Erlang communication. AKA “-name” in vm.args.



85
86
87
 
# File 'lib/riak/node/configuration.rb', line 85

def name
  vm['-name']
end

#pb_ip ⇒ String

Returns the interface to which the Protocol Buffers API is connected.

Returns:

  • (String) —

    the interface to which the Protocol Buffers API is connected



69
70
71
 
# File 'lib/riak/node/configuration.rb', line 69

def pb_ip
  env[:riak_kv][:pb_ip]
end

#pb_port ⇒ Fixnum

Returns the port to which the Protocol Buffers API is connected.

Returns:

  • (Fixnum) —

    the port to which the Protocol Buffers API is connected.



59
60
61
 
# File 'lib/riak/node/configuration.rb', line 59

def pb_port
  env[:riak_kv][:pb_port]
end

#peers ⇒ Array<String>

Returns a list of node names that are also members of this node’s cluster, and empty list if the #member_status fails or no other nodes are present.

Returns:

  • (Array<String>) —

    a list of node names that are also members of this node’s cluster, and empty list if the #member_status fails or no other nodes are present



177
178
179
180
 
# File 'lib/riak/node/control.rb', line 177

def peers
  all_nodes = member_status[:members] && member_status[:members].keys.reject {|n| n == name }
  all_nodes || []
end

#ping ⇒ String

Pings the node

Returns:

  • (String) —

    the output of the ‘riak ping’ command



55
56
57
58
59
60
61
62
63
64
 
# File 'lib/riak/node/control.rb', line 55

def ping
  begin
    riak 'ping'
  rescue SystemCallError
    # If the control script doesn't exist or has the wrong
    # permissions, we should still return something sane so we can
    # do the right thing.
    "Node '#{name}' not responding to pings."
  end
end

#read_console_log(*levels) ⇒ Object 



14
15
16
17
18
19
20
21
 
# File 'lib/riak/node/log.rb', line 14

def read_console_log(*levels)
  console_log = log + 'console.log'
  if console_log.exist?
    levels = levels.map { |level| expand_log_level(level) }.compact.flatten
    pattern = /(#{levels.map { |level| "\\[#{level}\\]" }.join("|")})/
    console_log.readlines.grep(pattern)
  end
end

#recreate ⇒ Object

Deletes the node and regenerates it.



11
12
13
14
 
# File 'lib/riak/node/generation.rb', line 11

def recreate
  delete
  create
end

#remove(node) ⇒ String

Forcibly removes a node from the current cluster without invoking handoff.

Returns:

  • (String) —

    the output of the ‘riak-admin remove <node>’ command



104
105
106
107
 
# File 'lib/riak/node/control.rb', line 104

def remove(node)
  node = node.name if Node === node
  riak_admin 'remove', node
end

#restart ⇒ String

Reboots the node

Returns:

  • (String) —

    the output of the ‘riak reboot’ command



43
44
45
 
# File 'lib/riak/node/control.rb', line 43

def restart
  riak 'restart'
end

#ring_size ⇒ Fixnum

Returns the size of the ring, i.e. number of partitions.

Returns:

  • (Fixnum) —

    the size of the ring, i.e. number of partitions



43
44
45
 
# File 'lib/riak/node/configuration.rb', line 43

def ring_size
  env[:riak_core][:ring_creation_size]
end

#ringready? ⇒ true, false

Detects whether the node’s cluster has converged on the ring.

Returns:

  • (true, false) —

    whether the ring is stable



121
122
123
124
 
# File 'lib/riak/node/control.rb', line 121

def ringready?
  output = riak_admin 'ringready'
  output =~ /^TRUE/ || $?.success?
end

#search_backend ⇒ Symbol

Returns the storage backend for Riak Search.

Returns:

  • (Symbol) —

    the storage backend for Riak Search.



74
75
76
 
# File 'lib/riak/node/configuration.rb', line 74

def search_backend
  env[:riak_search][:search_backend]
end

#services ⇒ Array<String>

Lists riak_core applications that have registered as available, e.g. [“riak_kv”, “riak_search”, “riak_pipe”]

Returns:



129
130
131
132
133
134
135
136
 
# File 'lib/riak/node/control.rb', line 129

def services
  output = riak_admin 'services'
  if $?.success?
    output.strip.match(/^\[(.*)\]$/)[1].split(/,/)
  else
    []
  end
end

#start ⇒ String

Starts the node.

Returns:

  • (String) —

    the output of the ‘riak start’ command



27
28
29
30
31
 
# File 'lib/riak/node/control.rb', line 27

def start
  res = riak 'start'
  wait_for_startup
  res
end

#started? ⇒ true, false

Is the node running?

Returns:

  • (true, false) —

    If the node is running



13
14
15
16
 
# File 'lib/riak/node/control.rb', line 13

def started?
  pinged = ping
  pinged.strip =~ /pong/ || pinged.strip !~ /Node '[^']+' not responding to pings/
end

#status ⇒ Hash

Captures the status of the node.

Returns:

  • (Hash) —

    a collection of information about the node



111
112
113
114
115
116
117
 
# File 'lib/riak/node/control.rb', line 111

def status
  output = riak_admin 'status'
  if $?.success?
    result = {}
    Hash[output.scan(STATS_REGEXP)]
  end
end

#stop ⇒ String

Stops the node

Returns:

  • (String) —

    the output of the ‘riak stop’ command



35
36
37
38
39
 
# File 'lib/riak/node/control.rb', line 35

def stop
  res = riak 'stop'
  wait_for_shutdown
  res
end

#stopped? ⇒ true, false

Is the node stopped? (opposite of #started?).

Returns:

  • (true, false) —

    If the node is stopped

See Also:



21
22
23
 
# File 'lib/riak/node/control.rb', line 21

def stopped?
  !started?
end

#version ⇒ String

Returns the version of the Riak node.

Returns:

  • (String) —

    the version of the Riak node



6
7
8
 
# File 'lib/riak/node/version.rb', line 6

def version
  @version ||= configure_version
end

#with_console {|console| ... } ⇒ Object

Execute the block against the Riak node’s console.

Yields:

  • (console) —

    A block of commands to be run against the console

Yield Parameters:



78
79
80
81
82
83
84
85
 
# File 'lib/riak/node/control.rb', line 78

def with_console
  begin
    console = attach
    yield console
  ensure
    console.close if console
  end
end