Top Level Namespace

Defined Under Namespace

Modules: Adhd, AdhdRESTServer, ProxyToServer, YAML Classes: Array, ContentDoc, ContentShard, Hash, Node, NodeDB, Object, ShardRange, ShardRangeDB

Instance Method Summary

• #assign_nodes_to_shards(node_list, shard_range_list, replication_factor) ⇒ Object
• #parse_config(file) ⇒ Object

  require ‘socket’.

Instance Method Details

#assign_nodes_to_shards(node_list, shard_range_list, replication_factor) ⇒ Object

# File 'lib/adhd/node_manager.rb', line 216

def assign_nodes_to_shards(node_list, shard_range_list, replication_factor)
  # This is an automatic way to allocate shards to nodes that just
  # arrive in the networks, as well as re-allocate shards if nodes
  # become unavailable or leave the network.

  # NOTE: How to build skynet (Part III)
  #
  #       The invarient we try to impost on the list of nodes part of a shard
  #       is that there should be at least replication_factor available nodes
  #       in it. At the same time we try to keep the list stable over nodes
  #       joining and leaving. To achieve this we hash in sequence the name of
  #       each node with the name of the shard. We sort this list by hash, and
  #       choose the first n nodes such that at least replication_factor nodes
  #       are available. Then we chose the first available node as the master
  #       for that shard.

  shard_range_list.each do |shard_range|
    # Sort all nodes using consistent hashing
    sorted_nodes = node_list.sort_by {|node| MD5.new("#{node.name}||#{shard_range.shard_db_name}").to_s}
    avail = 0
    master = nil
    shard_node_list = []
    sorted_nodes.each do |node|
      shard_node_list << node
      if node.status == "RUNNING"
        master = node if !master # Chose the first available to be the master
        avail += 1
        break if avail == replication_factor # We have enough available nodes
      end
    end

    # Now put this list in the shard_range and save it
    # but only if there were changes
    new_master = master.name if master
    new_node_list = shard_node_list.map {|node| node.name}

    if !(new_master == shard_range.master_node) or !(new_node_list == shard_range.node_list)
      shard_range.master_node = master.name if master
      shard_range.node_list = shard_node_list.map {|node| node.name}
      shard_range.save
    end
  end

end

#parse_config(file) ⇒ Object

require ‘socket’

# File 'bin/adhd', line 12

def parse_config(file)
  @config = YAML.load_openstruct(File.read(file))
end