Class: DiscordStore::ChannelShard

Inherits:
Object
  • Object
show all
Defined in:
lib/discord_store/channel_shard.rb

Overview

Routes a logical stream to one of several channels.

Discord's harshest limit is five messages per five seconds per channel. The global token budget is ten times that, so a single-channel store leaves ninety percent of its allowance unused, and the only way to spend it is to write to more channels.

That buys throughput and costs ordering, and it is worth being precise about which: message IDs are only strictly ordered within a channel, because Discord issues them from several workers. So this library makes the same promise Kafka does — a channel is a partition, records within a partition are totally ordered, and records in different partitions are not ordered with respect to each other at all. A stream always lands in the same partition, so per-stream order is total.

If you need one order across everything, use one channel and accept the ceiling. Correctness first; the ceiling is a documented number, whereas a silently reordered write-ahead log is a bug you find much later.

Instance Attribute Summary collapse

Instance Method Summary collapse

Constructor Details

#initialize(channel_ids) ⇒ ChannelShard

Returns a new instance of ChannelShard.

Parameters:

  • channel_ids (Array<String>)

Raises:



30
31
32
33
34
35
 
# File 'lib/discord_store/channel_shard.rb', line 30

def initialize(channel_ids)
  ids = Array(channel_ids).compact.map(&:to_s).uniq
  raise ConfigurationError, "at least one channel id is required" if ids.empty?

  @channel_ids = ids.freeze
end

Instance Attribute Details

#channel_ids ⇒ Array<String> (readonly)

Returns:

  • (Array<String>) 


26
27
28
 
# File 'lib/discord_store/channel_shard.rb', line 26

def channel_ids
  @channel_ids
end

Instance Method Details

#for(key) ⇒ String

The channel that owns key.

Rendezvous hashing rather than modulo, so that adding a channel moves only the fraction of streams that must move (1/n) instead of almost all of them. A stream that changes partition loses its ordering guarantee across the move, so the cheaper the reshuffle the better.

Parameters:

  • key (#to_s) —

    the stream name

Returns:

  • (String) —

    a channel id



46
47
48
49
50
 
# File 'lib/discord_store/channel_shard.rb', line 46

def for(key)
  return @channel_ids.first if @channel_ids.one?

  @channel_ids.max_by { |channel_id| weight(key, channel_id) }
end

#plan(keys) ⇒ Hash{String => String}

Every stream-to-channel assignment for a known set of streams, for inspection and for tests that assert stability across reconfiguration.

Parameters:

  • keys (Array<String>)

Returns:

  • (Hash{String => String}) 


63
64
65
 
# File 'lib/discord_store/channel_shard.rb', line 63

def plan(keys)
  keys.to_h { |key| [key.to_s, self.for(key)] }
end

#size ⇒ Integer

Returns:

  • (Integer) 


53
 
# File 'lib/discord_store/channel_shard.rb', line 53

def size = @channel_ids.size

#totally_ordered? ⇒ Boolean

Returns whether this shard preserves a single total order.

Returns:

  • (Boolean) —

    whether this shard preserves a single total order



56
 
# File 'lib/discord_store/channel_shard.rb', line 56

def totally_ordered? = @channel_ids.one?