Ruby pub/sub API Specification Draft

Draft State

This draft is under discussion and will be implemented by iodine starting with the 0.8.x versions.

Purpose

This document details a Rack specification extension for publish/subscribe (pub/sub) modules that can extend WebSocket / EventSource Rack servers.

The purpose of this specification is:

To keep separation of concerns by avoiding inter-process-communication logic (IPC) in the application code base.

This is important since IPC is often implemented using pipes / sockets, which could introduce network and IO concerns into the application code.

To specify a common publish/subscribe (pub/sub) API for servers and pub/sub modules, allowing applications to easily switch between conforming implementations and servers.

Since pub/sub design is idiomatic to WebSocket and EventSource approaches, as well as other reactive programming techniques, it is better if applications aren't locked in to a specific server / implementation.
To provide common considerations and guidelines for pub/sub implementors to consider when implementing their pub/sub modules.

Some concerns are common for pub/sub implementors, such as integrating third party message brokers (Redis, RabbitMQ, Cassandra)

Pub/Sub Instance Methods

Conforming pub/sub implementations MUST implement the following pub/sub instance methods:

pubsub? MUST return the pub/sub API version as an integer number. Currently, set to 0 (development version).
subscribe(to, is_pattern = false) { |from, message| optional_block } where:
- to is a named channel (or pattern).
  
  The implementation MAY support pattern matching for named channels (to). The pattern matching algorithm used, if any, SHOULD be documented.
  
  If the implementation does NOT support pattern matching and is_pattern is truthful, the implementation MUST raise and exception.
  
  to SHOULD be a String, but implementations MAY support Symbols as aliases to Strings (in which case :my_channel is the same 'my_channel').
- block is optional and accepts (if provided) two arguments (from which is equal to to and message which contains the data's content).
  
  block (if provided) MUST be called when a publication was received by the named channel.
  
  If no block is provided:
  - If the pub/sub instance is an extended WebSocket / EventSource (SSE) client object (see the WebSocket / EventSource extension draft) data should be directly sent to the client.
  - If the pub/sub isn't linked with a client / connection, an exception MUST be raised.
If a subscription to to already exists for the same pub/sub instance, it should be replaced by the new subscription (the old subscription should be canceled / unsubscribed).

If the pub/sub instance is an extended WebSocket / EventSource (SSE) client object (see the WebSocket / EventSource extension draft), the subscription MUST be closed automatically when the connection is closed (when on_close is called).

Otherwise, the subscription MUST be closed automatically when the pub/sub object goes out of scope and is garbage collected (if this ever happens).

The subscribe method MUST return nil on a known failure (i.e., when the connection is already closed), or any truthful value on success.
unsubscribe(from, is_pattern = false) should cancel a subscription to the from named channel / pattern.
publish(to, message, engine = nil) where:
- to is a named channel, same as detailed in subscribe.
  
  Implementations MAY support pattern based publishing. This SHOULD be documented as well as how patterns are detected (as opposed to named channels). Note that pattern publishing isn't supported by common backends (such as Redis) and introduces complex privacy and security concerns.
- message a String containing the data to be published.
  
  If message is NOT a String, the implementation MAY convert the data silently to JSON. Otherwise, the implementation MUST raise an exception.
  
  message encoding (binary / UTZ-8) MAY be altered during publication, but any change MUST result in valid encoding.
- engine routes the publish method to the specified pub/sub Engine (see later on). If none is specified, the default engine should be used. If false is specified, the message MUST be forwarded to all subscribed clients on the same process.
The publish method MUST return true if a publication was scheduled (not necessarily performed). If it's already known that the publication would fail, the method should return false.

An implementation MUST call the relevant PubSubEngine's publish method after performing any internal book keeping logic. If engine is nil, the default PubSubEngine should be called. If engine is false, the implementation MUST forward the published message to the actual clients (if any).

A global alias for this method (allowing it to be accessed from outside active connections) MAY be defined as Rack::PubSub.publish.

Integrating a Pub/Sub module into a WebSocket / EventSource (SSE) `client` object

A conforming pub/sub module MUST be designed so that it can be integrated into WebSocket / EventSource (SSE) client objects be include-ing their class.

This MUST result in a behavior where subscriptions are destroyed / canceled once the client object state changes to "closed" - i.e., either when the on_close callback is called, or the first time the method client.open? returns false.

The idiomatic way to add a pub/sub module to a client's class is:

client.class.prepend MyPubSubModule unless client.pubsub?

The pub/sub module MUST expect and support this behavior.

Note: the use of prepend (rather than include) is chosen so that it's possible to override the client's instance method of pubsub?.

Connecting to External Backends (pub/sub Engines)

It is common for scaling applications to require an external message broker backend such as Redis, RabbitMQ, etc'. The following requirements set a common interface for such "engine" implementation and integration.

Pub/sub implementations MUST implement the following module / class methods in one of their public classes / modules (iodine implements these under Iodine::PubSub):

attach(engine) where engine is a PubSubEngine object, as described in this specification.

When a pub/sub engine is attached, the implementation MUST inform the engine of any existing or future subscriptions.

The implementation MUST call the engine's subscribe callback for each existing (and future) subscription.

The implementation MUST allow multiple "engines" to be attached when multiple calls to attach are made.
detach(engine) where engine is a PubSubEngine object as described in this specification.

The implementation MUST remove the engine from the attached engine list. The opposite of attach.
default=(engine) sets a default pub/sub engine, where engine is a PubSubEngine object as described in this specification.

Implementations MUST forward any publish method calls to the default pub/sub engine, unless an engine is specified in arguments passes to the publish method.
default returns the current default pub/sub engine, where the engine is a PubSubEngine object as described in this specification.
reset(engine) where engine is a PubSubEngine object as described in this specification.

Implementations MUST behave as if the engine was newly registered and (re)inform the engine of any existing subscriptions by calling engine's subscribe callback for each existing subscription.

A PubSubEngine instance object MUST implement the following methods:

subscribe(to, is_pattern = false) this method informs the engine that a subscription to the specified channel / pattern exists for the calling the process. It MUST ONLY be called once for all existing and future subscriptions to that channel within the process.

The method MUST return true if a subscription was scheduled (or performed) or false if the subscription is known to fail.

This method will be called by the pub/sub implementation (for each registered engine). The engine MAY assume that the method would never be called directly by an application / client.

This method MUST NOT raise an exception.
unsubscribe(from, is_pattern = false) this method informs an engine that there are no more subscriptions to the named channel / pattern for the calling process.

The method's semantics are similar to subscribe only is performs the opposite action.

This method MUST NOT raise an exception.

This method will be called by the pub/sub implementation (for each registered engine). The engine MAY assume that the method would never be called directly by an application / client.
publish(channel, message) where both channel is either a Symbol or a String (both being equivalent) and message MUST be a String.

This method will be called by the server when a message is published using the engine.

This method will be called by the pub/sub implementation (for each registered engine).

The engine MUST assume that the method MAY be called directly by an application / client.

In order for a PubSubEngine instance object to publish messages to all subscribed clients on a particular process, it SHOULD call the implementation's global publish method with the engine set to false.

i.e., if the implementation's global publish method is in a class called Iodine: