Durability and related matters

This Documentation Has Moved to rubyamqp.info

amqp gem documentation guides are now hosted on rubyamqp.info.

About this guide

This guide covers queue, exchange and message durability, as well as other topics related to durability, for example, durability in cluster environment.

This work is licensed under a Creative Commons Attribution 3.0 Unported License (including images & stylesheets). The source is available on Github.

Covered versions

This guide covers Ruby amqp gem v0.8.0 and later.

Entity durability and message persistence

Durability of exchanges

AMQP separates concept of durability of entities (queues, exchanges) from messages persistence. Exchanges can be durable or transient. Durable exchanges survive broker restart, transient exchanges don’t (they have to be redeclared when broker comes back online). Not all scenarios and use cases mandate exchanges to be durable.

Durability of queues

Durable queues are persisted to disk and thus survive broker restarts. Queues that are not durable are called transient. Not all scenarios and use cases mandate queues to be durable.

Note that only durable queues can be bound to durable exchanges. This guarantees that it is possible to restore bindings on broker restart.

Durability of a queue does not make messages that are routed to that queue durable. If broker is taken down and then brought back up, durable queue will be re-declared during broker startup, however, only persistent messages will be recovered.

Persistence of messages

The concept of messages persistence is separate: messages may be published as persistent. That makes AMQP broker persist them to disk. If the server is restarted, the system ensures that received persistent messages are not lost. Simply publishing message to a durable exchange or the fact that queue(s) they are routed to is durable doesn’t make messages persistent: it all depends on persistence mode of the messages itself. Publishing messages as persistent affects performance (just like with data stores, durability comes at a certain cost in performance and vise versa). Pass :persistent => true to AMQP::Exchange#publish to publish your message as persistent.

Transactions

TBD

Publisher confirms

Because transactions carry certain (for some applications, significant) overhead, RabbitMQ introduced an extension to AMQP 0.9.1 called publisher confirms (documentation).

amqp gem implements support for this extension, but it is not loaded by default when you require “amqp”. To load it, use

require "amqp/extensions/rabbitmq"

and then define a callback for publisher confirms using AMQP::Channel#confirm:

# enable publisher acknowledgements for this channel
channel.confirm_select

# define a callback that will be executed when message is acknowledged
channel.on_ack do |basic_ack|
  puts "Received an acknowledgement: delivery_tag = #{basic_ack.delivery_tag}, multiple = #{basic_ack.multiple}"
end

# define a callback that will be executed when message is rejected using basic.nack (a RabbitMQ-specific extension)
channel.on_nack do |basic_nack|
  puts "Received a nack: delivery_tag = #{basic_nack.delivery_tag}, multiple = #{basic_nack.multiple}"
end

Note that the same callback is used for all messages published via all exchanges on the given channel.

Clustering

To achieve degree of durability critical applications need, it’s necessary but not enough to use durable queues, exchanges and persistent messages. You need to use a cluster of brokers because otherwise, a single hardware problem may bring broker down completely.

See Clustering guide for in-depth discussion of this topic.

Tell us what you think!

Please take a moment and tell us what you think about this guide on Twitter or Ruby AMQP mailing list: what was unclear? what wasn’t covered? maybe you don’t like guide style or grammar and spelling are incorrect? Readers feedback is key to making documentation better.

If mailing list communication is not an option for you for some reason, you can contact guides author directly