This document contains a summary of a presentation about using RabbitMQ for messaging. It discusses RabbitMQ's features like multi-protocol support, polyglot clients, and large users. It then describes challenges like distributing an application across servers. RabbitMQ solutions to these include federation to replicate across servers, sharding queues for scaling, and federating queues for load balancing consumers. The presentation argues RabbitMQ provides tools to ingest, distribute, and scale messaging applications globally.
28. Some users of RabbitMQ
⢠Instagram!
⢠Indeed.com
29. Some users of RabbitMQ
⢠Instagram!
⢠Indeed.com!
⢠Telefonica
30. Some users of RabbitMQ
⢠Instagram!
⢠Indeed.com!
⢠Telefonica!
⢠Mercado Libre
31. Some users of RabbitMQ
⢠Instagram!
⢠Indeed.com!
⢠Telefonica!
⢠Mercado Libre!
⢠NHS
32. Some users of RabbitMQ
⢠Instagram!
⢠Indeed.com!
⢠Telefonica!
⢠Mercado Libre!
⢠NHS!
⢠Mozilla
33. The New York Times on RabbitMQ
This architecture - Fabrik - has dozens of RabbitMQ instances
spread across 6 AWS zones in Oregon and Dublin.
Upon launch today, the system autoscaled to ~500,000 users.
Connection times remained flat at ~200ms.
http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2014-January/032943.html
52. Possible issues
⢠Remote server is offline
⢠Prevent unbounded local buffers
53. Possible issues
⢠Remote server is offline
⢠Prevent unbounded local buffers
⢠Prevent message loss
54. Possible issues
⢠Remote server is offline
⢠Prevent unbounded local buffers
⢠Prevent message loss
⢠Prevent unnecessary message replication
55. Possible issues
⢠Remote server is offline
⢠Prevent unbounded local buffers
⢠Prevent message loss
⢠Prevent unnecessary message replication
⢠No need for those messages on remote server
56. Possible issues
⢠Remote server is offline
⢠Prevent unbounded local buffers
⢠Prevent message loss
⢠Prevent unnecessary message replication
⢠No need for those messages on remote server
⢠Messages that became stale
61. RabbitMQ Federation
⢠Supports replication across different administrative domains
⢠Supports mix of Erlang and RabbitMQ versions
62. RabbitMQ Federation
⢠Supports replication across different administrative domains
⢠Supports mix of Erlang and RabbitMQ versions
⢠Supports Network Partitions
63. RabbitMQ Federation
⢠Supports replication across different administrative domains
⢠Supports mix of Erlang and RabbitMQ versions
⢠Supports Network Partitions
⢠Specificity - not everything has to be federated
86. Prevent Unbound Buffers
Îť = mean arrival time
Îź = mean service rate
if Îť > Îź what happens?
https://www.rabbitmq.com/blog/2014/01/23/preventing-unbounded-buffers-with-rabbitmq/
87. Prevent Unbound Buffers
Îť = mean arrival time
Îź = mean service rate
if Îť > Îź what happens?
Queue length goes to infinity over time.
https://www.rabbitmq.com/blog/2014/01/23/preventing-unbounded-buffers-with-rabbitmq/
91. The Problem
⢠Queues contents live in the node where the Queue was declared
92. The Problem
⢠Queues contents live in the node where the Queue was declared
⢠A cluster can access the queue from every connected node
93. The Problem
⢠Queues contents live in the node where the Queue was declared
⢠A cluster can access the queue from every connected node
⢠Queues are an Erlang process (tied to one core)
94. The Problem
⢠Queues contents live in the node where the Queue was declared
⢠A cluster can access the queue from every connected node
⢠Queues are an Erlang process (tied to one core)
⢠Adding more nodes doesnât really help
103. Sharded Queues
⢠Declare Queues with name: nodename.queuename.index
⢠Bind the queues to a consistent hash exchange
104. Sharded Queues
⢠Declare Queues with name: nodename.queuename.index
⢠Bind the queues to a partitioner exchange
⢠Transparent to the consumer (virtual queue name)
111. With RabbitMQ we can
⢠Ingest data using various protocols: AMQP, MQTT and STOMP
112. With RabbitMQ we can
⢠Ingest data using various protocols: AMQP, MQTT and STOMP
⢠Distribute that data globally using Federation
113. With RabbitMQ we can
⢠Ingest data using various protocols: AMQP, MQTT and STOMP
⢠Distribute that data globally using Federation
⢠Scale up using Sharding
114. With RabbitMQ we can
⢠Ingest data using various protocols: AMQP, MQTT and STOMP
⢠Distribute that data globally using Federation
⢠Scale up using Sharding
⢠Load balance consumers with Federated Queues