1. Adding 1.21 Gigawatts to
Applications with RabbitMQ
James Titcumb
PHP UK Conference 2015

2. James Titcumb
www.jamestitcumb.com
www.protected.co.uk
www.phphants.co.uk
www.phpsouthcoast.co.uk
@asgrim
Who is this guy?

3. What is message
queueing?

4. Separation of Concerns

5. Scaling with Rabbit
RabbitMQApplication
Background processing

6. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing

7. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing

8. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Background processing

9. Scaling with Rabbit
RabbitMQApplication
Background processing
Background processing
Background processing
Background processing
Background processing

10. Real world uses?

11. Why RabbitMQ?

12. (on precise64, other OSs may vary)
Installing RabbitMQ

13. ● add apt repo
○ deb http://www.rabbitmq.com/debian/ testing main
● add signing key
○ http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
● apt-get update
● apt-get install rabbitmq-server
● rabbitmq-plugins enable rabbitmq_management
● sudo service rabbitmq-server restart
Using Apt

14. http://localhost:15672/

15. http://localhost:15672/

16. http://localhost:15672/

17. http://localhost:15672/

18. http://localhost:15672/

19. Basic Message Queuing

20. https://github.com/asgrim/rmq-slides
Follow along...

21. Objective: Basic Queuing
Producer Consumer
test_queue
1 2 3 4 5

22. composer.json
1 {
2 "require": {
3 "videlalvaro/php-amqplib": "2.*"
4 }
5 }
then composer install

23. Please wait, connecting...
1 use PhpAmqpLibConnectionAMQPConnection;
2
3 $connection = new AMQPConnection(
4 'localhost', 5672, 'guest', 'guest', '/');
5
6 $channel = $connection->channel();

24. basic/producer.php
1 use PhpAmqpLibMessageAMQPMessage;
2
3 $channel->queue_declare(
4 'test_queue', false, true, false, false);
5
6 $message = new AMQPMessage('my test message');
7 $channel->basic_publish($message, '', 'test_queue');

25. basic/consumer.php
1 $channel->basic_consume(
2 'test_queue', '', false, true, false, false,
3 function(AMQPMessage $message) {
4 echo $message->body . "n";
5 });
6
7 while (count($channel->callbacks))
8 $channel->wait();

26. What to expect...

27. Exchanges: Fanout

28. Objective: Fanout Exchange
test_exchange
amq.KfgPZ3PE
amq.cK5Cp3FC
Consumer
Consumer
Producer
1
1
2
2
3
3
4
4
5
5

29. 1 use PhpAmqpLibMessageAMQPMessage;
2
3 $channel->exchange_declare(
4 'test_exchange', 'fanout', false, false, false);
5
6 $message = new AMQPMessage('test msg #' . $id);
7 $channel->basic_publish($message, 'test_exchange');
fanout/producer.php

30. fanout/consumer.php
1 $q = $channel->queue_declare(
2 '', false, false, false, true);
3 $queue_name = $q[0];
4
5 $channel->exchange_declare(
6 'test_exchange', 'fanout', false, false, false);
7 $channel->queue_bind($queue_name, 'test_exchange');

31. What to expect...

32. A word on Temporary Queues
test_exchangeProducer Messages
go nowhere

33. Exchanges: Direct

34. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
BK = orange, banana,
apple
Consumer

35. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= ORANGE
BK = orange, banana,
apple
Consumer

36. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= BANANA
BK = orange, banana,
apple
Consumer

37. Objective: Direct Exchange
test_direct
BK = apple
BK = banana, apple
Consumer
Consumer
Producer
MESSAGE
ROUTING KEY
= APPLE
BK = orange, banana,
apple
Consumer

38. direct/producer.php
1 $channel->exchange_declare(
2 'test_direct', 'fanout', false, false, false);
3
4 $messageContent = 'test msg, key=' . $routingKey;
5 $message = new AMQPMessage($messageContent);
6 $channel->basic_publish(
7 $message, 'test_direct', $routingKey);

39. direct/consumer.php
1 $q = $channel->queue_declare('', false, false, false, true);
2 $queue_name = $q[0];
3 $channel->exchange_declare(
4 'test_direct', 'direct', false, false, false);
5
6 // Bind for each routing key we want (BINDING KEY)
7 $channel->queue_bind($queue_name, 'test_direct', 'apple');
8 $channel->queue_bind($queue_name, 'test_direct', 'orange');
9 $channel->queue_bind($queue_name, 'test_direct', 'banana');

40. What to expect...

41. Exchanges: Topic

42. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

43. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
RED.VEGETABLE
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

44. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.VEGETABLE
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

45. Objective:
Topic Exchange
test_topic
BK = *.vegetable
BK = #
Consumer
Consumer
Producer
GREEN.GRASS.LONG
BK = green.#
Consumer
BK = *.grass.* / *.*.long
Consumer

46. Real World Example

48. Fetch message
Logging Sequence
ApplicationBrowser Log Server
HTTP request
JSON via AMQP
Error!
HTTP response
RabbitMQ

49. Flexibility!

50. Parallel Processing

51. Message
Acknowledgement

52. RPC

53. TTL

54. DLX

55. http://tryrabbitmq.com/

56. Infrastructure

57. Problem: SPOF

58. Solution 1: Clustering

59. Clustering
RabbitMQ
Node 1
RabbitMQ
Node 3
RabbitMQ
Node 2
RabbitMQ
Node 4
RabbitMQ
Node 5
RabbitMQ
Node 6
Load Balance / Floating IP / Low TTL DNS etc.

60. Everything Replicates
(except queues…)

61. Disk / RAM

62. Configuration...

63. /etc/rabbitmq/rabbitmq.config
[
{rabbit, [
{loopback_users, []},
{vm_memory_high_watermark, 0.8}
]}
].

64. /etc/rabbitmq/rabbitmq.config
[{{{[{{[{{}}{][[[{[{{}[[}{[[{}[][}{}}}{}}{{,},]{
[[{rabbit, [{{}[[}{,,{}[][}{[][][{}{{{{}}}}[[}{{
{{}}{loopback_users, []},[][][]{}{}{}<}{[}[][][}
[{{[{vm_memory_high_watermark, 0.8}]]{}{[[[]]{}]
{{]}[{[{{}[[}{]]{}[][,{}[][}{[][][{}.[]}{]][][]}
]...{}[][,]{.}[][}{}[[[{}{][]}{}{}[}{}{}{]{}{}}[

65. node1$ rabbitmqctl cluster_status
Cluster status of node rabbit@node1 ...
[{nodes,[{disc,[rabbit@node1]}]},{running_nodes,[rabbit@node1]}]
...done.
Creating a cluster

66. node2$ rabbitmqctl join_cluster --ram rabbit@node1
node3$ rabbitmqctl join_cluster rabbit@node2
node3$ rabbitmqctl cluster_status
Cluster status of node rabbit@node3 ...
[{nodes,[{disc,[rabbit@node3,rabbit@node1]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node1,rabbit@node3]}]
...done.
Creating a cluster

67. Starting/Stopping Nodes

68. node1$ rabbitmqctl stop_app
node2$ rabbitmqctl forget_cluster_node rabbit@node1
node1$ rabbitmqctl reset
node1$ rabbitmqctl start_app
node2$ rabbitmqctl cluster_status
Cluster status of node rabbit@node2 ...
[{nodes,[{disc,[rabbit@node3]},{ram,[rabbit@node2]}]},
{running_nodes,[rabbit@node2,rabbit@node3]}]
...done.
Removing Nodes

69. Solution 2: HA

70. HA + Queue Mirroring
RabbitMQ
Node 1
RabbitMQ
Node 2
Load Balance / Floating IP / Low TTL DNS etc.

71. https://github.com/asgrim/rmq-slides
Have a go yourself!

72. https://joind.in/13365
James Titcumb @asgrim
Any questions? :)