1. Mike Willbanks Sr. Software Engineer at CaringBridge Blog: http://blog.digitalstruct.com Twitter : mwillbanks IRC : lubs on freenode Talk: http://joind.in/1586 The Art of Message Queues TEK-X

3. PHP monkey

4. MNPHP User Group

5. Interested in Scalability & Performance

6. “ Message queues and mailboxes are software-engineering components used for interprocess communication, or for inter-thread communication within the same process. They use a queue for messaging – the passing of control or of content.” http://en.wikipedia.org/wiki/Message_queue

7. “ Messaging describes the sending and receiving of data (in the form of messages) between systems. Messages are exchanged between programs or applications, similar to the way people communicate by email but with guarantees on delivery, speed, security and the absence of spam.” http://www.rabbitmq.com/faq.html#what-is-messaging

8. Messages are Everywhere

9. A Basic Message Queue

11. Asynchronous push / pull

12. An application framework for sending and receiving messages.

13. A way to communicate between applications / systems.

14. A way to decouple components.

15. A way to offload work.

17. Communication between Applications / Systems

18. Image Resizing

19. Video Processing

20. Sending out Emails

21. Auto-Scaling Virtual Instances

22. Log Analysis

23. The list goes on...

25. You need to communicate between multiple applications / systems.

26. You have a limited number of processing slots.

27. You need to support legacy applications or use multiple programming languages. Good short presentation on why you might need it: blog.pasker.net/2008/06/16/you-might-need-messaging-if/

29. “Do it in the background.”

31. Distribution may also come into play.

32. Protocols

34. STOMP – Streaming Text Oriented Messaging Protocol

36. AMQP

38. Platform agnostic protocol.

39. Completely open, interoperable and broadly applicable.

40. Many severs available and many client libraries. http://amqp.org For more information on the protocol.

42. An exchange is routers with routing tables.

43. A queue is where the messages wait for a consumer.

44. A binding defines the routing rules.

45. In a nutshell....

47. OpenAMQ - openamq.org

48. RabbitMQ - rabbitmq.com

49. ZeroMQ - zeromq.org

51. Installation can be a bit painful since the c client needs to be linked.

53. Enough Text, lets see a picture

55. Create the exchange

56. Bind to the queue.

58. By default, nothing is made to be durable and will not survive in the event of the message server going down or a reboot or things of that nature.

59. A simple consumer:

60. A simple publisher:

61. Stomp

63. Several servers have plug-ins to handle the protocol.

65. Connect -> Subscribe -> [Ack] -> Disconnect

66. When you send in a message, you tell it which queue to go to.

67. When you subscribe you request a queue.

69. StompConnect - stomp.codehaus.org/StompConnect

70. StompServer - stompserver.rubyforge.org

71. Gozirra - germane-software.com/software/Java/Gozirra

72. RabbitMQ - rabbitmq.com

74. A simple publisher

75. A simple consumer

76. Job Queues

78. These are basically message queues with specialized purposes.

79. Gearman – gearman.org

80. Beanstalkd – kr.github.com/beanstalkd

81. Zend Job Queue – Zend Platform & Zend Server

83. It can optionally have persistence and works on a job / worker model.

84. You can respond back to the application and check on the status of a job.

85. PECL has a great PHP extension.

86. Very cool stuff.

87. Gearman Client Example

88. Gearman Worker Example

90. It has persistence and works on a job / worker model.

92. Pheanstalk Client Example

93. Pheanstalk Worker Example

94. Picking a Message Queue

95. Various Message Queue Servers

97. Scalability (Horizontal / Vertical)

98. Synchronous / Asynchronous

99. Batching Policies

100. Delivery Policies

101. Purging Policies

102. Routing Policies

103. Security Policies

104. Standards http://wiki.secondlife.com/wiki/Message_Queue_Evaluation_Notes

105. How to Use Your Queues

107. XML

110. A Simple Architecture Queue Receive Producer Message Queue Server Consumer

111. A Complex Architecture Producer Message Queue Server Consumer Consumer Consumer Consumer Consumer Message Queue Server Message Queue Server Producer Producer

114. Use Cases

117. Then allows us to focus on sending email through a single queue with multiple consumers.

118. “ Write programs to work together”

120. The consumer then utilizes a service call or whatever logic it needs to create the new instance.

121. Once the machine boots you could utilize another queue to send a message that states to put the server in the farm.

123. When a new video or photo comes in, send a message to the queue to do the work. Keep only enough workers / consumers available to not strain your machine.

124. Remember to monitor how many jobs are around.

126. In the event that you are not using gearman, you can create another queue to hold status updates. One way of doing this is to record an id number and check for it's status by utilizing another queue or updating a record in a database to poll it.

127. There are numerous ways to handle this.

129. We could send the payload through the message queue to be handled by the worker that is consistently connected to the APNs server.

130. Mike Willbanks Blog: http://blog.digitalstruct.com Twitter : mwillbanks IRC : lubs on freenode Talk: http://joind.in/1586 Thank You!