Introduction to message brokers; demo session with ActiveMQ and Python; quick overview of alternatives to ActiveMQ; Q&A

1. Message Brokers and
Python
Python Ireland Meetup - Jun/2012
Fernando Ciciliati
ciciliati@gmail.com

2. Contents
What sort of problems are we trying to solve?
Messaging concepts
A solution based on ActiveMQ
ActiveMQ + Python: STOMP and Stomp.py
ActiveMQ Queues and Topics
Other ActiveMQ concepts
Alternatives to ActiveMQ
AMQP
RabbitMQ
ZeroMQ
Others

3. Problems
Time decoupling
Scalability / Architecture flexibility
Technology mix

4. Concepts
Producer
Consumer
Message
Control x Data x Hybrid
Small x Large
Durability
Queue
Message Broker

5. Apache ActiveMQ
Disclaimer: Each broker has its own view of the world!
Apache Project
Written in Java
Designed for high performance
Flexible storage layer
Multiple communication protocols
Clustering capabilities
Very active community -> constant evolution ->
... bugs

6. Apache ActiveMQ
Web console on port 8161
let's have a look at it...
STOMP is not enabled by default
Configuration: activemq.xml
Running? Check it with netstat -ltn

7. ActiveMQ and Python
Easiest way (IMHO): STOMP
Libraries
stomp.py
stompy
twisted + stomper/stompest
Our choice
stomp.py

8. STOMP
Simple (or Streaming) Text Orientated Messaging Protocol.
in-band (like HTTP or SMTP)
Headers + body
Text, not binary
Simple
Let's telnet...

9. stomp.py
Created by Jason Briggs
Currently at version 3.x
Supports Python 2 and Python 3
Has a CLI !
let's have a look at it...

10. More ActiveMQ
Persistence
Producer: Receipt
Consumer: Ack/NoAck
Exclusive Consumers
Selectors
Queues X Topics
Prefetch

11. A simple Python producer
def sendJMS(message, host, queue, port=61613, user='', password='', persistent=True):
# Size of a buffer for storing MQServer STOMP answers to commands.
bufsize = 200
# Establish a TCP connection
t = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
t.connect ((host, port))
# Establish a STOMP connection
frame = 'CONNECTnnx00'
t.sendall(frame)
resp = t.recv(bufsize)
if resp[0:9] != 'CONNECTED':
print ('sendJMS: Failed to connect to MQ server')
return 1
# Send the message
if persistent:
persist="true"
else:
persist="false"
frame = 'SENDndestination:%snpersistent:%snn%sx00' % (queue,persist,message)
t.sendall(frame)
# Disconnect STOMP
frame = 'DISCONNECTnnx00'
t.sendall(frame)
# Disconnect TCP
t.close()
del(t)
return 0

12. A simple stomp.py producer
import time
import sys
import stomp
USAGE = """
producer.py <destination> <message>
<destination> should start by /queue or /topic
<message> is the text to be sent as message body (blank spaces accepted).
"""
if len(sys.argv) < 3:
print USAGE
else:
conn = stomp.Connection() # with default parameters
conn.start()
conn.connect()
conn.send(' '.join(sys.argv[2:]), destination=sys.argv[1])
time.sleep(2)
conn.disconnect()

13. A simple stomp.py consumer
import sys
import time
import stomp
class MyListener(object):
def on_error(self, headers, message):
print '*** received an error:n%sn***' % message
def on_message(self, headers, message):
print '*** received a message:n%sn***' % message
conn = stomp.Connection()
conn.set_listener('', MyListener())
conn.start()
conn.connect()
conn.subscribe(destination=sys.argv[1], ack='auto')
try:
while True:
time.sleep(1)
except KeyboardInterrupt:
pass
finally:
print "nDisconnecting..."
conn.disconnect()

14. AMQP
"""
OUR VISION:
To become the standard protocol for
interoperability between all messaging
middleware
"""
OASIS working group

15. RabbitMQ
Message broker written in Erlang
Supports AMQP
Concepts:
Producers
Messages
Exchanges
Queues
Consumers

16. ZeroMQ
A "brokerless" approach to messaging
"The Intelligent Transport Layer"
ØMQ zeromq:
Ø The socket library that acts as a concurrency framework.
Ø Faster than TCP, for clustered products and supercomputing.
Ø Carries messages across inproc, IPC, TCP, and multicast.
Ø Connect N-to-N via fanout, pubsub, pipeline, request-reply.
Ø Asynch I/O for scalable multicore message-passing apps.
Ø Large and active open source community.
Ø 30+ languages including C, C++, Java, .NET, Python.
Ø Most OSes including Linux, Windows, OS X.
Ø LGPL free software with full commercial support from iMatix.
(text from http://www.zeromq.org)

17. Even more questions?
:)
Thanks!