This document summarizes a presentation on using NoSQL databases as message stores with messaging architectures. The presentation introduces common messaging patterns like message store, claim check, and aggregator. It then discusses using Redis, Oracle NoSQL, and MongoDB as message stores and shows examples implementing messaging patterns with Spring Integration using these NoSQL databases. The document includes an architecture diagram and screenshots from demos of a sample application.
Messaging Architectures with NoSQL Databases as Message Stores
1. MESSAGING ARCHITECTURES WITH NOSQL
DATABASES AS MESSAGE STORES
JavaOne 2012 Conference (#JavaOne)
Srini Penchikala (@srinip)
10.4.12
2. GOALS OF THIS PRESENTATION
NoSQL databases to store messages with focus on EIP
Patterns
Target Audience: Integration Architects and Developers
Redis, Oracle NoSQL, and MongoDB databases
Demos (Java application)
Not a NoSQL or Messaging intro presentation
Format:
50 min presentation & demos
10 min for Q&A 2
3. WHO AM I
Software Architect
Co-author of Spring Roo in Action book
Editor at InfoQ.com
Background:
Financial services industry
Regulatory compliance (PCI & FISMA)
J2EE architecture model
Scrum for software development
Agile (v./and) Architecture
3
4. QUICK POLL
Who is using EIP Patterns?
Spring Integration?
NoSQL?
Spring Data?
4
5. TODAY’S DISCUSSION
Enterprise Integration Patterns
NoSQL databases
NoSQL as message stores
MongoDB / Redis / Oracle NoSQL Database
Spring Data
Spring Integration framework
Sample Java application
Best Practices
Conclusions
5
7. MESSAGING PATTERNS
Message Store
Claim Check
Queue Channel
Aggregator
Resequencer
Control Bus*
Wire Tap*
7
URL: http://www.eaipatterns.com
*System monitoring and management
8. MESSAGE STORE
Store Messages in a persistent
store (e.g. Database)
Why Message Store?
Performance
Availability
8
http://www.eaipatterns.com/MessageStore.html
9. MESSAGE STORE USE CASES
Message buffering
Aggregator
Resequencer
Queue Channel
Message loss prevention
Claim Check
9
10. CLAIM CHECK
Store message data in a
persistent store and pass a
Claim Check to subsequent
components.
These components can use
Claim Check to retrieve the
stored information.
10
http://www.eaipatterns.com/StoreInLibrary.html
11. PERSISTENCE
Key Value Data Store
Redis
Oracle NoSQL
Document Database
MongoDB
GemFire
JDBC
In-Memory
11
12. NOSQL DATABASES
Redis
Redis-based MessageStore
Publish-Subscribe Messaging adapters
Oracle NoSQL Database
Distributed key-value database
ACID transactions
Administration via Web console or CLI
Use Cases: Online Display Advertising & Mobile Social Gaming
MongoDB
Document database
12
http://www.redis.io, http://www.mongodb.org
http://www.oracle.com/technetwork/products/nosqldb/overview/index.html
13. SPRING INTEGRATION
Extension of the Spring framework
Implements Enterprise Integration Patterns (EIP)
Lightweight messaging
Integration with external systems via declarative adapters
Separation of concerns
13
*http://www.springsource.org/spring-integration
14. SPRING DATA
Spring applications that use new data access technologies
Non-relational databases
Map-reduce frameworks
Current Implementations:
Big Data (Apache Hadoop)
Data-Grid (GemFire)
Key Value Stores (Redis)
Document Stores (MongoDB)
Graph Databases (Neo4j)
Column Stores (Hbase) 14
*http://www.springsource.org/spring-data
15. SPRING ROO
Rapid Java application development
Nice integration for Spring Integration and Spring Data
frameworks
15
*http://www.springsource.org/spring-data
16. MESSAGE STORE “STRATEGY”
Based on Strategy design pattern
MessageStore strategy
Used by QueueChannel and ClaimCheck patterns
MessageGroupStore strategy
Used by Aggregator and Resequencer patterns
Implementations
SimpleMessageStore (In-memory, Map-based)
JdbcMessageStore (uses a relational database via JDBC)
NoSQL Data Store
16
26. MESSAGE STORE LIMITATIONS
When using JdbcMessageStore, only Serializable data is
persisted by default
Different instance of some Spring Bean is returned upon
deserialization for headers that represent certain types of
data
Temporary Reply Channel issue
26
28. RESOURCES - BOOKS
28
Enterprise Integration Patterns
(http://eaipatterns.com)
Spring Integration in Action
(http://www.manning.com/fisher)
Spring Roo in Action
(http://www.manning.com/rimple)
29. RESOURCES
Enterprise Integration Patterns (http://www.eaipatterns.com)
Spring Integration (http://static.springsource.org/spring-
integration/reference/html)
Spring Integration Support for Redis (http://static.springsource.org/spring-
integration/reference/html/redis.html)
Oracle NoSQL Database
(http://www.oracle.com/technetwork/products/nosqldb/overview/index.html)
http://blog.codecentric.de/en/2012/04/spring-data-redis/
Message Store (http://www.eaipatterns.com/MessageStore.html)
Redis (http://redis.io/topics/pubsub)
MongoDB
Spring Data
Spring Data Redis
Sample Application on Github
29
30. THANK YOU
Contact Information
http://www.infoq.com/author/Srini-Penchikala
srinipenchikala@gmail.com
@srinip
http://srinip2007.blogspot.com
Spring Roo in Action Book
Questions?
30
33. CONTROL BUS
Administer a messaging
system distributed across
multiple platforms
Uses same messaging
mechanism used by
application data, but uses
separate channels to
transmit data for the
management of message
flow components 33
http://www.eaipatterns.com/StoreInLibrary.html
34. WIRE TAP
Inspects messages that
travel on a point-to-point
channel
Recipient List into the
channel that publishes
each incoming message to
the main channel and a
secondary channel
34
http://www.eaipatterns.com/WireTap.html
35. XML CONFIGURATION OF MESSAGE STORE
<bean id="redisMessageStore"
class="org.springframework.integration.redis.store.RedisMess
ageStore">
<constructor-arg ref="redisConnectionFactory"/>
</bean>
<int:channel id="somePersistentQueueChannel">
<int:queue message-store="redisMessageStore"/> <int:channel>
<int:aggregator input-channel="inputChannel" output-
channel="outputChannel" message-store="redisMessageStore"/>
35