The features of a communication technology that yield the properties of interoperability and reliability can be visualized in layers: technical (at the level of bytes), syntactic (at the level of messages), semantic (at the level of data, i.e. what the messages refer to), and so on. Real-world systems require at least data-level interoperability and reliability. The question is: will you acquire something that already supports those capabilities, or will you build it atop something that doesn't? This talk compares and contrasts DDS and AMQP as technology exemplars in each category.

1. Engineering Interoperable and
Reliable Systems
Contrasting DDS, JMS, and AMQP
The Global Leader Rick Warren rick.warren@rti.com
in DDS Director of Technology Solutions

2. Introductions
Rick Warren rick.warren@rti.com
! Director of Technology Solutions
– Aligning product strategy with customer needs
– Coordinating existing product, accelerated product,
professional services
! OMG principal
– Participated DDS revision task force
– Authored or co-authored Extensible and Dynamic Types
(DDS-XTypes), C++ PSM, Java PSM, and other specs
– Chaired several specification finalization task forces
! Former Principal Engineer
– With RTI nearly 10 years
– Experience in R&D and Services
© 2011 Real-Time Innovations, Inc. 2

3. Most Important Points in this Presentation
Themes: Interoperability,
Reliability
! Flow first from architecture Agenda
– What is the information you’re 1. Architecture
modeling?
– How does it change over time? – Levels of
interoperability
– How to address dynamic – Levels of
systems? reliability
– How to address system – Data-centricity
evolution, integration?
! Pick the right tool for the job 2. Technology
– Different technologies provide
Implications
• Different degrees of architectural – DDS
support – JMS
• Different levels of portability, – AMQP
interoperability
– Biggest mistake: “It’s all just
‘messaging’.”
© 2011 Real-Time Innovations, Inc. 3

4. (Technology Refresher)
! Java Message Service (JMS)
– Messaging at syntactic level:
no assumed semantics
– Portable API for messaging
in Java ! Data Distribution Service (DDS)
– Specification from Java – Application of messaging to
Community Process (JCP) distributed state model
– Portable API for data distribution
! Advanced Message Queuing
in multiple languages (DDS)
Protocol (AMQP)
– Interoperable protocol for data
– Messaging at syntactic level: distribution (DDS-RTPS)
no assumed semantics
– Specifications from Object
– Interoperable protocol for Management Group (OMG)
messaging
– Specification from AMQP
Working Group, industry
consortium
© 2011 Real-Time Innovations, Inc. 4

5. Levels of Interoperability and Reliability
Interpretation of
Semantic messages e.g. DDS
Cumulative
Communication e.g. JMS,
Syntactic fragments—
“messages” AMQP
Technical Bits and bytes e.g. TCP
© 2011 Real-Time Innovations, Inc. 5

6. Levels of Interoperability and Reliability
Interoperability Means:
Data Level Agree on what information
Semantic refers to, how it’s related
Message Distinguish one piece of
Level information from another
Syntactic
Byte Level Exchange raw information,
Technical regardless of structure
© 2011 Real-Time Innovations, Inc. 6

7. Levels of Interoperability and Reliability
Reliability Means:
Order across “Summarize”
Data Level streams, data changes for
eliminating slow / late-joining
Semantic duplicates consumers
Message Route across
Store for later
Level multiple
connections
delivery
Syntactic
Repair dropped
Byte Level bytes over
reliable
Technical connection
© 2011 Real-Time Innovations, Inc. 7

8. Levels of Interoperability and Reliability
Data Level We call these technologies
Semantic “data-centric.”
Message We call these technologies
Level “message-centric.”
Syntactic
Higher levels implemented on top of lower
based on chosen conventions
Byte Level • Conventions not standardized => no
interoperability
Technical • Implementations of lower layers
constrain behavior, performance of upper
© 2011 Real-Time Innovations, Inc. 8

9. Data-Centricity by Example: Calendaring
Alternative Process #1:
1. Email: “Meeting Monday at 10:00.”
2. Email: “Meeting moved to Tuesday.”
3. Email: “Here’s dial-in info for meeting…”
4. You: “Where do I have to be? When?”
5. You: (sifting through email…)
© 2011 Real-Time Innovations, Inc. 9

10. Data-Centricity by Example: Calendaring
Alternative Process #2:
1. Calendar: (add meeting Monday at 10:00)
2. Calendar: (move meeting to Tuesday)
3. Calendar: (add dial-in info)
4. You: “Where do I have to be? When?”
5. You: (check calendar)
© 2011 Real-Time Innovations, Inc. 10

11. Data-Centricity by Example: FaceBook
Alternative Process #1:
! Use the FaceBook web site.
Alternative Process #2:
! The web site doesn’t exist.
! Deduce your friends’ information based on the
notifications FaceBook sends you.
© 2011 Real-Time Innovations, Inc. 11

12. What’s the Difference? State.
! Things have attributes and
characteristics “State” (“data”) is a
– The meeting will run 1:00–2:00 snapshot of those
in the conference room.
– My friend’s phone number is attributes and
555-1234 and he’s currently characteristics.
grooming his cat.
– The car is blue and is traveling
north from Sunnyvale at 65
mph.
! …whether they exist in the Best Practice:
real world, in the computer,
or both operate on state
! …whether or not we observe directly, not dialogs
or acknowledge them about state.
© 2011 Real-Time Innovations, Inc. 12

13. Data-Centric =
the part of you care about
Describing the world
as it is
at a certain point in time
Implication: State of the world can be maintained by
infrastructure, not each app
© 2011 Real-Time Innovations, Inc. 13

14. Not Data-Centric =
Saying anything else
! “Hey you: go do this.”
! “The thing changed in this way.”
Implication: State must be inferred, reconstructed,
managed by each app
(“Message-centricity”: focus on what’s said vs. what is)
© 2011 Real-Time Innovations, Inc. 14

15. Why is it better to just describe the world?
! Reconstructing the state of the world is hard.
– Must infer based on all previous messages Reduce
– Maintaining all these messages is expensive Effort
– Each app makes these inferences => duplicate effort
! People make mistakes, and systems fail.
– Many copies of state => may be different => bugs
vs. Improve
Quality
– Uniform operations on state => fewer bugs
! The world changes.
– Learning current state must not require replaying
everything ever said to you Future-
– Adding a function to a system must not require Proof
modifying every other function Design
© 2011 Real-Time Innovations, Inc. 15

16. Before We Forget: the Definition For example, data
structures in XSD
A data-centric architecture: or IDL file.
1. …is based on a data model that is: Calendar Event =
• Start Time
– Appropriately documented—
• Duration
i.e. understandable by humans • Location
– Formally defined— • Organizer
i.e. understandable by machines
– Discoverable—i.e. can be found during execution
2. The data model is independent of any domain-specific
functionality / application.
– i.e. made of nouns, not verbs
– Actions are uniform; typically: Create, Read, Update, Delete (CRUD)
3. The (instantiated) data model is the only authoritative source
of state in the system.
© 2011 Real-Time Innovations, Inc. 16

17. DDS is not messaging.
DDS is data-centric messaging.
That is,
! Messaging as a technique
! …for distributing, accessing, and modifying stateful data
App App App App App App
Data
Data in Motion at Rest
e.g. DDS complements e.g. database
© 2011 Real-Time Innovations, Inc. 17

18. Example: Data-Centric Radar Data
as in DDS
Data Schema Map this into XML; rows + cols
id : string (key) Express content-based filters
x : float
Propagate data efficiently
y : float
Delete Create Update Create
Subscribe
Publish
“AA123” “DL987” “AA123” “AA123”
65.4 56.7 45.6
X 32.1 89.0 78.9
© 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 18

19. Example: Data-Centric Radar Data
as in DDS
Data Schema Quality of Service
id : string (key) History
x : float Deadline
y : float Time-Based Filter
! Once infrastructure understands objects, can attach
QoS contracts to them
Subscribe
Publish
! “Keep only the latest value” or “I need updates at
this rate” make no sense unless per-object
– Flight AA123 updates shouldn’t overwrite DL987, even if
AA123 is updated more frequently
– Update rate for one track shouldn’t change just because
another track appeared
© 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 19

20. Example: Message-Centric Radar Data
as in JMS or AMQP
Nothing to base filters, xforms on
(No Data Error checking dev " integration
Schema,
Limited QoS) Self-describing data is verbose
Subscribe
“My app 0x00000006 id=“AA123”
Publish
knows this 4141010203 x=float(45.6)
means 0042366666
delete.” 429DCCCD y=float(78.9)
© 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 20

21. Data Level
Message Level
Message-Level Interoperability Byte Level
! JMS, AMQP cover syntax
of message exchange
– If we agree on an abstract
destination, I can send you
messages
– Not addressed: Needed for
• What these messages
represent interoperability!
• What they will contain – Especially when integrating
• How the data in the business independently developed
domain changes over time components
! (Interoperability Mitigation: implement
limitations within yourself, idiomatically
syntactic level):
– More work
– JMS does not specify
network interoperability – Difficult to govern
– AMQP does not specify API – Non-standard
© 2011 Real-Time Innovations, Inc. 21

22. Data Level
Message Level
Data-Level Interoperability Byte Level
! DDS applies semantics to ! DDS governs data
“messages”
– Topic = group of similar structure
objects / records
– Instance = particular object,
– At design time and run
identified by key time
– Sample = state of particular – Based on discoverable
object at particular point in time
– Like relational DB that changes schemas
over time
! DDS governs data
changes
– Among pubs, subs
Topic – Based on discoverable
QoS
Key 1 Value Value
Key 2 Value Value # Machine-readable
definitions amenable
Key 3 Value Value
to automation
© 2011 Real-Time Innovations, Inc. 22

23. Data Level
Message Level
Message-Level Reliability Byte Level
JMS, AMQP provide This model is limited:
syntactic reliability, 1-to-1.
# When communication is
! You send a message, many-to-many
I receive it – How to order messages
across multiple
! Messages have no producers?
particular relationships to – How to eliminate
each other, outside world duplicates?
# When networks are
Pub Sub dynamic / disconnected
– How to sync late joiners to
current state?
Pub Sub # When consumers can’t
keep up
– How to summarize / sub-
Pub Sub sample message stream?
© 2011 Real-Time Innovations, Inc. 23

24. Data Level
Message Level
Data-Level Reliability Byte Level
DDS provides semantic # Addresses many-to-many:
reliability, many-to-many. – Choose: Should updates come
! Some number of sources from all sources at once, or one
at a time?
publish the changing states
of some objects – Choose: If all at once, order by
source time or reception time?
! Samples represent – Duplicates detected and
snapshots of object state eliminated automatically
over time
# Addresses dynamic /
disconnected networks:
Pub Topic Sub – Identify number of relevant
previous states; late joiners get
just these
Instance
Pub Sub # Addresses slow consumers:
Instance – Doesn’t (re)deliver obsolete
previous states
Pub Instance Sub – Sub-sample rapid updates based
on configuration
© 2011 Real-Time Innovations, Inc. 24

25. Summary
1. Architect for interoperability and Technology Score Card
reliability
– Start with common data model of business ! DDS
domain – Interoperability, reliability
• …Including static and dynamic elements founded on system-defined
data model
– Design interactions based on applying uniform • Discoverable
operations (CRUD) on this model
• Machine-readable
– Account for enforcement / governance • Automatically enforced
2. Select technology based on – Portable API in multiple
conformance to architecture wherever languages
possible – Interoperable protocol
– Supports and enforces data model? ! JMS
– Provides standards-based code portability? – Interoperability, reliability
– Provides standards-based component defined at message level
interoperability? only: no pre-defined
semantics
3. Mitigate lack of architectural support – Portable API in Java
wherever necessary
– Document and implement data model ! AMQP
governance – Interoperability, reliability
– Create portable API wrappers where lacking defined at message level
– Define protocol usage idioms to increase only: no pre-defined
interop, reliability levels semantics
– Interoperable protocol
© 2011 Real-Time Innovations, Inc. 25