What are the issues integration in integrating sensor nets and other distributed systems collecting and sharing real time data? How does RTI's Data Distribution Service address the integration needs without sacrificing the real-time collaboration constraints?

1. Integration Platform
for JMPS using a Data
Oriented Architecture
Supreet Oberoi
Vice President of Engineering
Real-Time Innovations, Inc.

2. About Joint Mission Planning System

3. JMPS promises to…
Provide a framework to enable legacy systems
and new COTS products to collaborate
– Legacy systems include Portable Flight Planning Software
(PFPS), Air Force Mission Support Systems (AFMSS), Tactical
Aircraft Mission Planning Systems (TAMS)
– COTS and GOTS products could include IM, Lotus Notes,
databases, messaging systems…
Goals of JMPS essential to the success of
any Integration Platform

4. Goal 1: Leverage Existing Systems
Utilize significant legacy investments:
– Portable Flight Planning Software (PFPS)
– Air Force Mission Support System (AFMSS)
– Tactical Aircraft Mission Planning System (TAMPS)
Harness COTS technology and messaging
technologies:
– Email, Chat
– File Transfer, Phone…

5. Goal 2: Share Relevant Data
Route plans, saved as XML-based “.JRT” files can be
changed prior to each flight.
The changes made by COTS or legacy applications
need to be disseminated. Challenges in supporting “joint
file manipulation” include:
– Broadcasting the entire file for even a small change
– Tuning dissemination of changes based on nature of the
network links (connected/disconnected, LAN/wireless/radio)
– Identifying set of parties affected by the change
– Identifying parties classified to see the data
– Supporting conflict management when multiple application
change the same data

6. Goal 3: Automate Integration
Human-in-the-Loop design for integrating legacy and
COTS applications is error-prone, inefficient
– What is required is a data-centric design, where each
application gets data that they need, when then need, and how
they need in an automated manner
Cannot force changing the application to automate
integration
– Not possible in many cases (e.g., COTS products)
– Increases cost of QA, deployment and software maintenance
Automate transformation of data between different
applications

7. Goal 4: Support Diverse Platforms
Optimize use of network resources
– Each application may need data with a different sense of
urgency
– Data may need to be processed differently for sending over
different network links such as radio, wireless, LAN
Transport Protocols
– UDP, Radio, TCP, Shared Memory, Infiniband…
Operating Systems & Programming Languages

8. The Way Forward…

9. Net-Centric Objective
Goal: Build fully
integrated real-time
net-centric systems.
Need: Seamless
integration of many
platforms, systems,
and their data, over
many transports
using a variety of
technology.

10. Net-centric Challenges
- Interoperability
limitations
- Inflexible
implementations
- Performance &
Scalability Limitations
- Reliability,
Availability,
and Bandwidth
- Security concerns
- Cost

11. History: DDS the Standard
Data Distribution Service for Real-Time Systems
– Adopted in June 2003
– Finalized in June 2004
– Revised June 2005, June 2006
– Specification of API for Data-Centric Publish-Subscribe in real-time
distributed systems.
Interoperability wire protocol
– Adopted in July 2006
– Revised in July 2007
Related specifications
– UML Profile for DDS
– DDS for Light-Weight CCM
Multiple (7+) Implementations

12. DDS Adoption – Aerospace & Defense
Boeing
AWAKS E2C Hawkeye
Northtrop
Qinetiq
SAAB
Boeing
Future Combat
Systems
Raytheon
SSDS
Lockheed
AEGIS Boeing/Insitu
Unmanned
Air Vehicles
12

13. Net-Centric Integration Problem
Net-Centric Implementation Framework
Part 4: Node Design Guidance Figure 1. c

14. Air traffic control scenario
Avionics Airplane LAN
Edge Real-Time
Gateway Real-Time Gateway
Alarms
Control Tower LAN
Real-Time
Gateway Enterprise Gateway
Arrival time
Enterprise Airport LAN
Enterprise
Backbone

15. The Solution: DDS Global Data Space
Data accessible to all interested applications:
– Data distribution (publishers and subscribers): DDS
– Data management (storage, retrieval, queries): SQL
– ESB Integration, Business process integration: WSDL
– Rich QoS, automatic discovery and configuration
– Real-time and/or high-performance access to data
Distributed Distributed
SQL SQL
Node Node
DBMS
D T Distributed
DDS SQL Node
Global Data Space
Distributed WSDL DBMS DBMS Distributed
Node DDS Node

16. Why Data-Centric Middleware?
2.0 Sensors
2.0 Sensors
1.0 Common Services MUX
MUX RDR IFF ESM SAFE
Hawkeye has functionally RDR IFF ESM SAFE
DIA
DIA NAV
NAV MCP
MCP IPCC
IPCC
oriented software modules
Distributed Data Framework
FIL
FIL TDM
TDM aADNS TIS
Each module talks to many 4.0 BMC2
4.0 BMC2 DWC
other modules 3.0 Fusion WAC
WAC TDA
TDA RAIDER CHAT
RIP
RIP CEC
CEC TRK
TRK MSI
MSI
Adding new
functionality cascades 5.0 Communications 6.0 Sensor Control
6.0 Sensor Control
L4 SEN
SEN DSC
integration re-work L4 L11
L11 L16
L16 IPv6 DSC
across many other
modules 7.0 Visualization 8.0 Training
8.0 Training
HMI
HMI ACIS
ACIS T4O
Grouping the modules into functional clusters does nothing to change that reality and
ease software integration
Changing the communication between the modules can ease integration, when the new
‘Publish Subscribe’ approach is used – each module publishes its output w/o regard to who
is receiving it, in contrast to the point-to-point approach of traditional inter-process
communication
It’s about an architecture that can assimilate evolving functionality,
rather than remaining set in time
UNCLASSIFIED

17. Top reasons to use DDS
Flexibility and Power of the data-centric model
Performance & Scalability
Rich set of built-in services
Interoperability across platforms and Languages
Provides/integrates Pub-Sub into SOA

18. #1 RTI Data-Centric Model
“Global Data Space” generalizes Subject-Based Addressing
– Data objects addressed by DomainId, Topic and Key
– Domains provide a level of isolation
– Topic groups homogeneous subjects (same data-type & meaning)
– Key is a generalization of subject
Key can be any set of fields, not limited to a “x.y.z …” formatted string
Data Reader
Data Writer Topic (subject)
Key
Data Object
Data Reader
Data Writer
Data Reader
Data Writer

19. Subscriptions: By Topic, Subject, Content
Topic: “Market Data”
Payload
Field Source Symbol Type Exchange Volume Bid Ask …
Value * * * * *
Topic: “Order Entry”
Payload
Field Symbol Type Exchange OrderNumber Symbol OrderKind Stop Limit …
Value * * NYSE *
Subject Filter (for a Reader)
Topic: “Market Data”
Field Source Symbol Type Exchange Payload
Value REUTERS * EQ NYSE Volume > x, Ask < y
Subject Filter (for a Reader) Payload Filter (for a Reader)
Company Confidential

20. #2 Performance & Scalability
DDS was designed to support high performance
RTI DDS was developed to maximize performance and minimize
jitter
Advanced techniques employed:
– Pre-allocation of memory
Never allocate/free memory in the critical path
– Use dedicated threads per receive port
Minimize thread switching
Avoid expensing operating system calls (e.g. select())
– Maximize concurrency
Carefully design critical sections
Patented concurrent mutex-free thread-safe data structures
– Employ high-performance data-access APIs
Read data by array (no additional copies)
Scatter/gather APIs to access transport.
Buffer loaning for zero copy access

21. Study on impact of WS technologies for future European ATC:
XML is not suitable for European Flight Data Distribution
Data size explodes 10X vs. CDR
– Flight Plans go from 100KB to 1MB
Communication speed drops 20X
Not good for Radios!!
Source: Christian Esposito and Domenico Cotroneo, Dario Di Crescenzo.
SELEX-SI/Consorzio SESM/University of Naples.
“Flexible Communication Among DDS Publishers and Subscribers”
July 2008, Real-Time Systems Workshop, Washington, DC

22. #3 Powerful Services & Tools
– High-Availability
– Persistent Data
– Recording service
– Relational Database bridge
– Development & Monitoring Tools

23. #4 Interoperability between platforms & languages
Data accessible to all interested applications:
– Data distribution (publishers and subscribers): DDS
– Data management (storage, retrieval, queries): SQL
– ESB Integration, Business process integration: WSDL
– Legacy Java Integration: JMS
Distributed Distributed
SQL JMS
Node Node
DBMS
D T Distributed
DDS SQL Node
Global Data Space
Distributed WSDL DBMS DBMS Distributed
Node DDS Node

24. DDS: Multi- Architecture Support
Solaris Windows RTOS Linux
RTI DDS RTI DDS RTI DDS RTI DDS
• Same API for all platforms
• Language Independence: C, C++, Java, C#, .NET, ADA
• Enterprise and Embedded Support
VxWorks, INTEGRITY, LynxOS
Linux, Solaris, Windows
• Prototype on any platform

25. RTI DDS: Pluggable Transports
• Enables non-IP centric transports (e.g InfiniBand)
• Allows for multiple transports on same node
• Provides high-performance (zero-copy interface)
• Saves bandwidth (compact messages & encapsulation)
Real-time Applications
RTI DDS
UDP
IPv4 & IPv6 Shared Custom
InfiniBand
Memory (e.g. Radio)
Standard IP network
(Ethernet, Wifi, etc.)

26. #5 Provides Real-Time Pub-Sub in SOA
Real-Time
Devices Fault Auditing &
Tolerance Recording
WS-DDS
Real-Time Pub-Sub/Caching/Messaging
SOA &
Real-Time
Web Services
Database
Event
Processing
Tools &
Visualization

27. Real-Time SOA Architecture/Implementation
RT Architecture/Technology
High Performance
Event-Driven/Publish-Subscribe
Small footprint
Quality of Service
DDS Data Bus Support for embedded
environments
Support for unreliable & low-
bandwidth networks
Traditional Enterprise
Low Performance
Client-Server
Centralized (Server-based)
TCP based

28. Demo – Integrating COTS with DDS
Display live DDS Data in Excel
Perform real-time computations and charts
Publish DDS data from Excel
ShapesDemo blank history saved demo QoS

29. Integrating with DDS (1/3)
Direct DDS integration (programmatic)
– Most efficient approach to integration
– APIs available in C, C++, Java, .NET (C#, C++/CLI)
Direct JMS integration
– RTI provides peer-peer interoperability between DDS and RTI’s
Java Message Service driver
Bridging via a Database
– Synchronize between global data space and RDBMS
– Support for Oracle RDBMS, Oracle TimesTen, MySQL

30. Integration with DDS (2/3)
Web Services
– Web Services interface to DDS (WS-DDS)
– Mapping between DDS types and XML Schema
Complex Event Processing
– Designed for running queries of high-throughput streaming data
– Provide a broad-selection of off-the-shelf input and output
adapters, lowering the cost of integration
– Excellent technology to systems with network constraints,
requiring processing of high-throughput data in near-real time

31. Integrating with DDS (3/3)
Summary
Access to
Impedance Integration DDS
Type Performance matching effort Capabilities
Direct DDS P2P Highest DDS filters, Requires All
history cache programming
and QoS
Direct JMS P2P High DDS filters, Configuration Basic
history cache only
and QoS
Database Service- Medium to SQL in Configuration Almost all
bridge based low addition to only
DDS filters,
history cache
and QoS
WS bridge Service- Low DDS filters, Configuration Almost all
based history cache only
and QoS
CEP as Service- Medium CEP Minimal Basic
bridge based extensions to
SQL in
addition to
DDS filters,
history cache
and QoS

32. Conclusions
DDS is a mature international Standard from OMG
– Platform Neutral: Operating systems and Programming
Languages
– Deployed worldwide in Military systems and other
Demanding real-time applications
DDS Is mandated by US DoD for Publish-Subscribe
and data-distribution applications
DDS is ideally suited to Network-Centric Systems
– Highly Tunable via Quality of Service (QoS)
– Can accommodate unreliable & low-latency transports
– Rich services (persistence, filtering, high-availability)