OpenSplice DDS Delivers High Performance Pub/Sub

OpenSplice DDS
Delivering Performance, Openness, and Freedom

Angelo Corsaro, Ph.D.
The Open Source Messaging
Product Strategy & Marketing Manager
OMG RTESS and DDS SIG Co-Chair
angelo.corsaro@prismtech.com

Accelerating Wall Street

Speaker Bio

Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
Angelo CORSARO, Ph.D.
Product Strategy & Marketing Manager
OMG RTESS and DDS SIG Co-Chair
PrismTech
F +33 1 69 01 53 54 | M +33 6 43 30 75 65 | T http://twitter.com/acorsaro

Dr. Angelo Corsaro is the OpenSplice DDS Product Strategy & Marketing Manager at PrismTech, a market leader in the field
of high performance middleware. His responsibilities include product strategy, competitive positioning, technology evolution,
and technology evangelism. He is also responsible for strategic standardization at the Object Management Group (OMG),
where is co-chair of the Data Distribution Service (DDS) Special Interest Group, and the Real-Time Embedded and Specialized
Services Task Force.
Angelo received a Ph.D. and a M.S. in Computer Science from the Washington University in St. Louis, and a Laurea Magna
cum Laude in Computer Engineering from the University of Catania, Italy.

© 2009, PrismTech. All Rights Reserved

Agenda

‣ The Big News
‣ What is OpenSplice DDS
‣ DDS vs AMQP

‣ Why OpenSplice DDS?
‣ Concluding Remarks


OpenSplice DDS

April 20th, 2009
OpenSplice|DDS Goes Live!
:: http://www.opensplice.org

OpenSplice DDS

High Performance
Pub/Sub

High Performance Pub/Sub
The right data, at the right place, at the right time
‣ Fully distributed, Peer-to-Peer -- All the Time.
Communication
‣ No Single Point of Failure Subscriber
Publisher

‣ No Single Point of Bottleneck

‣ Multicast-enabled Brokers Subscriber
‣ High performance and highly Publisher

scalable
‣ High availability Publisher
‣hot-swap
Subscriber

‣hot-hot architecture

OpenSplice DDS

Data-Centric
Pub/Sub

Data-Centric Pub/Sub
‣ Data-Centric Features are built-in and
don’t rely on an external DBMS
‣ Providing thus performance, scalability,
and availability
‣ Distributed Relational Data Model Subscriber
DBMS
Publisher
‣ Local Queries B

‣ Continuous Queries / Content Based

m

Subscriptions A F

Subscriber
‣ Windows
Publisher J
D C

‣ Object/Relational Mapping K
E

‣ Support for a subset of SQL-92
Publisher
Subscriber

Perfect Blend of Data-Centric and Real-Time
Publish/Subscribe Technologies


Subscriber
Publisher

B

m

A F

Subscriber
Publisher J
D C

K
E

Publisher
Subscriber


Subscriber
Publisher J
B D

D C
K
E
A
B

B
A

Subscriber
m

D C
Publisher A F

J
J A F
D C
K

K
A F E

Subscriber
Publisher J
D

D K
E

E

Topics and Data-Centric Pub/Sub
‣ Topics. Unit of information exchanged between Topic
Publisher and Subscribers.

‣ Data Types. Type associated to a Topic must
be a structured type expressed in IDL
struct TempSensor {
int tID;
Topic Type
‣ Topic Instances. Key values in a datatype float temp;

float humidity;
uniquely identify a Topic Instance (like rows in };
#pragma keylist TempSensor tID
table)
TempSensor
‣ Content Awareness. SQL Expressions can be tID temp humidity
used to do content-aware subscriptions, 1 21 62
Instances
queries, joins, and correlate topic instances 2 27 78
3 25.5 72.3

y
idit ity SELECT * FROM TempSensor t
y
idit mid
Samples
um hu
um
h
WHERE t.temp > 25
h
62 63
65
p p
tem p tem
tem
21 22
tID 23 tID tID temp humidity
tID
1 1 2 27 78
1
3 25.5 72.3
Time
t1 t3
t2

Distributed Relational Information Modeling

‣ Topic Keys can be used to identify instances as well as relationships
‣ Relationships can be navigated by relying on a subset of SQL 92
‣ One-to-many relationships can be captured using foreign keys
‣ Many-to-many relationships need to be modeled using a topics

‣ Keys can be represented by an arbitrary number of Topic fields

TempSensor Room
Floor
tID: int rID: int fID: int
temp: float width: float level: int
humidity: float height: float open: int
rID: int length: float
fID: int


Object/Relational Mapping
TempSensor
temp: float
humidity: float
getTemp()

‣ Arbitrary object
* Room Floor
getHumidity()
1 width: float level: int
height: float open: int
reconstructions 1 climate: int
*
length: float
Conditioner
1 climate: int set_status()
temp: float

‣ Automatic Relationships set_climate(); set_climate()
humidity: float *

get_level()
setTemp()
setHumidity()
Management setFanLevel()
start()

‣ Inheritance stop()
pause():

‣ Local Operations TConditioner
cID: int TTempSernsor

‣ Local/Distributed State temp: float tID: int
humidity: float temp: float
fan_level: int TRoom humidity: float
status: int rID: int rID: int
rID: int width: float
height: float
length: float TFloor
fID: int fID: int
level: int
open: int


Data Centric Publish/Subscribe
Publishers Subscribers
TempSensor TempSensor
tID temp humidity tID temp humidity
18 60 18 60
1 1
22 75
2
21 71
3

struct TempSensor {
int tID;
TempSensor
TempSensor
float temp;
tID temp humidity
float humidity; tID temp humidity
22 75
2 };
22 75
2
21 71
3

SELECT * FROM TempSensor t
WHERE s.temp > 20
s.t
ID
TempSensor ==
Fully Distributed Global TempSensor
1
tID temp humidity

Data Space tID temp humidity
21 71
3
18 60
1


Data Centric Publish/Subscribe
Publishers Subscribers
TempSensor TempSensor
tID temp humidity tID temp humidity
1 21 62 1 21 62
22 75
2
21 71
3

struct TempSensor {
int tID;
TempSensor
TempSensor
float temp;
tID temp humidity
float humidity;
tID temp humidity
22 75
2 };
1 21 62
22 75
2
21 71
3

SELECT * FROM TempSensor t
s.t WHERE s.temp > 20
ID
TempSensor ==
Fully Distributed Global 1
tID temp humidity
TempSensor
Data Space
21 71
3
tID temp humidity
1 21 62


Processing Content & Structure Complex Event Processing...

‣ Local Queries OpenSplice DDS some of the most useful features
found in Complex Event Processing platforms!
‣ Continuous Queries (Content-based Subscriptions)
‣ Topics Joins & Projections
‣ Events windows (via History QoS)

S1
tID temp humidity
3 21.5 72.3
tID temp humidity {iID = 1}
2 20 78
tID temp humidity
1 21 62
S2
P {temp > 21 | humidity > 75}

S3
{ temp <16 }


Processing Content & Structure Complex Event Processing...

‣ Local Queries OpenSplice DDS some of the most useful features
found in Complex Event Processing platforms!
‣ Continuous Queries (Content-based Subscriptions)
‣ Topics Joins & Projections
‣ Events windows (via History QoS)

tID temp humidity
1 21 62

S1
{iID = 1} tID temp humidity
2 20 78
3 21.5 72.3

S2
P {temp > 21 | humidity > 75}

S3
{ temp <16 }


OpenSplice DDS

QoS-Enabled
Pub/Sub

QoS Model
‣ QoS-Policies are used to control relevant Type Matching
properties of OpenSplice DDS entities, QoS matching

such as: QoS QoS QoS
QoS QoS

‣ Temporal Properties Topic

‣ Priority
Name
Publisher Subscriber

... DataReader
reads
DataWriter writes Type
...
‣ Durability
...
‣ Availability Type reads DataReader
writes
DataWriter
‣ ... Name

‣ Some QoS-Policies are matched based on
Topic

a Request vs. Offered Model thus QoS- QoS QoS QoS

enforcement

‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible
‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability

Sample QoS Policies
QoS Policy Applicability RxO Modiﬁable QoS matching

T, DR, DW Y N
DURABILITY Data Availability
QoS QoS
QoS

T, DW N N
DURABILITY SERVICE Topic QoS
QoS
Name

DataReader
reads
DataWriter writes Type

Subscriber
T, DW - Y
LIFESPAN ...
Publisher

Type reads DataReader
T, DR, DW N N
writes
HISTORY
DataWriter

Name
Topic
P, S Y N
PRESENTATION Data Delivery
T, DR, DW Y N
QoS QoS QoS
RELIABILITY
QoS matching
P, S N Y
PARTITION

T, DR, DW Y N
DESTINATION ORDER

‣ Rich set of QoS allow to configure
T, DR, DW Y N
OWNERSHIP
DW - Y
OWNERSHIP
several different aspects of data
STRENGTH

availability, delivery and timeliness
T, DR, DW Y Y
DEADLINE Data Timeliness
T, DR, DW Y Y
LATENCY BUDGET

‣ QoS can be used to control and
T, DW - Y
TRANSPORT PRIORITY
optimize network as well as
DR - Y
TIME BASED FILTER Resources
computing resource
T, DR, DW N N
RESOURCE LIMITS

DP, DR, DW N Y
USER_DATA Conﬁguration
T N Y
TOPIC_DATA
P, S N Y
GROUP_DATA


Mapping QoS Which properties does QoS controls?

TimeBasedFilter Deadline History

Data Data
Throughput Lifespan Durability
Latency Availability

Ownership
LatencyBudget TransportPriority Ownership
Strength

Control over Latency/Throughput tradeoff
Control over data queueing
Control over data latency
Control over data persistency
Control over data priority
Control over data sources hot-swap

Reliability
OpenSplice DDS provides programmatic
QoS-driven support for configuring the most
Destination
Presentation Data Delivery
important properties of data distribution!
Order

Control over data distribution reliability
Control over data ordering
Control over presentation


OpenSplice DDS

Zero-Conf
Pub/Sub

Zero-Conf Pub/Sub
Code
‣ Dynamic Discovery. Publisher,
Bloat
Subscriber, and Topics are dynamically
discovered
‣ Separation of Concerns. Applications

are agnostic from networking details.
Thus you can re-deploy the same
business logic with different network,
partitioning, configurations
‣ Tool Based. All configuration is tool
assisted and happens outside of your
application


OpenSplice DDS

Standard-Based
Pub/Sub

The OMG Data Distribution Service
Data Distribution Service for Real-Time Systems
Application
‣ Leap forward in Topic-based Publish/Subscribe
Middleware state of the art Object/Relational Mapping

‣ Carefully specified to allow very high performance, Data Local Reconstruction Layer (DLRL)

DDS v1.2
scalable, predictable and high-availability
Content

implementations Ownership Durability
Subscription

‣ Language Independent, OS and HW architecture Minimum Proﬁle

independent Data Centric Publish/Subscribe (DCPS)

DDSI v2.1
Fully Standardized Solution Real-Time Publish/Subscribe Protocol
‣ Data Distribution Service (DDS) for Real-Time DDS Interoperability Wire Protocol
Systems v1.2
‣ RTPS, DDS Interoperability Wire Protocol UDP/IP

The OMG Data Distribution Service satisfies the most challenging information dissemination requirements
across a wide set of application domains, ranging from multi-board systems to system-of-systems

DDS!Recommendations is churning…
The infrastructure evolution cycle
– New -> Emerging -> Standard -> Commodity
Increasingly Mandated/Recommended by Administrations
– Middleware is emerging as OS declines
! …DDS is maturing…
‣ US Navy: Open Architecture
– OMG focus
– Wire spec

‣ DISR/DISA: Net-centric Systems
– Tools
– Enterprise integration
– Multiple products fielded
‣ EuroControl:– Deployed applications!
Air Traffic Control Center Operational
Interoperability
! …and adoption is on the rise
– Navy
– DISR
‣ QinetiQ: Recommending DDS for VSI
– FCS/SoSCOE

Scope of Standardization
AMQP OMG DDS
Application
Application
Object Relational Mapping
Standard API

Data Local Reconstruction Layer (DLRL)

DDS v1.2
Content
Ownership Durability
Subscription
API (?) Pub/Sub Standard API
Minimum Proﬁle

Data Centric Publish/Subscribe (DCPS)

DDSI v2.1
AMQP v0.10

Real-Time Publish/Subscribe Protocol
Advanced Message Queuing Protocol
Wire Protocol Standard
DDS Interoperability Wire Protocol
Interoperability Wire Protocol

UDP/IP
(TCP, SCTP, UDP)/IP


Architectural Style

AMQP OMG DDS
Subscriber
Subscriber
Publisher
Publisher

Broker
Broker

Broker Subscriber
Subscriber
Broker
Publisher
Publisher

Broker
Broker

Publisher Publisher
Subscriber Subscriber


Key Differences in Summary
AMQP DDS
‣ Brokered Communication ‣ Peer-to-Peer Communication
Architecture Architecture

‣ Wire Protocol Standard ‣ Wire Protocol + API Standard

‣ Interoperability across AMQP ‣ Interoperability across DDS implementations
implementations
‣ Application Portability across DDS
Implementation

‣ Header Based Routing ‣ Content-Based Filtering/Quering

‣ Designed bottom-up for maximum
‣ Not geared for High-Performance
performance and determinism


OpenSplice DDS

Best Performance

Fast and Furious

The Fastest Open Source Ultra-Low Stay Ahead of Market Data Rates
Latency Messaging Middleware
‣ Millions of Market Data Updates per
Second
‣ < 60 micro-seconds latency on
1GBps Ethernet
‣ Negligible inter-core latency


Stable and Reliable
Advanced Traffic Spikes-
Stability, Reliability, and Absorption Technology
Predictability, Under all Conditions
‣ Stable under the most extreme
traffic spikes
‣ Three configurable level of Spike
Absorption
‣ Guaranteed Reliability


OpenSplice DDS

Best Architecture

Multi-Core Ready Architecture


Networking Architecture
Architecture
Shared Shared
Single Copy per Node
‣ Network-channels Pack Across Topics/Applications
Memory Memory
‣ Priority bands Optimal Unmarshaling
‣ Network-partitions
‣ Multicast Groups
‣ Traffic-shaping

‣ Burst/Throughput
OpenSplice DDS OpenSplice DDS
Binding Binding
Scalability and Efficiency
Pre-emptive Network Scheduer
‣ Single shared library for applications & services Priority Scheduler
Networking Networking
Data Urgency Trafﬁc Pacing
‣ Ring-fenced shared memory segment
‣ Data urgency driven network-packing Network Channels
Priority Bands
Determinism & Safety
‣ Trafﬁc Shaping
Preemptive network-scheduler
‣ Data importance based network-channel selection
‣ Partition based multicast-group selection
‣ Managed critical network-resource


OpenSplice DDS

Best Technology
Ecosystem

Advanced Features
‣ Relational Data Modeling
‣ Object/Relational Mapping Application
MDE

‣ Event Processing Tuner

‣Queries PowerTools Object-Oriented Pub/Sub (DLRL)

DDS v1.2
‣Continuous Queries Content
Ownership Persistence
Subscription
DBMS
‣Join/Projections Minimum Proﬁle
Connectors

‣ High Performance Caching
Real-Time Pub/Sub(DCPS)

DDSI v2.1
Spike Absorber

‣ Persistency SOAP Real-Time Interoperable Wire
Secure Networking
Networking Protocol (DDSI)

‣ Security Connectors Networking Technology

‣ Development Tools
UDP/IP


High Performance Caching
‣ Fully Distributed Caching Technology OpenSplice Cache

‣ Supports SQL Queries on local Key Value
OpenSplice Cache
OpenSplice Cache
Cache
Key Value
Disk
Key Value
Key Value

‣ Support for Continuous SQL Queries
Key Value
Key Value
Key Value
Key Value Disk
Disk
Key Value
Key Value

on the whole distributed Cache Key Value
Key Value

‣ High-Performance, ultra-low-latency OpenSplice DDS

communication engine based on OpenSplice Cache
OpenSplice Cache

OpenSplice DDS Key Value
Key Value Key Value
Disk

‣ Fully exploits Data Locality and
Key Value Key Value
Disk
Key Value Key Value
Key Value

Prefetching

OpenSplice DDS

Best of Open Source

OpenSplice DDS is Open Source

OpenSplice DDS
‣ Product reorganized into Editions
‣ Community Edition
‣ Compact Edition
‣ Professional Edition
Enterprise Ed.
‣ Enterprise Edition

‣ Product Editions provide a a growing set of Professional Ed.
functionality to address the needs of
increasingly more sophisticated users Compact Ed.

‣ The Community Edition is Open Source
Community Ed.
‣ Compact, Enterprise and Professional Edition
are available only through Commercial
Subscriptions


A Feature Rich Community Edition

OpenSplice DDS
AMQP Delivering Performance, Openness, and Freedom

Application

Application

Data Local Reconstruction Layer (DLRL)

DDS v1.2
Content
Ownership Durability
Subscription
API (?) Minimum Proﬁle

Data Centric Publish/Subscribe (DCPS)

DDSI v2.1
AMQP v0.10

Real-Time Publish/Subscribe Protocol
Advanced Message Queuing Protocol

DDS Interoperability Wire Protocol
Interoperability Wire Protocol

UDP/IP
(TCP, SCTP, UDP)/IP


OpenSplice DDS

Best Way of
Spending Your Money

Financial Advantages
OpenSplice DDS subscriptions provides key advantages:

License vs. Subscription
Community Edition License Subscription

‣ Free like free speech, and 100

‣ Free like free beer! 75

50

Commercial Editions 25

‣ Cash/Flow Friendly Y1 0
Y2
Y3
Y4

‣ Lower Total Cost of Ownership (TCO)
Y5

‣ Flexible model to tune needs vs. subscription

OpenSplice DDS

Hottest Technology

DDS Adoption
Defense
‣ Combat Management Systems
‣ Tactical Systems
‣ Communication Systems
‣ Radar Processing

‣ Submarine Systems
‣ Future Combat Systems
SCADA/Utilities
‣ Industrial Automation
‣ Power Grids (Smart Grids)
‣ Complex Telemetry (e.g. NASA Rocket Launch
System)

DDS Adoption
Transportation
‣ Air Traffic Management/Control
‣ Metropolitan Traffic Management
‣ Underground Metropolitan Transportation

Financial Services
‣ Automated Trading
‣ Market Data Platforms
‣ Compliance Systems
‣ Risk Management


Customers

Nokia


Concluding Remarks
‣ OpenSplice DDS is the ideal fit for addressing
OpenSplice DDS
several of the data distribution and management
challenges faced by financial firms, such as: Delivering Performance, Openness, and Freedom
‣ Market Data Distribution
‣ High Performance Caching Enterprise Ed.
‣ Real-Time Business Intelligence

‣ OpenSplice DDS has great performance on Professional Ed.
standard COTS HW and its architecture is
designed for exploiting at best high-end multi- Compact Ed.
cores such as the Intel XEON 5500
‣ OpenSplice DDS is Commercially Supported Open
Community Ed.
Source Software

Embrace Performance, Openness and Freedom. Join the
OpenSplice DDS Community at www.opensplice.org

Online Resources

http://twitter.com/acorsaro/
http://www.opensplice.com/

emailto:opensplicedds@prismtech.com

http://bit.ly/1Sreg http://opensplice.blogspot.com

http://www.dds-forum.org

http://portals.omg.org/dds
http://www.youtube.com/OpenSpliceTube


Upcoming Webcasts on April 28

An Introduction to OpenSplice DDS Open Source
OpenSplice | DDS Webcast Series

April 28, 2009

Why Should I Attend?
• Learn everything about the OpenSplice DDS Forge
• Discover the Roadmap and Plans for OpenSplice DDS
• Understand how you might contribute to incubator projects


OpenSplice DDS Delivers High Performance Pub/Sub

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie OpenSplice DDS Delivers High Performance Pub/Sub

Ähnlich wie OpenSplice DDS Delivers High Performance Pub/Sub (20)

Mehr von Angelo Corsaro

Mehr von Angelo Corsaro (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

OpenSplice DDS Delivers High Performance Pub/Sub