The Open Source Messaging Powering Wall Street

OpenSplice DDS
Delivering Performance, Openness, and Freedom

Angelo Corsaro, Ph.D.
Chief Technology Officer
OMG RTESS and DDS SIG Co-Chair The Open Source Messaging
angelo.corsaro@prismtech.com

Accelerating Wall Street

Agenda

‣ The Big News
‣ What is OpenSplice DDS
‣ DDS vs AMQP

Proprietary Information - Distribution without Expressed Written Permission is Prohibited.
‣ Why OpenSplice DDS?
‣ Concluding Remarks

© 2009, PrismTech. All Rights Reserved

OpenSplice DDS

April 20th, 2009
OpenSplice|DDS Goes Live!
:: http://www.opensplice.org

OpenSplice DDS

High Performance
Pub/Sub

High Performance Pub/Sub
The right data, at the right place, at the right time
‣ Fully distributed, Peer-to-Peer -- All the Time.
Communication
‣ No Single Point of Failure Publisher
Subscriber

‣ No Single Point of Bottleneck

‣ Multicast-enabled Brokers Subscriber
‣ High performance and highly Publisher

scalable
‣ High availability Publisher
‣hot-swap
Subscriber

‣hot-hot architecture

OpenSplice DDS

Data-Centric
Pub/Sub

Data-Centric Pub/Sub
‣ Data-Centric Features are built-in and
don’t rely on an external DBMS
‣ Providing thus performance, scalability,
and availability
‣ Distributed Relational Data Model Subscriber
Publisher DBMS
‣ Local Queries B

‣ Continuous Queries / Content Based

m

Subscriptions A F

Subscriber
‣ Windows
Publisher J
D C

‣ Object/Relational Mapping K
E

‣ Support for a subset of SQL-92
Publisher
Subscriber

Perfect Blend of Data-Centric and Real-Time
Publish/Subscribe Technologies


Subscriber
Publisher

B

m

A F

Subscriber
Publisher J
D C

K
E

Publisher
Subscriber


Subscriber
Publisher B D J

D C
E K
A
B

A B

m
Subscriber
D C
Publisher A F

J
J A F
D C
K

A F K
E

Subscriber
Publisher D
J

D K
E

E

Topics and Data-Centric Pub/Sub
‣ Topics. Unit of information exchanged between Topic
Publisher and Subscribers.

‣ Data Types. Type associated to a Topic must
be a structured type expressed in IDL
struct TempSensor {
int tID;
Topic Type
‣ Topic Instances. Key values in a datatype float temp;

float humidity;
uniquely identify a Topic Instance (like rows in };
#pragma keylist TempSensor tID
table)
TempSensor
‣ Content Awareness. SQL Expressions can be tID temp humidity
used to do content-aware subscriptions, Instances
1 21 62
queries, joins, and correlate topic instances 2 27 78
3 25.5 72.3

y
idit idit
y ity SELECT * FROM TempSensor t
um Samples mid
h um hu
62 h
65 63 WHERE t.temp > 25
p p
tem tem
p tem
21 22
tID 23 tID tID temp humidity
tID
1 1
1 2 27 78
3 25.5 72.3
t1 t2 t3 Time

Distributed Relational Information Modeling

‣ Topic Keys can be used to identify instances as well as relationships
‣ Relationships can be navigated by relying on a subset of SQL 92
‣ One-to-many relationships can be captured using foreign keys
‣ Many-to-many relationships need to be modeled using a topics

‣ Keys can be represented by an arbitrary number of Topic fields

TempSensor Room
Floor
tID: int rID: int fID: int
temp: float width: float level: int
humidity: float height: float open: int
rID: int length: float
fID: int


Object/Relational Mapping
TempSensor
temp: float
humidity: float
getTemp()

‣ Arbitrary object getHumidity() * Room Floor
1 width: float level: int
height: float open: int
reconstructions Conditioner length: float * 1 climate: int
temp: float 1 climate: int set_status()

‣ Automatic Relationships humidity: float * set_climate(); set_climate()

setTemp() get_level()
setHumidity()
Management setFanLevel()
start()

‣ Inheritance stop()
pause():

‣ Local Operations TConditioner
cID: int TTempSernsor

‣ Local/Distributed State temp: float
humidity: float
fan_level: int TRoom
tID: int
temp: float
humidity: float
status: int rID: int rID: int
rID: int width: float
height: float
length: float TFloor
fID: int fID: int
level: int
open: int


Data Centric Publish/Subscribe
Publishers Subscribers
TempSensor TempSensor
tID temp humidity tID temp humidity
1 18 60 1 18 60
2 22 75
3 21 71

struct TempSensor {
TempSensor int tID;
float temp; TempSensor
tID temp humidity
float humidity; tID temp humidity
2 22 75 };
2 22 75
#pragma keylist TempSensor tID
3 21 71

SELECT * FROM TempSensor t
WHERE s.temp > 20
s.t
TempSensor ID
==
tID temp humidity
Fully Distributed Global 1 TempSensor
3 21 71 Data Space tID temp humidity
1 18 60


Data Centric Publish/Subscribe
Publishers Subscribers
TempSensor TempSensor
tID temp humidity tID temp humidity
1 21 62 1 21 62
2 22 75
3 21 71

struct TempSensor {
TempSensor int tID;
float temp; TempSensor
tID temp humidity
float humidity;
2 22 75 tID temp humidity
};
#pragma keylist TempSensor tID 1 21 62
2 22 75
3 21 71

SELECT * FROM TempSensor t
s.t WHERE s.temp > 20
TempSensor ID
==
tID temp humidity
Fully Distributed Global 1
TempSensor
3 21 71 Data Space
tID temp humidity
1 21 62


Processing Content & Structure Complex Event Processing...

‣ Local Queries OpenSplice DDS some of the most useful features
found in Complex Event Processing platforms!
‣ Continuous Queries (Content-based Subscriptions)
‣ Topics Joins & Projections
‣ Events windows (via History QoS)

tID temp humidity S1
3 21.5 72.3
tID temp humidity {iID = 1}
2 20 78
tID temp humidity
1 21 62
S2
P {temp > 21 | humidity > 75}

S3
{ temp <16 }


Processing Content & Structure Complex Event Processing...

‣ Local Queries OpenSplice DDS some of the most useful features
found in Complex Event Processing platforms!
‣ Continuous Queries (Content-based Subscriptions)
‣ Topics Joins & Projections
‣ Events windows (via History QoS)

tID temp humidity
1 21 62

S1
{iID = 1} tID temp humidity
2 20 78
3 21.5 72.3

S2
P {temp > 21 | humidity > 75}

S3
{ temp <16 }


OpenSplice DDS

QoS-Enabled
Pub/Sub

QoS Model
‣ QoS-Policies are used to control relevant Type Matching
properties of OpenSplice DDS entities, QoS matching

such as: QoS QoS QoS QoS QoS

‣ Temporal Properties Topic

‣ Priority
Name
Publisher Subscriber

... DataWriter writes Type reads DataReader
...
‣ Durability
...
‣ Availability DataWriter writes Type reads DataReader

‣ ... Name

‣ Some QoS-Policies are matched based on
Topic

a Request vs. Offered Model thus QoS- QoS QoS QoS

enforcement

‣ Publications and Subscriptions match only if the declared vs. requested QoS are compatible
‣ e.g., it is not possible to match a publisher which delivers data unreliably with a subscriber which requires reliability

Sample QoS Policies
QoS Policy Applicability RxO Modiﬁable QoS matching

DURABILITY T, DR, DW Y N Data Availability
QoS QoS QoS

DURABILITY SERVICE T, DW N N QoS
Topic
Name
QoS

DataWriter writes Type reads DataReader

Subscriber
LIFESPAN T, DW - Y Publisher ...
Type reads DataReader
T, DR, DW N N
writes
HISTORY
DataWriter

Name
Topic
PRESENTATION P, S Y N Data Delivery
T, DR, DW Y N
QoS QoS QoS
RELIABILITY
QoS matching
PARTITION P, S N Y

DESTINATION ORDER T, DR, DW Y N

OWNERSHIP T, DR, DW Y N ‣ Rich set of QoS allow to configure
OWNERSHIP DW - Y
STRENGTH several different aspects of data
DEADLINE T, DR, DW Y Y Data Timeliness availability, delivery and timeliness
LATENCY BUDGET T, DR, DW Y Y

TRANSPORT PRIORITY T, DW - Y
‣ QoS can be used to control and
optimize network as well as
TIME BASED FILTER DR - Y Resources
computing resource
RESOURCE LIMITS T, DR, DW N N

USER_DATA DP, DR, DW N Y Conﬁguration
TOPIC_DATA T N Y
GROUP_DATA P, S N Y


Mapping QoS Which properties does QoS controls?

TimeBasedFilter Deadline

Data
Throughput
Latency

LatencyBudget TransportPriority

Control over Latency/Throughput tradeoff
Control over data latency
Control over data priority



TimeBasedFilter Deadline History

Data Data
Throughput Lifespan Durability
Latency Availability

Ownership
LatencyBudget TransportPriority Ownership
Strength

Control over data queueing
Control over data persistency
Control over data sources hot-swap



TimeBasedFilter Deadline History

Data Data
Throughput Lifespan Durability
Latency Availability

Ownership
LatencyBudget TransportPriority Ownership
Strength

Control over data queueing
Control over data persistency
Control over data sources hot-swap

Reliability
OpenSplice DDS provides programmatic
Destination
QoS-driven support for configuring the most
Presentation Data Delivery
Order important properties of data distribution!
Control over data distribution reliability
Control over data ordering
Control over presentation


OpenSplice DDS

Zero-Conf
Pub/Sub

Zero-Conf Pub/Sub
Code
‣ Dynamic Discovery. Publisher,
Subscriber, and Topics are dynamically
Bloat
discovered
‣ Separation of Concerns. Applications

are agnostic from networking details.
Thus you can re-deploy the same
business logic with different network,
partitioning, configurations
‣ Tool Based. All configuration is tool
assisted and happens outside of your
application


OpenSplice DDS

Standard-Based
Pub/Sub

The OMG Data Distribution Service
Data Distribution Service for Real-Time Systems
Application
‣ Leap forward in Topic-based Publish/Subscribe
Middleware state of the art Object/Relational Mapping

‣ Carefully specified to allow very high performance, Data Local Reconstruction Layer (DLRL)

DDS v1.2
scalable, predictable and high-availability
Content

implementations Ownership Durability
Subscription

‣ Language Independent, OS and HW architecture Minimum Proﬁle

independent Data Centric Publish/Subscribe (DCPS)

DDSI v2.1
Fully Standardized Solution Real-Time Publish/Subscribe Protocol
‣ Data Distribution Service (DDS) for Real-Time DDS Interoperability Wire Protocol
Systems v1.2
‣ RTPS, DDS Interoperability Wire Protocol UDP/IP

The OMG Data Distribution Service satisfies the most challenging information dissemination requirements
across a wide set of application domains, ranging from multi-board systems to system-of-systems

DDS!Recommendations is churning…
The infrastructure evolution cycle
– New -> Emerging -> Standard -> Commodity
Increasingly Mandated/Recommended by Administrations
– Middleware is emerging as OS declines
! …DDS is maturing…
‣ US Navy: Open Architecture
– OMG focus
– Wire spec

‣ DISR/DISA: Net-centric Systems
– Tools
– Enterprise integration
– Multiple products fielded
‣ EuroControl:– Deployed applications!
Air Traffic Control Center Operational
Interoperability
! …and adoption is on the rise
– Navy
– DISR
‣ QinetiQ: Recommending DDS for VSI
– FCS/SoSCOE

Scope of Standardization
AMQP OMG DDS
Application
Application

Data Local Reconstruction Layer (DLRL)

DDS v1.2
Content
Ownership Durability
Subscription
API (?) Minimum Proﬁle

Data Centric Publish/Subscribe (DCPS)

DDSI v2.1
AMQP v0.10

Advanced Message Queuing Protocol Real-Time Publish/Subscribe Protocol

Interoperability Wire Protocol DDS Interoperability Wire Protocol

(TCP, SCTP, UDP)/IP UDP/IP


AMQP OMG DDS
Application
Application


DDS v1.2
Content
Subscription


DDSI v2.1
AMQP v0.10

Wire Protocol Standard



AMQP OMG DDS
Application
Application


DDS v1.2
Content
Subscription
Pub/Sub Standard API


DDSI v2.1
AMQP v0.10




AMQP OMG DDS
Application
Application
Object/Relational Mapping Object Relational Mapping
Standard API


DDS v1.2
Content
Subscription
Pub/Sub Standard API


DDSI v2.1
AMQP v0.10




Architectural Style

AMQP OMG DDS
Subscriber Subscriber
Publisher Publisher

Broker Broker

Publisher
Broker Subscriber
Publisher
Broker Subscriber

Broker
Broker

Publisher Publisher
Subscriber Subscriber


Key Differences in Summary
AMQP DDS
‣ Brokered Communication ‣ Peer-to-Peer Communication
Architecture Architecture

‣ Wire Protocol Standard ‣ Wire Protocol + API Standard

‣ Interoperability across AMQP ‣ Interoperability across DDS implementations
implementations
‣ Application Portability across DDS
Implementation

‣ Header Based Routing ‣ Content-Based Filtering/Quering

‣ Not geared for High-Performance ‣ Designed bottom-up for maximum
performance and determinism


OpenSplice DDS

Best Performance

Fast and Furious

The Fastest Open Source Ultra-Low Stay Ahead of Market Data Rates
Latency Messaging Middleware
‣ Millions of Market Data Updates per
Second
‣ < 60 micro-seconds latency on
1GBps Ethernet
‣ Negligible inter-core latency


Stable and Reliable
Advanced Traffic Spikes-
Stability, Reliability, and Absorption Technology
Predictability, Under all Conditions
‣ Stable under the most extreme
traffic spikes
‣ Three configurable level of Spike
Absorption
‣ Guaranteed Reliability


OpenSplice DDS

Best Architecture

Multi-Core Ready Architecture


Networking Architecture
Architecture
‣ Network-channels Shared Single Copy per Node Shared
Pack Across Topics/Applications
‣ Priority bands Memory Optimal Unmarshaling Memory
‣ Network-partitions
‣ Multicast Groups
‣ Traffic-shaping

‣ Burst/Throughput
OpenSplice DDS OpenSplice DDS
Binding Binding
Scalability and Efficiency
Pre-emptive Network Scheduer
‣ Single shared library for applications & services Networking Priority Scheduler
Networking
Data Urgency Trafﬁc Pacing
‣ Ring-fenced shared memory segment
‣ Data urgency driven network-packing Network Channels
Priority Bands
Determinism & Safety
‣ Preemptive network-scheduler Trafﬁc Shaping

‣ Data importance based network-channel selection
‣ Partition based multicast-group selection
‣ Managed critical network-resource


OpenSplice DDS

Best Technology
Ecosystem

Advanced Features
‣ Relational Data Modeling
‣ Object/Relational Mapping MDE Application

‣ Event Processing Tuner

‣Queries PowerTools Object-Oriented Pub/Sub (DLRL)

DDS v1.2
‣Continuous Queries DBMS
Ownership Persistence
Content
Subscription

‣Join/Projections Connectors
Minimum Proﬁle

‣ High Performance Caching
Real-Time Pub/Sub(DCPS)

DDSI v2.1
Spike Absorber

‣ Persistency SOAP
Secure Networking
Real-Time
Networking
Interoperable Wire
Protocol (DDSI)

‣ Security Connectors Networking Technology

‣ Development Tools
UDP/IP


High Performance Caching
‣ Fully Distributed Caching Technology OpenSplice Cache

‣ Supports SQL Queries on local Key Value
OpenSplice Cache
OpenSplice Cache
Cache
Key Value
Disk
Key Value
Key Value

‣ Support for Continuous SQL Queries
Key Value
Key Value
Key Value
Key Value Disk
Disk
Key Value
Key Value

on the whole distributed Cache Key Value
Key Value

‣ High-Performance, ultra-low-latency OpenSplice DDS

communication engine based on OpenSplice Cache
OpenSplice Cache

OpenSplice DDS Key Value
Key Value
Key Value
Disk

‣ Fully exploits Data Locality and
Key Value Key Value
Disk
Key Value Key Value
Key Value

Prefetching

OpenSplice DDS

Best of Open Source

OpenSplice DDS is Open Source
‣ Product reorganized into Editions
‣ Community Edition OpenSplice DDS
‣ Compact Edition
‣ Professional Edition
Enterprise Ed.
‣ Enterprise Edition

‣ Product Editions provide a a growing set of Professional Ed.
functionality to address the needs of
increasingly more sophisticated users Compact Ed.

‣ The Community Edition is Open Source
Community Ed.
‣ Compact, Enterprise and Professional Edition
are available only through Commercial
Subscriptions


A Feature Rich Community Edition

OpenSplice DDS
AMQP Delivering Performance, Openness, and Freedom

Application

Application


DDS v1.2
Content
Subscription


DDSI v2.1
AMQP v0.10





OpenSplice DDS

Best Way of
Spending Your Money

Financial Advantages
OpenSplice DDS subscriptions provides key advantages:

License vs. Subscription
Community Edition License Subscription

‣ Free like free speech, and 100

‣ Free like free beer! 75

50

Commercial Editions 25

‣ Cash/Flow Friendly Y1
Y2
Y3
0
Y4

‣ Lower Total Cost of Ownership (TCO)
Y5

‣ Flexible model to tune needs vs. subscription

OpenSplice DDS

Hottest Technology

DDS Adoption
Defense
‣ Combat Management Systems
‣ Tactical Systems
‣ Communication Systems
‣ Radar Processing

‣ Submarine Systems
‣ Future Combat Systems
SCADA/Utilities
‣ Industrial Automation
‣ Power Grids (Smart Grids)
‣ Complex Telemetry (e.g. NASA Rocket Launch
System)

DDS Adoption
Transportation
‣ Air Traffic Management/Control
‣ Metropolitan Traffic Management
‣ Underground Metropolitan Transportation

Financial Services
‣ Automated Trading
‣ Market Data Platforms
‣ Compliance Systems
‣ Risk Management


Customers

Nokia


Concluding Remarks
‣ OpenSplice DDS is the ideal fit for addressing
several of the data distribution and management
challenges faced by financial firms, such as:
OpenSplice DDS
‣ Market Data Distribution
‣ High Performance Caching Enterprise Ed.
‣ Real-Time Business Intelligence

‣ OpenSplice DDS has great performance on Professional Ed.
standard COTS HW and its architecture is
designed for exploiting at best high-end multi- Compact Ed.
cores such as the Intel XEON 5500
‣ OpenSplice DDS is Commercially Supported Open
Source Software Community Ed.

Embrace Performance, Openness and Freedom. Join the
OpenSplice DDS Community at www.opensplice.org

Online Resources

http://www.opensplice.com/ http://twitter.com/acorsaro/

emailto:opensplicedds@prismtech.com

http://bit.ly/1Sreg http://opensplice.blogspot.com

http://www.dds-forum.org

http://www.youtube.com/OpenSpliceTube http://portals.omg.org/dds


Upcoming Webcasts on April 28

An Introduction to OpenSplice DDS Open Source
OpenSplice | DDS Webcast Series

April 28, 2009

Why Should I Attend?
• Learn everything about the OpenSplice DDS Forge
• Discover the Roadmap and Plans for OpenSplice DDS
• Understand how you might contribute to incubator projects


The Open Source Messaging Powering Wall Street

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (20)

Ähnlich wie The Open Source Messaging Powering Wall Street

Ähnlich wie The Open Source Messaging Powering Wall Street (20)

Mehr von Angelo Corsaro

Mehr von Angelo Corsaro (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

The Open Source Messaging Powering Wall Street