How to Troubleshoot Apps for the Modern Connected Worker
RTI Data-Distribution Service (DDS) Master Class 2011
1. DDS: A Next-Generation Approach to Building Distributed Real-Time Systems 2011 Masterclass Gerardo Pardo-Castellote, Ph.D. Co-chair OMG DDS SIG CTO, Real-Time Innovations [email_address] http://www.rti.com
91. How to Get Data? (Listener-Based) // Listener creation and attachment Listener listener = new MyListener(); reader->set_listener(listener); // Listener code MyListener::on_data_available( DataReader reader ) { TextSeq received_data; SampleInfoSeq sample_info; TextDataReader reader = TextDataReader::narrow(reader); treader->take( &received_data, &sample_info, …) // Use received_data printf(“Got: %s”, received_data[0]->contents); }
92. How to Get Data? (WaitSet-Based) // Creation of condition and attachement Condition foo_condition = treader->create_readcondition(…); waitset->add_condition(foo_condition); // Wait ConditionSeq active_conditions; waitset->wait(&active_conditions, timeout); // Wait returns when there is data (or timeout) FooSeq received_data; SampleInfoSeq sample_info; treader->take_w_condition (&received_data, &sample_info, foo_condition); // Use received_data printf(“Got: %s”, received_data[0]->contents);
93.
94.
95.
96.
97.
98.
99. IDL vs. XML: IDL Example struct MemberStruct{ short sData; } typedef MemberStructType; //@top-level false
100. IDL vs. XML: XML Example <? xml version ="1.0“ encoding ="UTF-8"?> < types xmlns:xsi ="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation ="../rti_dds_topic_types.xsd"> < struct name ="MemberStruct" topLevel ="false"> < member name ="sData“ type ="short"/> </ struct > < typedef name ="MemberStructType" type ="nonBasic“ nonBasicTypeName ="MemberStruct“ topLevel ="false"/> </ types >
101. IDL vs. XSD: XSD Example <? xml version ="1.0" encoding ="UTF-8"?> < xsd:schema xmlns:xsd ="http://www.w3.org/2001/XMLSchema" xmlns:dds ="http://www.omg.org/dds" xmlns:tns ="http://www.omg.org/IDL-Mapped/" targetNamespace ="http://www.omg.org/IDL-Mapped/"> < xsd:import namespace ="http://www.omg.org/dds" schemaLocation ="rti_dds_topic_types_common.xsd"/> < xsd:complexType name ="MemberStruct"> < xsd:sequence > < xsd:element name ="sData" minOccurs ="1" maxOccurs ="1" type ="xsd:short"/> </ xsd:sequence > </ xsd:complexType > <!-- @topLevel false --> < xsd:complexType name ="MemberStructType"> < xsd:complexContent > < xsd:restriction base ="tns:MemberStruct"> < xsd:sequence > < xsd:element name ="sData" type ="xsd:short" minOccurs ="1" maxOccurs ="1"/> </ xsd:sequence > </ xsd:restriction > </ xsd:complexContent > </ xsd:complexType > <!-- @topLevel false --> </ xsd:schema >
137. Pub/Sub middleware (DDS) Application Secure OS Secure Transport DDS Application Secure DDS Global Data Credential Mgmt Authentication Access Control Data Tagging Encryption DDS Application DDS Application ? ? ? ?
138. App. Other DDS System Secure DDS middleware Authentication Plugin Access Control Plugin Data Encryption Plugin Data Tagging Plugin Crypto Module (e.g. TPM ) Secure Transport (e.g. TLS) CS application component certificates Kernel Policies Network Driver Network Encrypted Data TAGS Other DDS System Other DDS System App. App. Secure Kernel (e.g. SE Linux, MILS) ? Data cache Protocol Engine DDS Entities ? 1 2 3 4 5 6 7 8 9
139.
140.
141.
142.
143.
144.
145.
146.
147.
148.
149.
150.
151.
Hinweis der Redaktion
Time: 45 minutes Title: The Data Distribution Service (DDS) Standard: A Next-Generation Approach to Building Distributed Real-Time Systems Abstract: DDS has been adopted worldwide by major air force, army, marine and navy programs as an open architecture standard for integrating real-time tactical systems with each other and with enterprise applications such as command and control systems. This breakout will introduce the DDS standard and show how it provides a net-centric, service-oriented approach to meeting the messaging and integration requirements of mission-critical embedded systems. ** Booth demo ** RTI will be demonstrating its real-time publish/subscribe middleware based on the Data Distribution Service (DDS) standard. DDS dramatically reduces software lifecycle costs by making it easy to develop, integrate and scale distributed real-time applications. DDS applications are loosely coupled and can communicate seamlessly across platforms, programming languages and network transports (including shared memory, backplane, LAN, WAN, wireless and satellite links). Supported operating systems include VxWorks, VxWorks MILS 2.0, Linux, Windows, Solaris and AIX. A small-footprint version is available for systems that require DO-178B certification.
Make the point that this precept has been fundamentally understood by the GVA, but the concept is empowering as a way to integrate larger systems of systems into a net-centric whole. In fact the data model of the vehicle in def-stan 23-09 can be assessed to determine what parts of its data set could and should be communicated to the wider net-centric environment. 2 nd bullet point – you can note: Otherwise how would crusty old Generals add the value they do in leading combat situations? (Joke – up to you if you use this)
Last slide was at one moment in time. Now, longer-term view… Example: with 12 apps, effort is order 12 vs. order 144: order of magnitude savings
Mostly for technical folks
DDS provides an infrastructure for integrating real-time applications. It also facilitates integrating real-time applications with non-real-time (enterprise) applications, such as command and control systems.
Work on the standard began in 2001 and version 1.0 was formally adopted in December 2004. RTI released the first commercial solution to comply with the standardized API in 2005. Implementations: RTI* PrismTech/Thales* MilSOFT* Twin Oaks* OpenDDS Gallium/Kongsberg Boeing SoSCOE *Claim support for wire protocol OCERA ORTE is RTPS only. http://www.ocera.org/download/components/WP7/orte-0.3.1.html
Applications that want to contribute information to the Global Data Space can declare their intent to publish the information. Applications that want to access portions of the Global Data Space can declare their intent to subscribe to the information Decoupling in several dimensions: - Space (location): Each side does not need to know the location of the other side. They publish/subscribe to the shared “global data space” - Redundancy: It is possible for the same data to be subscribe by multiple nodes, or to be written by multiple nodes. This is all managed transparently by the infrastructure. - Time: The reception of data does not need to be synchronous with the writing. A subscriber may, if so configured, receive data that was written even before the subscriber joined the network. - Platform: Applications do not have to worry about data representation, processor architecture, Operating System, or even programming on the other side. It is possible for example to publish from a real-time node using the C language and subscribe from a Linux node running Java. Each side is isolated from the details of the other. Mechanisms in place to allow access to data only by specific applications / nodes.
Applications that want to contribute information to the Global Data Space can declare their intent to publish the information. Applications that want to access portions of the Global Data Space can declare their intent to subscribe to the information Decoupling in several dimensions: - Space (location): Each side does not need to know the location of the other side. They publish/subscribe to the shared “global data space” - Redundancy: It is possible for the same data to be subscribe by multiple nodes, or to be written by multiple nodes. This is all managed transparently by the infrastructure. - Time: The reception of data does not need to be synchronous with the writing. A subscriber may, if so configured, receive data that was written even before the subscriber joined the network. - Platform: Applications do not have to worry about data representation, processor architecture, Operating System, or even programming on the other side. It is possible for example to publish from a real-time node using the C language and subscribe from a Linux node running Java. Each side is isolated from the details of the other. Mechanisms in place to allow access to data only by specific applications / nodes.
Applications that want to contribute information to the Global Data Space can declare their intent to publish the information. Applications that want to access portions of the Global Data Space can declare their intent to subscribe to the information Decoupling in several dimensions: - Space (location): Each side does not need to know the location of the other side. They publish/subscribe to the shared “global data space” - Redundancy: It is possible for the same data to be subscribe by multiple nodes, or to be written by multiple nodes. This is all managed transparently by the infrastructure. - Time: The reception of data does not need to be synchronous with the writing. A subscriber may, if so configured, receive data that was written even before the subscriber joined the network. - Platform: Applications do not have to worry about data representation, processor architecture, Operating System, or even programming on the other side. It is possible for example to publish from a real-time node using the C language and subscribe from a Linux node running Java. Each side is isolated from the details of the other. Mechanisms in place to allow access to data only by specific applications / nodes.
Start first window. Publish one instance of each shape (i.e., topic). Start second window. Subscribe to all three shapes. Point out: automatic discovery, peer-to-peer communication. Illustrates one-to-one communication. Start third window. Subscribe to all three shapes. Illustrates one-to-many. Start fourth window. Publish one of each shape, using different colors than are already being published. Illustrates many-to-many. Click “Delete All” and then exit the first window. Notice how well-suited DDS is for dynamic and ad hoc systems. Applications could come and go without impacting other applications. This also provides fault tolerance. Also see how this makes it easy to insert new applications and technology into already deployed systems. Note: keep the three other windows running. Will use them for showing content filter and time-based filter later.
In one of the two subscribing windows, delete all of the subscriptions and then subscribe to one shape with a content-filtered topic and another shape with a time based filter. Points: Applications have fine-grained control over which data is received. This optimizes performance and reduces/simplifies application logic Filtering has no impact on either the publisher or other subscriber. It is very loosely coupled. Every application can specify its own requirements. Delete all of the publications in the publishing window and all of subscriptions in the non-filtering window. Publish a shape with Durability, Reliability, History=250 and Deadline=1000. (Publish a shape that is being subscribed with a filter.) In the window with no subscriptions, enable “Show Reliability” and subscribe to the shape being published with Durability, Reliability, History=250 and Deadline=2000. Points: Late joining applications can get the state they need to start processing, including historic data that may be necessary to “prime” algorithms. Even though the QoS of the publisher was changed, the subscriber that was running with the old QoS is still getting data. Subscribers can get data as long as the request QoS is less than or equal to the published QoS. In the subscribing window with Deadline QoS set, view the Output pane and make sure to scroll to the bottom (<Control><End>). In the publishing window, click-hold on the shape so that it won’t be published. Note the missed deadline message. Point: Applications are notified if timing constraints are not being met so that they can take corrective action and won’t do anything erroneously.
The first thing to notice is that the knowledge of your data model that was associated with the data stream in the data-centric technology disappears when you use a message-centric technology. That makes it much harder to develop a generic component such as the Web Integration Service, which much transform arbitrary data types to and from XML, downsample data by based on content, etc. First message arrives. It has the same structure as we saw before, except without a known type definition, the type information must be embedded within the message itself, significantly increasing its size. The second message arrives. It’s in a totally different format than the first! This one is just a blob of binary-encoded data. Maybe the consuming application understands how to decode it and maybe not. Each application connected to the network will have expectations about the formats of the messages it receives. But a messaging infrastructure can’t support those expectations, so they have to be enforced by an organizational policy. I write up a Word document that describes how you should format your messages and email it to you, and you have to follow my instructions. If you make a mistake, we’ll have to debug it at integration time. In a data-centric approach, data type enforcement is built in: developers work with typed objects in their programming languages, errors are detected when the code is compiled before it’s ever deployed, and runtime mismatches that do occur are detected automatically by the middleware. How do I describe a content-based filter on a binary blob? How do I transform it into another format? How do I map it into a database? The third message arrives. It’s in yet a third format: a plain text string. Because the messaging system doesn’t have any concept of object lifecycle, each system has to define its own ad hoc system of sentinels: “create” messages, “dispose” messages, etc. More work, and it makes it much more difficult to leverage something you’ve built for one project on the next project. By comparison, Web Integration Service takes advantage of the built-in lifecycle support in DDS – you saw that when tracks were marked with “X” or “?”. And without any knowledge of your objects or their lifecycle, a messaging infrastructure can only support qualities of service that make sense across an entire topic: for example time-to-live (“lifespan” in language of DDS).
From the beginning, the data stream is associated with the schema of the data that will be propagated on that stream. Your applications already have some expectations; if you express those to a data-centric infrastructure, it can help you. For example, you can use this schema to automatically transform data into other formats. (This is how the Routing Service and Web Integration Service work.) The infrastructure can also dissect your data to filter on content (for example “give me updates where x > 5”). “ Key” means “this field establishes the identity of a unique object.” Like the key in a relational database table. In DDS, can be any number of fields of any type(s). New track you’ve never seen before. Notice that since type is already known, only need to send field values, not field names or types. Update to a track you’ve already seen Another new track – notice that the key is different A track you’ve seen before has gone away
Effector is responsible for transition algo. In this case, x y z. But could have chosen “curved” path x … z that never reached y.
Time: 45 minutes Title: The Data Distribution Service (DDS) Standard: A Next-Generation Approach to Building Distributed Real-Time Systems Abstract: DDS has been adopted worldwide by major air force, army, marine and navy programs as an open architecture standard for integrating real-time tactical systems with each other and with enterprise applications such as command and control systems. This breakout will introduce the DDS standard and show how it provides a net-centric, service-oriented approach to meeting the messaging and integration requirements of mission-critical embedded systems. ** Booth demo ** RTI will be demonstrating its real-time publish/subscribe middleware based on the Data Distribution Service (DDS) standard. DDS dramatically reduces software lifecycle costs by making it easy to develop, integrate and scale distributed real-time applications. DDS applications are loosely coupled and can communicate seamlessly across platforms, programming languages and network transports (including shared memory, backplane, LAN, WAN, wireless and satellite links). Supported operating systems include VxWorks, VxWorks MILS 2.0, Linux, Windows, Solaris and AIX. A small-footprint version is available for systems that require DO-178B certification.
So, what is RTI Routing Service? In a nutshell, Routing Service provides high-performance real-time data forwarding and transformation across DDS Domains, Communities of Interests and Wide Area Networks (WANs) including firewall and NAT traversal. With its plug-in architecture to accommodate new transports and bridging functionality, it has been designed from the ground up to meet custom integration requirements, such as: Bridging between new DDS applications and legacy systems; and Increasing data security by controlled information flows. With the help of our Services team, custom integrations can be easily created and with low risk. My colleague Gordon will explain this further in a little while. As we’re introducing our edition-based packaging, Routing Service will be available as a component of the new RTI Enterprise Edition. Later we’ll give details about how to get access to Enterprise Edition and Routing Service at a significant discount.
Hands-On: show rtiddsgen –help output
Short, easy to write Efficient, OMG standard
AUTOCOMPLETION XML friendly, easy to extend More powerful
First step to interoperate with WS Some customers have a lot of sources in XSD or WSDL
This is a great tool for seeing the organization of your distributed environment. Having the ability to change QoS settings on the fly and observing how they impact the overall environment is extremely useful Having the ability to determine “WHY” a DataWriter is not talking to a specific DataReader is also very beneficial. This has the ability to show both Node views and Topic views of the system.
Note: only one document can be specified with the string_profile variable
How it is realized http directly to the data bus
RTI is your best partner for a successful DDS deployment: Broadly proven commercial technology >11 years of commercial availability RTI is the de facto standard, with >70% market share Selected for nearly 500 unique applications – many mission critical – many deployed Industry-leading expertise and services capability We know the DDS standard better than anyone We have the most experience putting DDS to use in real applications We have a large team of senior, experienced engineers at your disposal to allow you to leverage our expertise: whether for training, consulting or as a partner in your development Corporate focus and commitment All of RTI’s technology is built on DDS – we are completely committed to it – we sold off our non-DDS business Proven, financially strong DDS business DDS is not acquired or peripheral technology that we could drop or dispose of if it became financially expedient Comprehensive infrastructure built around DDS DDS is only part of your infrastructure and application – RTI provides the most comprehensive overall solution to your real-time application and data management requirements Development tools, database integration, real-time data recording, Complex Event Processing, real-time data visualization dashboards Plus the largest set of partners for additional capabilities like modeling, high-performance transports, real-time operating systems and JVMs Superior architecture and implementation Significantly higher performance: lower latency and higher throughput with less overhead on your compute resources Much more fault tolerant and highly available, with no single points of failure and full redundancy for all services Most flexibility for supporting additional transports, legacy or other non-DDS data types, and custom discovery requirements, Best suited for resource-limited embedded systems Broadest availability across enterprise and embedded platforms Quality by design Mature, formal processes for design, development and Quality Assurance Invest in comprehensive user documentation Proof point: 98% customer satisfaction - extraordinary in any industry In summary, RTI provides: Highest performance and fault tolerance – because of our superior peer-to-peer architecture Fastest time-to-market Leveraging our training, consulting, engineering services and high-quality support organization Superior documentation Most complete infrastructure Flexible implementation Lowest risk Proven commercial technology Most experience and expertise with successful DDS deployment Quality processes and support Corporate focus and commitment to DDS