To view recording please use the below URL:
http://wso2.com/library/webinars/2015/03/esb-meets-iot-a-primer-on-wso2-enterprise-service-bus-esb/
Ishan from WSO2 and Yenlo’s Rob will discuss the use of WSO2 ESB for Internet of Things applications. Topics will be:
What WSO2 components do you need for the Internet of Things?
What deployment do you need for a large sensor network?
How do you analyze and display data?
Examples of WSO2-enabled Internet of Things solutions (e.g. Trimble’s Connected Plants)
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
WSO2 Guest Webinar - ESB meets IoT, a Primer on WSO2 Enterprise Service Bus (ESB)
1. Rob Blaauboer, Senior Consultant, Yenlo
ESB meets IoT, a primer on
the Enterprise Service Bus
March 11th 2015
Ishan Jayawardena], WSO2
2. About the presenters
2
Rob Blaauboer
Senior Consultant, Yenlo
Rob is a Senior Business Consultant and Solution Architect with more
than twenty years experience. In addition to his work he is an active
blogger working on a number of articles on the 'Internet of Things' and
a WSO2 'Getting Started with ...' series in which he talks about WSO2
components and their purpose especially aimed at non technical
readers.
Ishan Jayawardena
Senior Software Engineer, WSO2
Ishan is a Senior Software Engineer in the integration technologies
team where he mainly focuses on the WSO2 Enterprise Service Bus. In
addition to his product development efforts he has also provided
technology consulting on customer engagements, including customer
QuickStart programs focused on Enterprise Application Integration
projects.
3. 3
About Yenlo
๏ Global enterprise, founded in 2007
with an international focus on
delivering integration solutions
based on Java open source
๏ #1 in the field of Integration Solutions
๏ #1 in Managed Services for
middleware environments
๏ #1 Global Strategic Alliance partner of
WSO2
๏ WSO2 Product Support
๏ WSO2 Development
๏ WSO2 QuickStarts
๏ WSO2 Training & Certifications
๏ WSO2 24/7 Managed Services
๏ WSO2 Events
5. Agenda
5
An introduction to the WSO2 ESB
๏The need to integrate or connect systems
๏Key functions of the ESB
๏Use case Internet of Things doorlock and smart thermostat
6. Hop on the Enterprise Service Bus
6
Perhaps you have heard one of your
colleagues mentioning ‘what we need is
an ESB to tackle our integration issues’ or
read in a magazine about the wonders of
the Enterprise Service Bus. But what is it
exactly?
7. Service Oriented Architecture
6
o What is SOA?
o Architectural approach based on
discrete pieces of software
providing functionality as services
to other applications. Services are
well defined and reusable.
o ESB in the Context of SOA?
o SOA: Design/develop smaller
components as services for
reusability
o ESB: Communication and
interaction between these
services
8. The WSO2 ESB Manager works
together with other components
7
9. The ESB is like a traffic cop
6
“The Enterprise Service
Bus acts like a traffic cop,
directing communication
between systems but also
telling you to walk faster”
10. ESB in the broader picture
6
An ESB is foremost an architectural model.
Core of the model is the fact that the ESB acts
as a central place for message exchange. An
ESB is a critical component of a Service
Oriented Architecture (an architectural model
where components offer services via messages
to other components).
The fact that it is called enterprise service bus
gives an indication where the ESB can play an
important role: the Enterprise. Especially in
these environments, where a large number of
systems and applications are in use, ESB can
help organizations to enable new products and
services built on these disparate systems.
But lets not forget IoT and Cloud!
Image pnwra CC Flickr
https://www.flickr.com/photos/pnwra/
11. Monoliths
6
Originally computer systems were
designed as so called monoliths.
The system performed one or a
couple of tasks and kept its data
pretty much to itself. With the
advent of client server
architecture and the three-tier
model: presentation – logic –
data, monolithic systems became
less popular. The separation of
these tiers or layers was the
beginning of the Service Oriented
Architecture movement.
Image Brian Glanz CC Flickr
https://www.flickr.com/photos/brianglanz/
12. Point to point connections
6
At first, connections between systems were
so-called point-to-point communication. This
means that a direct connection was made
between the two communicating systems.
When the number of systems that needed to
be connected or exchange information grew,
the number of connections that need to be
maintained grew more rapidly. Because if you
want to have five systems connected to each
other you need to define 25 connections
(5^2). This becomes quickly unmanageable;
imagine an enterprise with 100 separate
systems.
A
B
C
D
E
1
2
3
4
5
13. Hub and Spoke EAI architecture
6
A better solution was needed and
that led to the enterprise application
integration (EAI) products. An
advantage over a point-to-point
connection but the hub and spoke
architecture has now proven to
become bottlenecks in the system.
The enterprise service bus is the next
logical step in connecting disparate IT
systems and programs.
14. Not only for legacy systems!
6
We already talked about the application
infrastructure that you will find in an enterprise.
Given the fact that most enterprises have been using
IT systems for the better part of 40 years it’s not
uncommon to find systems written in almost any
imaginable language (from PL/1 to Cobol to
PowerDesigner) running on in some cases obscure
hardware and operating systems.
New devices are also added, how are we going to
connect those?
15. Working with WSO2 ESB
High-level Message Flow (Programming Model)
Client Service
In Sequence
Out Sequence
Fault Seq.
1 2 3
6 5 4
!
17. Building Blocks
o Sequences
o Define logic for handling incoming (request) and outgoing
(response) messages
o Sequences list mediators in order of execution
o Mediators
o Take action on the message
o Filter, Transform, Drop, Send, Property, Payload Factory
o Endpoints
o Define external destination for a message, usually a service
o Transports
o Carry messages in a specific format
23. Tasks and events
6
Apart from this you can also use tasks and events with the
ESB. A task in WSO2 ESB allows you to run a specific piece of
code by inserting it automatically into the timeline.
Events are notifications published to any system that is
interested in this event. Even the ESB itself can listen to
events and take action when they arise. An integration
pattern facilitating this is the publish-subscribe pattern.
25. Scalable
6
In order to have a high performance and high availability
solution the ESB supports thousands of concurrent nonblocking
HTTP(s) connections per server that will allow even the most
demanding organizations to utilize the ESB. One of the examples
of high load implementations is eBay that has implemented
WSO2 ESB and has over 1 billion transactions per day. Of course
the ESB can also support organizations that have lower demands
or requirements.
26. Try the ESB yourself!
Trying the API manager is quite simple. What you need to do to try it on
your own PC is:
•Download the WSO2 ESB from the WSO2 website
•Unzip the ESB
•If you don’t have JDK installed, download it and install
•Set the JAVA_HOME parameter
•Go to the BIN directory and start the WSO2Server (bat or sh) file
•Access the consolehttps://localhost:9943/carbon
29
28. On the roadmap for ESB
ESB 4.9.0
The core foundation for a comprehensive
Integration platform as a Service (iPaaS)
» Connectors,
» Connector Store,
» multi-tenanted inbound messaging;
» integration templates
Carbon Version: 4.3.0
28
Flickr CC Jon Rawlinson
https://www.flickr.com/photos/london/
33.
Toon®
Monitor locking the frontdoor lock. If it is closed with a key
or from the app we will log in to the smart thermostat and
see what program is chosen.
If the thermostat is locked to a specific program we leave it
as is, if not we set the program to ‘Away’ turning it off.
34. What would we like to achieve?
Login Okidokey
Call lock status API
every 5 minutes
Login Toon (oAuth)
Door
closed?
No
Yes
Get Thermostat
Status
Locked
on prog
ram?
Stop
No
Yes
Call API
Set to Away
Status
Stop
Good evening or good afternoon depending on your location in the world. This is something that we take for granted that we can have a webinar where people from all over the world can listen in to.
We're broadcasting this from two places I, Rob and based in the Netherlands is on is based in Sri Lanka and together will do this presentation with the title ESB meets IoT, a primer on the enterprise service bus. Good evening or good afternoon depending on your location in the world. This is something that we take for granted that we can have a webinar where people from all over the world can listen in to.
We're broadcasting this from two places I, Rob and based in the Netherlands is on is based in Sri Lanka and together will do this presentation with the title ESB meets IoT, a primer on the enterprise service bus.
Okay well it's time to get started then.
About 1 min
Okay a little bit about Yenlo. Company was founded in 2007 and focus is really on application integration and enterprise middleware. We are very proud to be the number one global strategic Alliance partner for WSO2. As a partner we offer a whole range of WSO2 services like product support, QuickStart, training and certifications and so on.
We are a rapidly growing with our headquarters in the Netherlands and officers in the UK Germany United States already and plans to expand the number of countries where we operate.
This is a more graphical overview of what we deliver in everything we do is focused on integration and of course the open source WSO2 platform. You can take a look at our website YENLO.com will find more information, among others blogs about WSO2 and its products. I think that's enough said about Yenlo. Let's start talking about the API manager at one of the components that we see a lot with our clients, from the small ones to the really big fortune 500 organisations. That goes to show that integration is not only an issue
Let's look at the topics of this webinar. For the next 50 minutes this is what we're going to talk about. Going to start with some background on integration of systems but also services and components.
We're going to look at the key functions of the API manager in somewhat detail. Please be remembered that this is what payments for people are interested in the enterprise service bus but not necessarily experts. If you’re interested in that sort of content I would suggest to follow a course on the enterprise service bus of developers which is quite good and the evolved also for developers but that goes in two more detail into the enterprise service bus.
Going to look at other WSO2 components that can and often are used in conjunction with ery technical.
We are going to look into the use of internal and external API’s and the connection to billing systems.
In the next 45 minutes we look in more detail into the functions of the enterprise service bus but not before we have talked about why we need an enterprise service bus in the first place. My colleague ishan will demo the enterprise service bus all to be more specific how to create a mediation sequence. What exactly a mediation sequence is we will see at a later stage presentation. Off the demonstration we will talk about the new area of technology where the enterprise service bus is a great component to use that is of course the Internet of things. Goes to show that the enterprise service bus is not only there to connect legacy systems but it’s also very much a tool or a component if you like to get the benefits from the world of sensors that’s going to be built over the next 5 to 10 years.
Products from WSO2 are always on the development functions there are added and new releases are used quite frequently but not too frequently might I might add. So we going to look at what's on the development roadmap for the next release. With the disclaimer that what we describe here might not necessarily be part of the next release but you know these disclaimers.
Enterprise service bus or ESB for short is one of those acronyms that you can hear buzzing around in many organizations. Some people might say we need an ESB for our service oriented architecture. But I think that all of you are wiser than that and don’t jump all hop on the first bus that comes along with out looking at where it’s going.
ESB in the Context of SOA
What is SOA?
Architectural approach based on discrete pieces of software providing functionality as services to other applications. Services are well defined and reusable.
The whole idea of the service oriented architecture is of course a very strong and very valid one the. The idea that programs or components CAN perform services for other programs and therefore can be integrated in a larger program without having to write the same piece of code several times is quite powerful.
It must be said that finding a service that will benefit a larger part of your organization that only the part you’re working in or responsible for is more difficult. You need to know what their requirements are in order to be able to develop something that they can use and perhaps even more important that they will use.
Pre-requisite Reading:
Web Services – Application component that communicates via open protocols. Deployed on a server that can expose the service
WSDL – Web Service Description Language. A standard meta-language to describe the services offered
UDDI – Universal Description, Discovery and Integration specification. A mechanism to register and locate WS-based applications
SOAP – Simple Object Access Protocol. A Standard way for communication
It must be said that finding a service that will benefit a larger part of your organization that only the part you’re working in or responsible for is more difficult. You need to know what their requirements are in order to be able to develop something that they can use and perhaps even more important that they will use.
WSO2 was designed with this in mind components can offer services or features two other components for instance the business activity monitoring with can be integrated in the API manager to show statistics in the identity server can be added to the same API manager if a more high performance situation is needed.
Bridging Heterogeneous Technologies
Abstract necessary components
Use services and messages together with a service bus
Key Benefits
Open Standards (WSDL, SOAP, JMS, HTTP, FTP…)
Increased flexibility; easier to change as requirements change
Scales from point-solutions to enterprise-wide deployment (distributed bus)
Configuration rather than integration coding
No central rules-engine, no central broker
When to Use an ESB?
Common use cases
Heterogeneous systems
Moving files between systems
Moving messages between queues
Transforming data between systems
Orchestrating a process between several systems
Processing messages or data according to defined business rules
Exposing legacy systems as web services and providing data from web services to legacy systems
What is your use case?
Using standards, Integration Architects can exploit the value of messaging without writing code for any of these use cases
already talked a bit about the fact that the ESB is an architectural model. The fact that the word enterprise is included in the acronym is an indication what kind of organizations would benefit from an ESB, namely the enterprise. I spent most of my life working for banks and insurance companies in the number of systems they have on average is staggering many of these systems are silo so will have one administration for pensions one administration for health insurance in one administration for other products and none of these administrations are connected. The same goes for banks many systems written in all kinds of languages. I’ve heard a story and I’m not sure if it’s true but it would be a 20+ year task for someone to document on the higher-level all the systems on major international bank.
But like it says in the slide will let’s not focus solely on legacy the enterprise service bus is going to be a key component in the emerging Internet of things arena as will see later on.
Over the years that we’ve worked with IT we’ve learned a lot about what works and what doesn’t work. Parallel to that we’ve also seen the capabilities of computers and software grow toward point that we can now talk about a service oriented architecture.
These systems often still have critical functions within organizations and therefore integration of these applications is vital to the enterprise. The enterprise by way is an ever-changing entity. Through mergers or acquisitions of systems are added to the ICT landscape. Developments in technology (e.g. the ESB) make it increasingly possible to connect clients, suppliers and partners to your network.
So how does the enterprise service bus work?
But the ESB is also about IoT .
IoT is very much in the top of Gartner hype cycle of emerging technologies to thousand and 14. For those of you that now live cycle, the basic premise is that any new technology will be hired to such an extent that the things we contribute to that technology are not realistic anymore. The peak by the way is aptly named the peak of inflated expectations and technologies will fall off the the peak into the trough of the solution. Gartner as you can as you can hear quite a bit of drama in the graphs. One of the things that becomes clear that that’s going to be around for many years is the problem of interoperability. I don’t know any vendor that doesn’t say standards are not good. But as someone told me last week standards are like toothbrush everybody wants them but it has to be there own.
At a high-level, ESB is the middleware between a service and a client requesting the service.
* Messages navigate between client and service through sequences built in the ESB
* 3 main sequences exist: In Sequence, Out Sequence and Fault Sequence
Message Flow:
A client sends a request to a service
Message flows thru the inSequence
Message is delivered to Service
Service generate a response
Response is sent thru OutSequence
Message is delivered to Client
! FaultSequence can be used if necessary (i.e. error handling)
By default, the fault sequence will log the message, the payload, and any error/exception encountered, and the drop mediator stops further processing. You should configure the fault sequence with the correct error handling instead of simply dropping messages.
Instructor Notes:
<Use board to explain a simple example: Sequence with 3 mediators: LOG Mediator, XSLT Transformation, SEND Mediator>
Each building block is explained in the next slide
Sequences are the configuration component for mediators. Sequences allow you to organize mediators to implement pipes and filter patterns.
Mediators are individual processing units that perform a specific function, such as sending or filtering messages. WSO2 ESB includes a comprehensive mediator library that provides functionality for implementing widely used enterprise integration patterns (EIPs). You can also easily write a custom mediator to provide additional functionality using various technologies such as Java, scripting, and Spring.
An endpoint defines an external destination (such as a service) for a message. An endpoint can be specified as an address endpoint, WSDL endpoint, a load balancing endpoint , etc.
An endpoint is defined independently of transports, allowing you to use the same endpoint with multiple transports. When you configure a message mediation sequence or a proxy service to handle the incoming message, you specify the transport to use and the endpoint where the message will be sent.
Transports
Carry messages in a specific format
WSO2 ESB supports all the widely used transports including HTTP/s, JMS, and VFS, and domain-specific transports like FIX. All WSO2 transports are directly or indirectly based on the Apache Axis2 transports framework.
Eclipse and ESB
Also in training used
Programming visually
There are some basic steps that need to be taken. The first step of course is that an application sends a message that is picked up by the enterprise service bus. The message is picked up by something that is called a transport. The transport is nothing more than a message in a specific format. You can have an HTTP transport which of course uses HTTP protocol whereas a HL7 transport uses a specific Health Level 7 message format. WSO2 supports the most common transports. In addition to the aforementioned two transports also sending email messages (MailTo transport) is possible.
After the message is picked up by a transport it is sent through a message pipe that handles things like security and other quality of service aspects (QoS). With quality of service we mean the way the message should be handled in the ESB, should it be encrypted or should it get priority when sending a message through.
The enterprise service bus can work in two modes:• message mediation services;• proxy servicesIn the message mediation mode the ESB is able to take a message and to perform one or more transformations on the message. Depending on what’s needed a message can be cloned or aggregated, data formats can be changed or you can even write your own message mediation. This allows you to do virtually anything that you can imagine with a message.
You should see the message transformation and message routing as one single unit. In some cases the transformation will take place before the routing decision, in other cases after the routing decision.
When the ESB is used in proxy services mode it will create a virtual endpoint that acts as a façade for an internal service. Transformations and mediation can still take place within such proxied services.
The message is put into a message pipe where again the quality of services aspects are taken care of, for instance encryption or decryption or other aspects. The transport layer takes care of the transport and delivers it to the actual endpoint (the application that should receive it in the first place).
After the message is picked up by a transport it is sent through a message pipe that handles things like security and other quality of service aspects (QoS). With quality of service we mean the way the message should be handled in the ESB, should it be encrypted or should it get priority when sending a message through.
After describing the functionality of the WSO2 enterprise service bus in general terms, let’s look into very specific functionality that WSO2 ESB offers.
WSO2 claims that it can connect anything to anything by supporting a number of transports (the most commonly used e.g. HTTP, POP, TCP, FIX and SMS), multiple formats and protocols like JSON, XML, HTML and HL7. They also include adapters to commercial off-the-shelf systems like SAP BAPI (SAP’s basic application programming interface) & iDoc (SAP’s standard data structure for Electronic Data Interchange) and IBM WebSphere MQ. Lastly there are adapters to cloud services like
When you look at routing, mediation and transformation the possibilities are almost limitless. We already indicated that its possible to add custom mediation rules to the ESB. When routing messages you can choose to do that based on the message header, its content and several other rules that you combine yourself and in the right order.
All of this is of course done with complete control over security including authentication, authorization and entitlement. For this the enterprise service bus works together with a number of other WSO2 components like the Governance Registry, API Manager, Identity Server, Business Activity Monitor and so on.
Connectors and connector store
In order to make the ESB simple to use it uses declarative development rather than coding to configure it. You can deploy the WSO2 ESB on your own servers, in a public, private or hybrid cloud and you even have the choice to the WSO2 enterprise service bus as a service offering running on top of Amazon and being fully configured.
In order to make the ESB simple to use it uses declarative development rather than coding to configure it. You can deploy the WSO2 ESB on your own servers, in a public, private or hybrid cloud and you even have the choice to the WSO2 enterprise service bus as a service offering running on top of Amazon and being fully configured.
We already seen this graph the hype cycle with all the technologies on there. This is the full graph and four technologies to read to reach the plateau productivity, the ultimate and stage show might say can and will take many years. For example in this graph you can see speech recognition finally on the plateau productivity [in speech technology has been around for many years].
So I said that we would have a use case in the area of the Internet of things. And use case that was chosen is it a simple one or at least how we describe it in this webinar. It has two components on one hand there is the smart thermostat. In the Netherlands the leading thermostat is called toon it’s developed by Dutch company and it sold exclusively by eneco energy company in the Netherlands it’s a great success they sold more than 100,000 in I think the last two years. It is the thermostat with a touchscreen in color and as you can see on the slide it also has an API. In this case it’s actually the API manager that they’re using to open the API to developers would like to develop new apps using the thermostat.
Even if there were not using WSO2 products we could still use it it’s just a nice nugget of information and for is quite good that are actually using our products.
On the other hand we have the French company okey-dokey’s rule have developed a smart lock that can be opened in different ways with the key of course but also with on your smart phone with her bracelets and NFC card etc. the okey-dokey’s also have an API as far as I know they’re not using our products but you never know the future
and what would like to do is connect those two. As I said it’s not going to the silver bullet of iot integration, and the example that we give should be optimized. Regretfully we received information quite late so we weren’t able to develop it into a working demo so we will describe the way it should work and also say where we could improve it.
Today’s use case if I’m leaving my home and I locked the door we can assume that no one is home since I really lock the door with either the key or the smart phone when I do that it would be good to see if the thermostat program is on away. Perhaps I forgot to put it on away and system can do it for me. This is one of the points that you need to think about what you’re doing, for instance if I’m just going away for five minutes turning off the thermostat and five minutes later switching it on again when I opened the door makes little sense. It might also be that there are other people in the home so if you want to develop this into full system you need to do some more thinking on how it should work and how it would benefit all people
Activity
Login
https://devportal.okidokeys.com:80/index.php?page=account/login&json=true
Recognising closing of door (key, smartphone or tag)
Currently this should be done by calling the doorlock status API on the gateway every 5 minutes
IF the return answer is that the door is closed we go to to the TOON thermostat
oAuth
Logging in with Toon
Requesting current program
If fixed on program, no change
If not on ‘away’ put Toon on ‘away’
Here we put some information about the Connected Pigs or plants on IOT and ESB.
Here we put some information about the Connected Pigs or plants on IOT and ESB.