As more systems and applications are adopted into an organization’s ecosystem, point-to-point connections can become complex and unmanageable. This webinar outlines the concept of an enterprise service bus (ESB). Through demos, attendees will learn how FME Server can be implemented as an event-driven hub for managing messages between systems to integrate them in a decoupled way which facilitates future extensions and integration of additional systems.
Integrating the Enterprise with Event-Driven Messaging – Using FME Server as an ESB
1. HOW TO USE FME SERVER AS AN
Enterprise Service Bus
2. WHO WE ARE
Dean Hintz
Senior Analyst,
Strategic Experts
Safe Software
Neil Hellas
Enterprise Solutions Architect
Spatial DNA
Tens of thousands of data pros use FME,
the data integration platform with the
best support for location data.
Don Murray
President,
Safe Software
3. AGENDA
● FME
● Enterprise integration
challenges & ESB
● FME Server as ESB
● Spatial DNA examples
● 311 ESB Demo
● Q&A
4. Technology is always changing
Innovative solutions are possible
New systems are constantly being added
System integration is hard
CHALLENGE
7. 400+ SUPPORTED SYSTEMS AND DATA TYPES
Database servers: Oracle, SQL Server, PostGreSQL, MySQL,...
Web systems: MongoDB, CouchDB, Dropbox, Box.com, Amazon S3,
OData, Google Sheets, Socrata, Salesforce, Sharepoint, Azure, OGC ...
Business Intelligence: Tableau, Excel, ElasticSearch, Qlik
Extensibility: integration with traditional and emerging systems
● JDBC - often traditional
● HTTP - often cloud
Tabular, XML, JSON, CSV, HTML ...
8. FME TERMINOLOGY
Transformers: Used to connect to systems, extract data, transform and
remodel data and then load data. Over 400 available!
Transformers use and produce Features.
Features: Records or rows in spreadsheets and databases
or objects in other systems.
Features have Attributes.
Attributes: Fields or columns attached to records, rows and objects.
15. Enterprise Integration Challenge
● Multiple, separate applications for different business
systems: HR, CRM, BI, ERP, Accounting
● Data silos, duplicate records
● Integration is manual, ad-hoc, point to point
● The more systems that are added, the more difficult it
is to manage
16. Enterprise Integration Challenge: 311 Reporting
Background: Citizens submit reports on the website about problems they notice in the
community: missed garbage, potholes, etc. Those reports need to go to
the service delivery team who manages related services.
Challenge: The more systems that need to be connected to the 311 database, the
more complex the interconnections between databases become.
Specifics: Use topics within the notification service on FME Server as an ESB. All
requests between client systems and the 311 database pass through
the ESB topics on Server using a common messaging structure. Each
system only needs to manage connections to this bus, not other systems.
ESB
D
em
o
23. An ESB can… has…
• Route data between systems • Testers, Filters, Notifiers
Capabilities of an ESB
24. An ESB can… has…
• Route data between systems
• Transform formats and protocols
• Testers, Notifiers
• XML and JSON Templater,
Extractors and HTTPCaller
Capabilities of an ESB
25. An ESB can… has…
• Route data between systems
• Transform formats and protocols
• Request/Response or
Aynchronous
• Testers, Notifiers
• XML and JSON Templater,
Extractors and HTTPCaller
• Service Types, JobSubmitter
Capabilities of an ESB
26. An ESB can… has…
• Route data between systems
• Transform formats and protocols
• Request/Response or
Aynchronous
• Publish/Subscribe
• Testers, Notifiers
• XML and JSON Templater,
Extractors and HTTPCaller
• Service Types, JobSubmitter
• Notifications
Capabilities of an ESB
27. An ESB can… has…
• Route data between systems
• Transform formats and protocols
• Request/Response or
Aynchronous
• Publish/Subscribe
• Events and messages
• Testers, Notifiers
• XML and JSON Templater,
Extractors and HTTPCaller
• Service Types, JobSubmitter
• Notifications
• Publisher/Subscriber Protocols
Capabilities of an ESB
28. • Route data between systems
• Transform formats and protocols
• Request/Response or
Aynchronous
• Publish/Subscribe
• Events and messages
• Extensible & pluggable
• Testers, Notifiers
• XML and JSON Templater,
Extractors and HTTPCaller
• Service Types, JobSubmitter
• Notifications
• Publisher/Subscriber Protocols
• Chaining topics
An ESB can… has…
Capabilities of an ESB
29. • Route data between systems
• Transform formats and protocols
• Request/Response or
Aynchronous
• Publish/Subscribe
• Events and messages
• Extensible & pluggable
• Publish Metadata
• Testers, Notifiers
• XML and JSON Templater,
Extractors and HTTPCaller
• Service Types, JobSubmitter
• Notifications
• Publisher/Subscriber Protocols
• Chaining topics
• FME Server API
An ESB can… has…
Capabilities of an ESB
56. FME Server as ESB: Key Take-aways
● ESB - connection of many systems without increasing architecture complexity
● FME Server’s notification service provides ESB functions and components
● 311 ESB configuration tested using scheduled publication workspaces and
subscription workspaces connected to a set of topics
● Initial ESB design needs to be carefully laid out so the events and message
interactions are clearly mapped
● Define message structure and common data model: key elements, routing
● A good design is easily extendible: start simple, then expand
59. Field Names and Values: AttributeManager
Copy from one field to another.
Provide a hard-coded default for
a field.
Concatenate multiple field values
together into a single field.
Copy part of a value from one
field to another.
62. Summary
● Dilemma of enterprise growth and complexity
● ESB approach / pattern helps manage growth
● FME Server’s notification service as an ESB
● Easy to add new systems and scale capacity without
increasing complexity
● Notification service - a great way to integrate FME into
your enterprise workflows whether or not you use ESB
63. Related Webinars
● How to Use FME as an Enterprise Integration Platform (Aug 2017)
https://www.safe.com/solutions/enterprise-data-integration
● Web Services & Web Data – Best Practices for Integration (March 2017)
https://www.safe.com/webinars/web-services-fme-2017/
Upcoming:
● Deep Dive into FME Server 2018 (Feb 22, 2018)
● FME Server Automations (March 28, 2018)
64. Resources
● Neil’s FME UC 2017 Talk: “Making FME Your ESB”
● FME Server Playground https://playground.fmeserver.com/
● FME Server Notification Docs
● Tutorial: Notifications
● Tutorial: Automation Planning and Deployment
● Tutorial: Reading and Writing JSON
● FME Hub: Custom transformers, templates, coming: Server Projects
ESB Demo project package will be sent to all attendees
65. Thank you
Dean Hintz
Senior Analyst,
Strategic Experts
Safe Software
dean@safe.com
Neil Hellas
Enterprise Solutions Architect
Spatial DNA
neil.hellas@spatialdna.com
Don Murray
President,
Safe Software