More Related Content Similar to TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA (20) More from Hortonworks (20) TIME SERIES: APPLYING ADVANCED ANALYTICS TO INDUSTRIAL PROCESS DATA4. 4 © Hortonworks Inc. 2011–2018. All rights reserved.
Highest Value Data
Always on, always connected devices generate a
constant stream of data related to the operations of
industrial businesses
These datasets contain:
• What events occurred
• Why and event occurred, or not
• Quantification of an event’s impact
These datasets go by many names:
• “SCADA Data”
• “Control System Data”
• “Historian Data”
• “Machine Data”
• “Measurement Logs”
• “Telemetry”
How are my …
People?
Processes?
Equipment?
Lots of misnomers
5. 5 © Hortonworks Inc. 2011–2018. All rights reserved.
Instrumentation
§ Commonly only output is
electrical signals
§ Integration with sensors
requires specialized
hardware
§ serial bus, or wireless are
increasingly available
Challenges in accessing data in the ICS landscape
Control Systems
§ Data is transmitted via
proprietary vendor specific
protocols
§ Direct Integration with
control systems requires
protocol translation/parsing
for each platform family
Nifi’s is a toolbox of connectors
§ Ingest text files and interrogate REST APIs
using built in connectors
§ Connect to industry standard protocols like
OPC UA with custom processors
§ Build your own
Existing ICS Components
PLC, RTU & DCS
Open Source Tools
Governance
&Integration
Security
Operations
Data Access
Data Management
Process Historians &
OPC Servers
§ Data is typically available via
programmatic access such
as OPC, API or SQL
§ There is almost always an
option to create text files
11. 11 © Hortonworks Inc. 2011–2018. All rights reserved.
Ethernet/IP Nifi S2S Hbase API
PLC program cyclically updates the values
of 20k PLC registers
Kepware automatically downloads the AB
PLC’s tag database and configures polling
of all available tags in the AB PLC.
In a similar fashion Nifi browses Kepware’s
database and polls each tag found in the
IoT gateway database. The frequency of
this polling is set in nifi
Nifi merges then compresses a
configurable number of responses from
the kepware server before transmitting
them over the Site to site protocol
Nifi recieves, decompresses then splits the
merged text documents into small JSON
documents containing individual data point
samples
Each sample is inserted into Hbase serially
using the Nifi Put processors
kb/sMB/s MB/s
GB/d
Allen Bradley Example
Azure Based NifiAsset Based NifiEquipment Control System
D A T A I N M O T I O N D A T A A T R E S T
Data Sources Data Flow Data Platform