This webinar covered new features in Ironstream for enhanced z/OS analytics, including advanced filtering for SMF data and data loss protection. It discussed integrating Ironstream with Splunk's IT Service Intelligence platform to provide a unified view of critical mainframe services and metrics. The webinar demonstrated several sample Splunk apps available on Splunkbase that leverage mainframe data from Ironstream to monitor things like CICS regions, MQ queues, system performance, and syslog messages.

1. Syncsort Mainframe Customer Education Webinar
New Ironstream® Facilities For Enhanced z/OS
Analytics
2Q 2017

2. Today’s Presenters
Ed Wrazen Director, Mainframe Product Management is responsible for
the product strategy & roadmap for Syncsort’s Mainframe products and
solutions. With a career in Enterprise IT spanning 35 years, Ed has held
roles in software development, database administration, product
management, consulting and marketing in global businesses and
enterprise technology companies. Ed has experience in Enterprise
systems architectures, performance management, database and data
management technologies and is a regular speaker at industry events
worldwide.
2Syncsort Confidential and Proprietary - do not copy or distribute
Ed Hallock is a highly experienced Information Technology Professional
with a broad experience base in software product development, support,
product management, marketing, and business development. In his
diverse career Ed has benefited from working for some of the largest
independent software vendors, in a variety of roles, providing enterprise
solutions to Global 1000 corporations. Ed has extensive experience in
performance and availability management for systems and applications.
He holds a bachelor’s degree in Computer Science from Montclair State
University in Upper Montclair, New Jersey and has presented at numerous
industry events as well as corporate related conferences and seminars.

3. Agenda
Introduction to Ironstream®
New Features:
– Advanced Filtering for SMF data
– Data Loss Protection
Integration with Splunk’s IT Service Intelligence
Ironstream Sample Splunk Applications

4. Splunk: The Industry-Leading Platform For Machine Data
Syncsort Confidential and Proprietary - do not copy or distribute
Machine Data: Any Location, Type, Volume
Online
Services
Web
Services
Servers
Security GPS
Location
Storage
Desktops
Networks
Packaged
Applications
Custom
Apps
Messaging
Telecoms
Online
Shopping
Cart
Web
Clickstreams
Databases
Energy
Meters
Call Detail
Records
Smartphones
and Devices
RFID
On-
Premises
Private
Cloud
Public
Cloud
Platform Support (Apps / API / SDKs)
Enterprise Scalability
Universal Indexing
Answer Any Question
Developer
Platform
Report &
analyze
Custom
dashboards
Monitor
& alert
Ad hoc
search
Mainframe
4

5. Critical Mainframe Data 
Normalized and Streamed to Splunk with Ironstream®
Log4jFile
Load
SYSLOG
SYSLOGD
logs
security
SMF
50+
types
RMF
Up to 50,000
values
DB2SYSOUT
Live/Stored
SPOOL Data
Alerts
Network
Components
Ironstream
API
Application Data
Assembler
C
COBOL
REXX
USS

6. Value of an End-to-End View, Inclusive of Mainframe
Extend What Splunk Does Already, to include critical z/OS systems:
– 360ᵒ Degree View: Make the Splunk View of the Enterprise Complete via
Including Mainframe Data
– Same Splunk Dashboards, Bigger, More Complete Data Sets; Free Ironstream
Splunk Apps and Modules
Security and Compliance/SIEM
- Ensure Audits Passed
IT Operational Analytics/ITOA
-Ensure Ops SLAs Met
IT Service Intelligence/ITSI
-Ensure Services Health

7. Polling Question #1
What analytics platforms are you using today for z/OS IT operational
intelligence:
 Splunk
 Hadoop
 ELK (Elastic Stack)
 Spark
 Custom/Home Grown solution
 None
7

8. 8Syncsort Confidential and Proprietary - do not copy or distribute
Advanced Filtering for SMF Data

9. Why Filter SMF Data?
SMF volumes can be enormous – large CICS and DB2 installations can
generate TBs of data daily
Transferring data that is not useful puts a strain on network and other
system resources
Need to provide control over volume of SMF data processed and
forwarded by Ironstream to Splunk
Need to eliminate data clutter by forwarding only those fields that are
truly needed
9Syncsort Confidential and Proprietary - do not copy or distribute

10. SMF Filtering and WHERE Processing
Ability to select only desired fields within individual SMF records
– INCLUDE statement in configuration file or via field selection in the Ironstream
Desktop GUI
New extension enables selection of fields based upon the value of
field
– WHERE clause in configuration file
10Syncsort Confidential and Proprietary - do not copy or distribute

11. Basic WHERE Syntax
"SELECT":"SMFnnn"
"INCLUDE":"field_1,field_2,...,field_n" Optional Statement
– If omitted, INCLUDE defaults to ALL
"WHERE":"search_condition AND/OR search_condition“
– Any number of search conditions can be specified
– If multiple search conditions are given, each must be separated by a
logical AND or OR operator
– Search_condition: Field_1 operator operand
• Field_1 must be the name of a field from the SMF record
• The operator can be: EQ, NE, LT, LE, GE, GT
• Operands can be another field name, character strings, decimal values, hex
values, date, time
• Wildcards supported for character strings
11Syncsort Confidential and Proprietary - do not copy or distribute

12. Basic WHERE Syntax
"SELECT":"SMFnnn"
"INCLUDE":"field_1,field_2,...,field_n" Optional Statement
– If omitted, INCLUDE defaults to ALL
"WHERE":"search_condition AND/OR search_condition“
– Any number of search conditions can be specified
– If multiple search conditions are given, each must be separated by a
logical AND or OR operator
– Search_condition: Field_1 operator operand
• Field_1 must be the name of a field from the SMF record
• The operator can be: EQ, NE, LT, LE, GE, GT
• Operands can be another field name, character strings, decimal values, hex
values, date, time
• Wildcards supported for character strings
12Syncsort Confidential and Proprietary - do not copy or distribute

13. Examples
"DATATYPE":"SMF"
"SELECT":"SMF030"
"WHERE":"SMF30TME GT T'11:17:00.00' AND SMF30TME LT T'11:30:00.00'“
"DATATYPE":"SMF"
"SELECT":"SMF030"
"WHERE":"(SMF30JBN EQ ‘WWC*' AND (SMF30_TIME_ON_ZIIP GT T’00:00:01.00’ +
OR SMF30_TIME_ZIIP_ON_CP GT 0)) OR SMF30JBN EQ ‘CYB*'"
13Syncsort Confidential and Proprietary - do not copy or distribute

14. 14Syncsort Confidential and Proprietary - do not copy or distribute
Data Loss Protection (DLP)

15. Why is DLP Needed?
To prevent loss of data forwarded by Ironstream to Splunk
– Early implementations of Splunk did not include any mechanism for ensuring
that data forwarded by Ironstream was both received and successfully indexed
by the Splunk platform
• If Splunk encountered an error prior to indexing the data it received from
Ironstream, that data was lost even though Ironstream had successfully forwarded
it
– Network failures preventing Ironstream from forwarding data for a long
enough period would cause the in-storage data buffers to overflow resulting in
data loss
15Syncsort Confidential and Proprietary - do not copy or distribute

16. New Feature: Data Loss Protection (DLP)
Minimizes data loss during times of network or other external failures.
Uses IBM’s Coupling Facility’s System Logger functions, and Splunk’s
Indexer Acknowledgement feature.
– Splunk indexer acknowledgement feature allows Ironstream to detect when
data it has forwarded has been successfully received and indexed by the
Splunk platform.
16Syncsort Confidential and Proprietary - do not copy or distribute

17. New Feature: Data Loss Protection (DLP)
Minimizes data loss during times of network or other external failures.
Uses IBM’s Coupling Facility’s System Logger functions, and Splunk’s
Indexer Acknowledgement feature.
– Splunk indexer acknowledgement feature allows Ironstream to detect when
data it has forwarded has been successfully received and indexed by the
Splunk platform.
Optional feature that must be enabled….
– Must define and configure a log stream within a coupling facility and make
Ironstream configuration parameter changes.
– No modifications required to existing Ironstream configuration files for those
customers not requiring DLP.
– More information is available in the Ironstream Configuration and Users Guide.
17Syncsort Confidential and Proprietary - do not copy or distribute

18. Ironstream SMF Processing without DLP
SMF Exits
Store data in DataStore
Extract from Data Store
and normalize data
Buffers and send data
No mechanism to ensure Splunk
has indexed data received from
Ironstream
Potential DataStore overflow on
network or Splunk failure
18Syncsort Confidential and Proprietary - do not copy or distribute

19. Ironstream SMF Processing with DLP
SMF Exits
Store data in DataStore
Move function takes records
from DS and stores in CF
Extracts from CF and
normalize data
Buffers and sends data
ACK is received from Splunk
before deleting CF records
CF data only deleted from there
once a positive acknowledgement
has been received from the
Splunk indexer.
CF continues to collect and retain
data during extended network or
Splunk outages
Ironstream re-sends it once
Splunk is active or network
problems are resolved.
19Syncsort Confidential and Proprietary - do not copy or distribute

20. Polling Question #2
What analytics platforms are you considering or evaluating to use for
z/OS IT operational intelligence:
 Splunk
 Hadoop
 ELK (Elastic Stack)
 Spark
 Custom/Home Grown solution
 Other
20

21. 21Syncsort Confidential and Proprietary - do not copy or distribute
IT Service Intelligence (ITSI)
Integation

22. What is Splunk IT Service Intelligence (ITSI)?
Splunk IT Service Intelligence delivers
machine learning-powered analytics to
simplify operations, prioritize issue resolution
and provide visibility into critical services.
22Syncsort Confidential and Proprietary - do not copy or distribute
 Delivers a central, unified view of
critical IT services for powerful, data-
driven monitoring
 Maps critical services with KPIs to
easily pinpoint what matters most
 Uses machine learning to detect
patterns, dynamically adapt
thresholds, highlight anomalies and
pinpoint areas of impact
 Provides business and service
context to prioritize incident
investigation and triage
 Supports drill downs to profile an
entity and rapidly troubleshoot
outages and service degradations

23. Ironstream ITSI Integration
Providing z/OS Metrics & Analysis to IT Service Intelligence
23Syncsort Confidential and Proprietary - do not copy or distribute
3 Levels
Overall Mainframe Central Processor Complex
LPAR Logical Partition (virtual machine equivalent)
Software Components
 CICS online transaction processing
 DB2 database, typically used with CICS

24. ITSI z/OS Data Sources provided by Ironstream
24Syncsort Confidential and Proprietary - do not copy or distribute
2 Data Sources
- Resource Measurement Facility (RMF)
- System Management Facilities (SMF)
Mainframe CPC
CPU Load % RMF M8D2550
Delay % RMF M8D0160
I/O Rate % RMF M8D0E90
Service Rate % RMF M8D1FB0
Workflow % RMF M8D0550
LPAR
CPU Load % RMF M8D0460
Delay % RMF M8D0160
Free Memory % RMF M8D0380
Free Storage % RMF M8D2A50
Using % RMF M8D04A0
Workflow % RMF M8D0550
CICS System & Individual Transactions
ABENDs SMF 110
Response Times SMF 110
DB2
Deadlocks SMF 101, IFCID 3
Exclusive Escalations SMF 101, IFCID 3
Shared Escalations SMF 101, IFCID 3
Lock Waits SMF 101, IFCID 3
Timeouts SMF 101, IFCID 3
Ironstream Desktop (IDT)

25. Ironstream ITSI Integration
25Syncsort Confidential and Proprietary - do not copy or distribute
Mainframe
TCP/IP
SSL or non-SSL
Data Forwarder DCE IDT
Ironstream DesktopData Collection Extension
Data ForwarderData Forwarder
DB2SYSOUT
Live/Stored
SPOOL Data
Alerts
Network
Components
Ironstream API
Application Data
Assembler
C
COBOL
REXX
USSLog4jFile
Load
z/OS
SYSLOG
SYSLOGD
logs
security
SMF
50+
types
RMF
Up to 50,000
values
Enterprise
Security
IT Service
Intelligence SPLUNK

26. Ironstream ITSI Integration
Service Analyzer
26
KPIs provided for
mainframe systems in
Service Analyzer
– CEC (Central
Electronic Complex),
i.e. “the box”
– LPARs (logical
partitions)
– Critical services
Glass Tables for
visualization

27. Ironstream Integration with ITSI
Glass Table with 3 Level Overview
27
Critical services
CEC (Central
Electronic Complex)
LPARs (logical
partitions)

28. Ironstream Integration with ITSI
Glass Table for Online Banking Services
28

29. Ironstream Integration with ITSI
DB2 Deep Dive
29

30. Ironstream Integration with ITSI
CICS Details
30

31. Polling Question #3
What Security Information and Event Management (SIEM) platform is in
use within your Enterprise:
 IBM zSecure/QRadar
 Correlog
 Splunk Enterprise Security
 HP Arcsight
 Logrythm
 Other
31Syncsort Confidential and Proprietary - do not copy or distribute

32. 32Syncsort Confidential and Proprietary - do not copy or distribute
Ironstream Splunkbase Apps

33. What are apps and how are they used?
Sample dashboards that utilize specific data sources to demonstrate
the value of Ironstream
Downloaded from splunkbase and installed into the Splunk Enterprise
environment for use with Ironstream supplied data
– 3-step simple process for a Splunk admin
1. Upload/install the app
2. Ensure an Ironstream index is defined to Splunk
3. Define the TCP/IP connection to Splunk for the Ironstream datasource
Note: Splunk admin has the ability to modify queries and reports
contained within each dashboard to meet their individual
requirements
33Syncsort Confidential and Proprietary - do not copy or distribute

34. Search on Ironstream or Syncsort in splunk base to see our apps
34Syncsort Confidential and Proprietary - do not copy or distribute

35. Ironstream Applications on splunkbase
Syslog
– RACF violations and message trends
CICS Region Monitor
– CICS Region Health Check
– CICS Region transaction rates, response times, CPU usage, & failures
MQ Monitor
– Queue depths and response time
– Message Get/Put rates and CPU use
– Ability to filter by connection name and queue name
35Syncsort Confidential and Proprietary - do not copy or distribute

36. Ironstream Applications on splunkbase
System Performance Monitor
– CEC MSU capacity alongside the 4-hour rolling average figures (4HRA) for each
LPAR
– z/OS system performance data including:
• CPU utilization, memory and common storage utilization, Paging rates
Dataset Analyzer
– Critical datasets to be monitored are defined via a .CSV file in Splunk
36Syncsort Confidential and Proprietary - do not copy or distribute

37. Select an app and download Product Documentation from the Details Tab
37Syncsort Confidential and Proprietary - do not copy or distribute

38. Syslog App: Benefits
View RACF violations by type and user
– Understand invalid logon attempts
– Unauthorized attempts to access datasets
Look at message trends over time to determine potential security
threats
View messages by subsystem
38Syncsort Confidential and Proprietary - do not copy or distribute

39. Syslog App: RACF Violations and Message Trends
Syncsort Confidential and Proprietary - do not copy or distribute 39
RACF Violations by type RACF Violations by user
Trend message volumes today vs. same time last week and 2 weeks ago

40. CICS App: Benefits
Monitor CICS regions and transactions supporting critical business
services
Understand transaction rates, response times, and resource utilization
to determine if business services are being met or impacted
Identify transaction failures that are impacting business services
40Syncsort Confidential and Proprietary - do not copy or distribute

41. CICS Region Health Check
Syncsort Confidential and Proprietary - do not copy or distribute 41
Transaction Response Time Transaction Rates
Dispatch Time

42. MQ App: Benefits
Monitor MQ connections and queues supporting critical business
services
Understand message rates, response times, and resource utilization to
determine if business services are being met or impacted
42Syncsort Confidential and Proprietary - do not copy or distribute

43. MQ Monitor
43Syncsort Confidential and Proprietary - do not copy or distribute
Queue Response Time by Connection
GET and PUT CPU Time by Connection

44. System Performance App: Benefits
Monitor all critical resources for a z/OS LPAR to ensure business
services are not impacted
Determine if specialty processors(zIPP) are being used to reduce
general processor utilization
Monitor the 4-hour rolling average for MSUs by LPAR
44Syncsort Confidential and Proprietary - do not copy or distribute

45. System Performance Monitor
45Syncsort Confidential and Proprietary - do not copy or distribute
4HRA by LPAR CEC MSUs
CPU usage by processor type

46. Dataset Analyzer App: Benefits
Define and monitor access to critical datasets
Determine potential security threats based on unauthorized access
attempts
Ensure only authorized users are accessing critical datasets
Understand when dataset access conflicts could be impacting overall
application performance
46Syncsort Confidential and Proprietary - do not copy or distribute

47. Dataset Analyzer App
47Syncsort Confidential and Proprietary - do not copy or distribute
Access by type for each critical dataset

48. Summary: Value Today for Enterprises with a z/OS Mainframe
Syncsort Confidential and Proprietary - do not copy or distribute
Less Complexity
Collect mainframe data; correlate
with data from other platforms; no
mainframe expertise required
Clearer Security Information
Identify unauthorized mainframe
access, other security risks; prepares
and visualizes key data for
compliance audits
Healthier IT Operations
Real-time alerts identify problems in
all key environments View latency,
transactions per second, exceptions,
etc.
Effective Problem-Resolution
Management
Real-time views to identify real or
potential failures earlier; view related
'surrounding' information to support
triage repair or prevention
Higher Operational Efficiency
Enhanced event correlation across
systems; Staff resolves problems faster;
“do more with less”
Eliminate Your Mainframe
“Blind-Spot”
Splunk + Ironstream = Your 360ᵒ
Enterprise View

49. Industry Leader in Mainframe
Software Products

50. What Now?
50
Get Ironstream® for SYSLOG for free
VISIT:
HTTP://WWW.SYNCSORT.COM/EN/PRODUCTS/MAINFRAME/IRONSTREAM
CONTACT:
INFO@SYNCSORT.COM
http://www.syncsort.com/en/TestDrive/Ironstream-Starter-Edition

51. Thank You.
Questions?