This document discusses how Splunk can provide analytics across the DevOps lifecycle to help organizations realize quality, velocity, and efficiency gains from continuous integration and continuous delivery (CI/CD). It provides examples of metrics and events that can be collected at each phase of the lifecycle to help stakeholders like development, operations, security, and business teams. The document demonstrates Splunk's ability to integrate different machine data sources for comprehensive visibility. It also briefly outlines some Splunk apps that can support DevOps processes and tools.

1. © 2019 SPLUNK INC.© 2019 SPLUNK INC.
The DevOps Promise: Helping
Management Realize the Quality, Velocity
& Efficiency Gains of CI/CD
Domnick Eger | Global DevOps Practitioner
Endre Peterfi | Sales Engineer

2. © 2019 SPLUNK INC.
During the course of this presentation, we may make forward-looking statements regarding future events or
the expected performance of the company. We caution you that such statements reflect our current
expectations and estimates based on factors currently known to us and that actual events or results could
differ materially. For important factors that may cause actual results to differ from those contained in our
forward-looking statements, please review our filings with the SEC.
The forward-looking statements made in this presentation are being made as of the time and date of its live
presentation. If reviewed after its live presentation, this presentation may not contain current or accurate
information. We do not assume any obligation to update any forward-looking statements we may make. In
addition, any information about our roadmap outlines our general product direction and is subject to change
at any time without notice. It is for informational purposes only and shall not be incorporated into any contract
or other commitment. Splunk undertakes no obligation either to develop the features or functionality
described or to include any such feature or functionality in a future release.
Splunk, Splunk>, Listen to Your Data, The Engine for Machine Data, Splunk Cloud, Splunk Light and SPL are trademarks and registered trademarks of Splunk Inc. in
the United States and other countries. All other brand names, product names, or trademarks belong to their respective owners. © 2019 Splunk Inc. All rights reserved.
Forward-Looking Statements

3. © 2019 SPLUNK INC.
Splunk for Dev, Sec, Ops, Biz:
A 30,000ft Overview

4. © 2019 SPLUNK INC.
Visibility Across the Dev Lifecycle
API
SDKs UI
Other Tools
Escalation/
Collaboration
Plan Code Build Test/QA Stage Release MonitorConfig
Examples…
On-Premise, Cloud, Hybrid | Analytics for Hadoop
No rigid schemas – add in data from any other source.

5. © 2019 SPLUNK INC.
Visibility Across the Ops Environment
API
SDKs UI
Server,
Storage. N/W
Server
Virtualization
Operating
Systems
Mobile
Applications
Cloud Services
Other Tools
Ticketing/Help
Desk
No rigid schemas – add in data from any other source.
Custom
Applications
API Services
Infrastructure
Applications
Examples…
On-Premise, Cloud, Hybrid | Analytics for Hadoop

6. © 2019 SPLUNK INC.
Visibility Across the Security Environment
API
SDKs UI
Firewalling IDS/IPS
Vulnerability
Management
DLP
Threat
Intelligence
NBAD
Other Tools
Ticketing/Help
Desk
Proxy / Users
Malware /
Endpoint
proofpoint
Qualys
PAN
ThreatConnect
VectraNetworks
Anomali FireEye
CBlack
Phantom Recorded Future
Examples…
Bro
TippingPoint
FirePower
Rapid7
On-Premise, Cloud, Hybrid | Analytics for Hadoop
No rigid schemas – add in data from any other source.

7. © 2019 SPLUNK INC.
Visibility Across the Business Environment
API
SDKs UI
Clickstream
Data
Endpoint
Devices
Network
Streams
BI/Data
Warehouse
GPS/Cellular
Data
Servers
Other Tools and
Data Sources
Analyst Systems
Applications eCommerce
Data
Examples…
On-Premise, Cloud, Hybrid | Analytics for Hadoop
No rigid schemas – add in data from any other source.
CRM, ERP, HR,
Finance, Products

8. © 2019 SPLUNK INC.
Splunk – Complete Visibility for Dev, Ops, Sec, Biz
Platform for turning machine data into answers
Clickstream
Data
Endpoint
Devices
Network
Streams
BI/Data
Warehouse
GPS/Cellular
Data
Servers
Applications eCommerce
Data
Splunk For Dev Lifecycle Analytics
Plan Code Build Test/QA Stage MonitorReleaseConfig
Server, Storage,
Network
Server
Virtualization
Operating
Systems
Infrastructure
Applications
Mobile
Applications
Cloud Services
Custom
Applications
API Services
Firewalling IDS/IPS
Vulnerability
Management
DLP
Threat
Intelligence
NBAD Proxy / Users
Malware /
Endpoint
Splunk For IT Operations Analytics
Splunk For Security Operations Centers
Splunk For Business Analytics

9. © 2019 SPLUNK INC.
Analytics At Every Phase of The DevOps Lifecycle
Plan Code Build Config Stage Release MonitorTest/QA
• time to deliver
• idea to cash
• ROI
• process times
• team efficiency
• unplanned work
• code volume
• commit volume
• release speed
• test volume
• code coverage
• exception counts
• build speed
• failure rates
• manual builds
• performance
• latency
• scalability
• response time
• uptime/availability
• resource usage
• revenue
• signups
• cust. sat.
• remediation time
• code quality
• access rates

10. © 2019 SPLUNK INC.
Biz PMO Dev Build QA Sec Stage Ops Biz
Specific Data For Each Stakeholder
• time to deliver
• idea to cash
• ROI
• process times
• team efficiency
• unplanned work
• code volume
• commit volume
• release speed
• test volume
• code coverage
• exception counts
• build speed
• failure rates
• manual builds
• performance
• latency
• scalability
• response time
• uptime/availability
• resource usage
• revenue
• signups
• cust. sat.
• remediation time
• code quality
• access rates

11. © 2019 SPLUNK INC.
Biz PMO Dev Build QA Sec Stage Ops Biz
Shared Data for Multiple Stakeholders
• time to deliver
• idea to cash
• ROI
• process times
• team efficiency
• unplanned work
• code volume
• commit volume
• release speed
• test volume
• code coverage
• exception counts
• build speed
• failure rates
• manual builds
• performance
• latency
• scalability
• response time
• uptime/availability
• resource usage
• revenue
• signups
• cust. sat.
• remediation time
• code quality
• access rates

12. © 2019 SPLUNK INC.
Biz PMO Dev Build QA Sec Stage Ops Biz
Shared Data for Multiple Stakeholders
• time to deliver
• idea to cash
• ROI
• process times
• team efficiency
• unplanned work
• code volume
• commit volume
• release speed
• test volume
• code coverage
• exception counts
• build speed
• failure rates
• manual builds
• performance
• latency
• scalability
• response time
• uptime/availability
• resource usage
• revenue
• signups
• cust. sat.
• remediation time
• code quality
• access rates

13. © 2019 SPLUNK INC.
Biz PMO Dev Build QA Sec Stage Ops Biz
Specific Data For Each Stakeholder
• time to deliver
• idea to cash
• ROI
• process times
• team efficiency
• unplanned work
• code volume
• commit volume
• release speed
• test volume
• code coverage
• exception counts
• build speed
• failure rates
• manual builds
• performance
• latency
• scalability
• response time
• uptime/availability
• resource usage
• revenue
• signups
• cust. sat.
• remediation time
• code quality
• access rates

14. © 2019 SPLUNK INC.
Biz PMO Dev Build QA Sec Stage Ops Biz
Specific Data For Each Stakeholder
• time to deliver
• idea to cash
• ROI
• process times
• team efficiency
• unplanned work
• code volume
• commit volume
• release speed
• test volume
• code coverage
• exception counts
• build speed
• failure rates
• manual builds
• performance
• latency
• scalability
• response time
• uptime/availability
• resource usage
• revenue
• signups
• cust. sat.
• remediation time
• code quality
• access rates

15. © 2019 SPLUNK INC.
Shared Data Platform for Dev and Ops
… and PMO, and QA, and Release, and Security, and Business, and …
ROOT CAUSE
AND ISSUE
RESOLUTION
PROACTIVE
MONITORING
AND REAL-TIME
ALERTING
DELIVER BETTER
QUALITY CODE
FASTER
CLOUD APP AND
INFRASTRUCTURE
MONITORING
MOBILE APP
TROUBLESHOOTING
USER & USAGE
ANALYTICS
Platform for turning machine data into answers

16. © 2019 SPLUNK INC.
How can I speed
up security
investigations and
reduce the impact
of insider threats?
Is my poor app
performance due
to code-level
errors or
infrastructure?
How do I predict
service-level
degradation
before
it occurs?
Do my marketing
campaigns drive
more orders
through the website
or mobile app?
How can I monitor
and analyze data
from tens of
thousands of
sensors in real
time?
Answer Any Question, Across Your Organization
IT
Operations
Application
Performance
Analytics
Security and
Compliance
Business
Analytics
Internet of
Things

17. © 2019 SPLUNK INC.
Outcome – Improve Speed, Quality, and Impact
for Application Development and IT Operations
Accelerate
Delivery Velocity
Get from idea to customer
faster by rapidly finding and
removing delivery bottlenecks,
waste, and other workflow
issues that add to cycle time
“We can monitor automation and
handoffs to deploy 5-10 times a day.”
Increase
Business Impact
Drive continuous improvement
using data-driven feedback
loops to share business-
relevant insight from real user
behavior and application use
“My code isn’t ready until
it’s Splunk-ready.”
Improve
Application Quality
Enhance customer experience
and site reliability by using data
analytics for better dev, test,
release, manage, and secure
decisions for faster MTTI, MTTR
“Our devs are able to find and fix
issues 5-10 times faster.”
Platform for turning machine data into answers

18. © 2019 SPLUNK INC.
Demo

19. © 2019 SPLUNK INC.
Observability
“In control theory, observability is a
measure of how well internal states of
a system can be inferred from
knowledge of its external outputs.
The observability and controllability of
a system are mathematical duals.”
Wikipedia

20. © 2019 SPLUNK INC.
‘Observable’ Metrics and Events
Two distinct machine data sources that have been hard to integrate…until now
Metrics
▶ Numbers describing a particular process or activity
▶ Measured over intervals of time–
i.e., time series data
▶ Common metrics sources:
• System metrics (CPU, memory, disk)
• Infrastructure metrics (AWS CloudWatch)
• Web tracking scripts (Google Analytics)
• Application agents (APM, error tracking)
Events
▶ Immutable record of discrete events that happen
over time
▶ Come in three forms: plain text, structured, binary
▶ Common event sources:
• System and server logs (syslog, journald)
• Firewall and intrusion detection system logs
• Social media feeds (Twitter…)
• Application, platform and server logs (log4j, log4net,
Apache, MySQL, AWS)
Timestamp Metric Name Value Dimensions
1481050800 os.cpu.user 42.12345 hq:us-west-1
Sample Metric
[29/Aug/2018 08:47:05:316503] "POST /cart.do?uid=84e8d742-a31d69&action=remove&&product_id=BS-
2&JSESSIONID=SD6SAL4FF1ADFF9 HTTP 1.1" 200 2569 "http://www.buttercupenterprises.com/product.screen?
product_id=BS-2" "Mozilla/5.0 (Intel Mac OS X 10_12_2) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/57.0.2957.0 Safari/537.36" 98
Sample Log
Equivalent to
1 metric value

21. © 2019 SPLUNK INC.
► Written explicitly into the application specifically for operational analytics
Example Bogus Pseudo-Code:
void submitPurchase(purchaseId)
{
log.info("action=submitPurchaseStart, purchaseId=%d", purchaseId)
//these calls throw an exception on error
submitToCreditCard(...)
generateInvoice(...)
generateFullfillmentOrder(...)
log.info("action=submitPurchaseCompleted, purchaseId=%d", purchaseId)
}
► Log anything that can add value when aggregated, charted or analyzed
• What is my purchase volume, success, failure – by hour, by day, by month?
• How long are purchases taking at different times of day, or days of the week?
• Are transactions failing more or less than they did last month?
• Is the end-to-end service getting slower? Which components are most at fault?
Semantic Logging for Application Telemetry

22. © 2019 SPLUNK INC.
► timestamp
► metric name
► data point
► dimensions (host, instance, etc.)
► More at https://www.splunk.com/en_us/form/a-beginners-guide-to-collectd.html
Creating New Signals -
collectd
A daemon that collects metrics

23. © 2019 SPLUNK INC.
► collects stats data from counters and
timers
► sends aggregates to backend services
(e.g. Splunk, Graphite)
► More at https://github.com/etsy/statsd
Creating New Signals -
statsd
A daemon that listens for statistics
Source: https://medium.com/@itmarketplace.net/golang-example-04-28-16-495e82cb1e3e

24. © 2019 SPLUNK INC.
► collects, filters, buffers log data
► outputs to multiple destinations
► allows advanced processing by
multiple 3rd party systems
Creating New Signals -
fluentd
A daemon that unifies log data
collection

25. © 2019 SPLUNK INC.
Got Splunk? Get Splunk Apps for Dev, Ops, Sec, Biz!
Enable collaborative troubleshooting and knowledge sharing across roles
Apps for DevOps Processes and
Development Tools
Operational Intelligence for On-Premises,
Cloud, and Mobile Operations
*nix
Visit www.splunkbase.com

26. © 2019 SPLUNK INC.
Looking for something else? Some other ideas …
Collaborative troubleshooting and post-incident review
Service intelligence to enable data-driven automation
ChatOps integration for war rooms & post-incident reviews
Application telemetry and metrics for real-time observability
Audit intelligence for the DevOps Lifecycle (DevSecOps)

27. © 2019 SPLUNK INC.© 2019 SPLUNK INC.
Don't forget to rate this session
in the .conf18 mobile app
Thank You.