2. Intuit—Financial Management Solutions
Applications include:
– QuickBooks (Online/Desktop) -
Small Business financial software
– Intuit Partner Platform (IPP) SaaS platform
App Center - Main customer site
Data services - Cloud Sync Service
– Intuit Market - Custom checks and forms
for Intuit products
2
3. Mark Russell
Worked for Intuit for 16 years
Last 3 years in Operations (Prod Operations and Monitoring)
– Prod Operations
– IPP App center- IIS/custom apps/SQL
– IPP Data Services – Apache/Java/Oracle
– Monitoring
– Primarily Splunk focused - supporting entire FMS organization
– Consulting other group on starting with Splunk
– Admin, internal user consulting, development
“If someone took Splunk away ...I'd Hide!”
3
4. Splunk at Intuit FMS
Real-time Proactive
Business Insight
Data Center
Operations
450 GB
Splunk 4.2
Apps/Dashboards
220 GB
FMS customers: ~4.5 Million (FY12)
Splunk 4.1 FMS revenue: ~ $700M (FY12)
Current Total Intuit License: 4 TB
120 GB
Reactive
4
5. What I Inherited
– Splunk 4.1
Started with 2 Splunk Servers
– Custom Splunk app
built by Professional Services
– Under the impression that
dashboards need to be created by
Professional Services
– Dashboard Creation not part of
customer driven development.
5
6. Where We Were in Summer 2011
Task: Preparing for data center move
Problems:
– New Infrastructure and system
– Bare metal to virtual machines for applications
– Performance concerns
– Limited budget and resources for monitoring
How:
– Upgraded to Splunk 4.2
– College Interns + Splunk Dashboard project
– Learn Splunk, learned our app and logs.
– Splunk Office hours with Splunk Peeps
6
7. What we got
Results:
– Splunk Dashboards
App Performance
Troubleshooting
System Performance
Usage Metrics
– PS to college interns
– Successful switch over to new
data center!
7
9. Summer/Fall 2012: Building HA/DR
Building secondary Data Center (DR)
Doubled our environments including Splunk
Preparing for DC Switch over
– Upgraded to Splunk 4.3
– Expanded our existing dashboard
– New Dashboard
Primary concern was performance
9
10. Summer/Fall 2012: HA/DR Live Test
Performance comparisons: Prod vs DR
Specialized Search head:
– License server had access to both DC’s
– New dashboard to compare DC metrics
simultaneously
– Decision management –thresholds for
performance
– Scheduled reports sent out in PDF
– Relieved stress on core search heads
2 switchovers from Primary DC to DR
10
11. Future Goals and Challenges
Splunk 5.0
– Deployment Server?
Splunk Architecture – Enterprise-level ?
– 8 separate Splunk instances in Production alone (30 Splunk servers)
– Need to consolidate, but DC design mandates separate instances
– Growth – more users, more logs.
Virtual Architecture
– Proves challenging for both Applications and Splunk (NFS)
– Lack of visibility to physical infrastructure
Splunk driven from Apps instead of Security or IT
Implement IT operations based Splunk Apps
– VMware, NetApp
11
12. Splunk Dashboards – From Operations to Execs
IT Ops Dev Marketing CEO
Look-up / Search / App management User behavior User Acquisition
Splunk Use
Troubleshooting Dashboards
12
