SlideShare ist ein Scribd-Unternehmen logo
1 von 36
Downloaden Sie, um offline zu lesen
Copyright © 2014 Splunk Inc.
Data Onboarding
Ingestion Without the
Indigestion
David Millis
Client Architect
2
Legal Notice
During the course of this presentation, we may make forward looking statements regarding future events or the
expected performance of the company. We caution you that such statements reflect our current
expectations and estimates based on factors currently known to us and that actual events or results could differ
materially. For important factors that may cause actual results to differ from those contained in our forward-looking
statements, please review our filings with the SEC. The forward-looking statements made in this presentation are
being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation
may not contain current or accurate information. We do not assume any obligation to update any forward looking
statements we may make. In addition, any information about our roadmap outlines our general product direction
and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated
into any contract or other commitment. Splunk undertakes no obligation either to develop the features or
functionality described or to include any such feature or functionality in a future release.
Splunk®, Splunk>®, Listen to Your Data®, The Engine for Machine Data®, Hunk™, Splunk Cloud™,
Splunk Storm® and SPL™ are registered trademarks or trademarks of Splunk Inc. in the United States
and/or other countries. All other brand names, product names or trademarks belong to their respective
owners. © 2014 Splunk Inc. All rights reserved.
• Systematic way to bring new data sources into Splunk
• Make sure that new data is instantly usable
& has maximum value for users
• Goes hand-in-hand with the User Onboarding process
(sold separately)
What is the Data Onboarding Process?
Know Your (Data) Pipeline
The Data Pipeline
The Data Pipeline
Any Questions?
The Data Pipeline
• Input Processors: Monitor, FIFO, UDP, TCP, Scripted
• No events yet-- just a stream of bytes
• Break data stream into 64KB blocks
• Annotate stream with metadata keys (host, source,
sourcetype, index, etc.)
• Can happen on UF, HF or indexer
Inputs– Where it all starts
• Check character set
• Break lines
• Process headers
• Can happen on HF or indexer
Parsing Queue
• Merge lines for multi-line events
• Identify events (finally!)
• Extract timestamps
• Exclude events based on timestamp (MAX_DAYS_AGO, ..)
• Can happen on HF or indexer
Aggregation/Merging Queue
• Do regex replacement (field extraction, punctuation
extraction, event routing, host/source/sourcetype
overrides)
• Annotate events with metadata keys
(host, source, sourcetype, ..)
• Can happen on HF or indexer
Typing Queue
• Output processors: TCP, syslog, HTTP
• Indexandforward
• Sign blocks
• Calculate license volume and throughput metrics
• Index
• Write to disk
• Can happen on HF or indexer
Indexing Queue
The Data Pipeline
Data Pipeline: UF & Indexer
Data Pipeline: HF & Indexer
Data Pipeline: UF, IF & Indexer
Data Onboarding Process
• Pre-board
• Build the index-time configs
• Build the search-time configs
• Create data models
• Document
• Test
• Get ready to deploy
• Bring it!
• Test & Validate
Process Overview
• Identify the specific sourcetype(s) - onboard each separately
• Check for pre-existing app/TA on splunk.com-- don't reinvent the wheel!
• Gather info
• Where does this data originate/reside? How will Splunk collect it?
• Which users/groups will need access to this data? Access controls?
• Determine the indexing volume and data retention requirements
• Will this data need to drive existing dashboards (ES, PCI, etc.)?
• Who is the SME for this data?
• Map it out
• Get a "big enough" sample of the event data
• Identify and map out fields
• Assign sourcetype and TA names according to CIM conventions
Pre-Board
• The Common Information Model (CIM) defines
relationships in the underlying data, while leaving the raw
machine data intact
• A naming convention for fields, eventtypes & tags
• More advanced reporting and correlation requires that the
data be normalized, categorized, and parsed
• CIM-compliant data sources can drive CIM-based
dashboards (ES, PCI, others)
Tangent: What is the CIM and why should I care?
• Identify necessary configs (inputs, props and transforms)
to properly handle:
• timestamp extraction, timezone, event breaking,
sourcetype/host/source assignments
• Do events contain sensitive data (i.e., PII, PAN, etc.)?
Create masking transforms if necessary
• Package all index-time configs into the TA
Build the Index-time configs
• Assign sourcetype according to event format; events with
similar format should have the same sourcetype
• When do I need a separate index?
• When the data volume will be very large, or when it will
be searched exclusively a lot
• When access to the data needs to be controlled
• When the data requires a specific data retention policy
• Resist the temptation to create lots of indexes
Tangent: Best & Worst Practices
• Always specify a sourcetype and index
• Be as specific as possible: use /var/log/fubar.log,
not /var/log/
• Arrange your monitored filesystems to minimize
unnecessary monitored logfiles
• Use a scratch index while testing new inputs
Best & Worst Practices – [monitor]
• Lookout for inadvertent, runaway monitor clauses
• Don’t monitor thousands of files unnecessarily–
that’s the NSA’s job
• From the CLI: splunk show monitor
• From your browser:
https://your_splunkd:8089/services/admin/inputstatus/
TailingProcessor:FileStatus
Best & Worst Practices – [monitor]
• Find & fix index-time problems BEFORE polluting your index
• A try-it-before-you-fry-it interface for figuring out
• Event breaking
• Timestamp recognition
• Timezone assignment
• Provides the necessary props.conf parameter settings
Your friend, the Data PreviewerAnother
Tangent!
Data Onboarding Process
Continued
• Identify "interesting" events which should be tagged with an existing CIM tag
(http://docs.splunk.com/Documentation/CIM/latest/User/Alerts)
• Get a list of all current tags: | rest splunk_server=local /services/admin/tags | rename tag_name as tag,
field_name_value AS definition, eai:acl.app AS app | eval definition_and_app=definition . " (" . app . ")" |
stats values(definition_and_app) as "definitions (app)" by tag | sort +tag
• Get a list of all eventtypes (with associated tags): | rest splunk_server=local /services/admin/eventtypes
| rename title as eventtype, search AS definition, eai:acl.app AS app | table eventtype definition app tags
| sort +eventtype
• Examine the current list of CIM tags: for each "interesting" event, identify which tags should be applied to
each. A particular event may have multiple tags
• Are there new tags which should be created, beyond those in the current CIM tag library? If so, add them
to the CIM library
Build the Search-time Configs:
eventtypes & tags
• Extract "interesting" fields
• If already in your CIM library, name or alias appropriately
• If not already in your CIM library, name according to CIM conventions
• Add lookups for missing/desirable fields
• Lookups may be required to supply CIM-compliant fields/field values (for example,
to convert 'sev=42' to 'severity=medium'
• Make the values more readable for humans
• Put everything into the TA package
Build the Search-time Configs:
extractions & lookups
• Create data models. What will be interesting for end users?
• Document! (Especially the fields, eventtypes & tags)
• Test
• Does this data drive relevant existing dashboards correctly?
• Do the data models work properly / produce correct results?
• Is the TA packaged properly?
• Check with originating user/group; is it OK?
Keep Going
• Determine additional Splunk infrastructure required; can
existing infrastructure & license support this?
• Will new forwarders be required? If so, initiate CR process(es)
• Will firewall changes be required? If so, initiate CR process(es)
• Will new Splunk roles be required? Create & map to AD roles
• Will new app contexts be required? Create app(s) as necessary
• Will new users be added? Create the accounts
Get Ready to Deploy
• Deploy new search heads & indexers as needed
• Install new forwarders as needed
• Deploy new app & TA to search heads & indexers
• Deploy new TA to relevant forwarders
Bring it!
• All sources reporting?
• Event breaking, timestamp, timezone, host, source,
sourcetype?
• Field extractions, aliases, lookups?
• Eventtypes, tags?
• Data model(s)?
• User access?
• Confirm with original requesting user/group: looks OK?
Test & Validate
Done!
• Bring new data sources in correctly the first time
• Reduce the amount of “bad” data in your indexes– and the
time spent dealing with it
• Make the new data immediately useful to ALL users– not
just the ones who originally requested it
• Allow the data to drive all sorts of dashboards without
extra modifications
Gee, This Seems Like a Lot of Work…
• http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Datapipelin
e
• http://wiki.splunk.com/Community:HowIndexingWorks
• http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings
• http://docs.splunk.com/Documentation/CIM/latest/User/Overview
• http://docs.splunk.com/Documentation/CIM/latest/User/Alerts
• http://splunk-base.splunk.com/apps/29008/sos-splunk-on-splunk
Reference
Copyright © 2014 Splunk Inc.
Thank You!
David Millis
dmillis@splunk.com

Weitere ähnliche Inhalte

Was ist angesagt?

Conf2014_SplunkSearchOptimization
Conf2014_SplunkSearchOptimizationConf2014_SplunkSearchOptimization
Conf2014_SplunkSearchOptimizationSplunk
 
Splunk Cloud
Splunk CloudSplunk Cloud
Splunk CloudSplunk
 
Splunk for IT Operations
Splunk for IT OperationsSplunk for IT Operations
Splunk for IT OperationsSplunk
 
Getting Data into Splunk
Getting Data into SplunkGetting Data into Splunk
Getting Data into SplunkSplunk
 
Splunk Overview
Splunk OverviewSplunk Overview
Splunk OverviewSplunk
 
Splunk HTTP Event Collector
Splunk HTTP Event CollectorSplunk HTTP Event Collector
Splunk HTTP Event CollectorSplunk
 
Getting Started with Splunk (Hands-On)
Getting Started with Splunk (Hands-On) Getting Started with Splunk (Hands-On)
Getting Started with Splunk (Hands-On) Splunk
 
Splunk Overview
Splunk OverviewSplunk Overview
Splunk OverviewSplunk
 
Worst Splunk practices...and how to fix them
Worst Splunk practices...and how to fix themWorst Splunk practices...and how to fix them
Worst Splunk practices...and how to fix themSplunk
 
Splunk Architecture overview
Splunk Architecture overviewSplunk Architecture overview
Splunk Architecture overviewAlex Fok
 
Splunk conf2014 - Onboarding Data Into Splunk
Splunk conf2014 - Onboarding Data Into SplunkSplunk conf2014 - Onboarding Data Into Splunk
Splunk conf2014 - Onboarding Data Into SplunkSplunk
 
Getting started with Splunk - Break out Session
Getting started with Splunk - Break out SessionGetting started with Splunk - Break out Session
Getting started with Splunk - Break out SessionGeorg Knon
 
Taking Splunk to the Next Level - Architecture
Taking Splunk to the Next Level - ArchitectureTaking Splunk to the Next Level - Architecture
Taking Splunk to the Next Level - ArchitectureSplunk
 
Taking Splunk to the Next Level - Architecture Breakout Session
Taking Splunk to the Next Level - Architecture Breakout SessionTaking Splunk to the Next Level - Architecture Breakout Session
Taking Splunk to the Next Level - Architecture Breakout SessionSplunk
 
Splunk 101
Splunk 101Splunk 101
Splunk 101Splunk
 
How to justify the economic value of your data investment
How to justify the economic value of your data investmentHow to justify the economic value of your data investment
How to justify the economic value of your data investmentSplunk
 

Was ist angesagt? (20)

Conf2014_SplunkSearchOptimization
Conf2014_SplunkSearchOptimizationConf2014_SplunkSearchOptimization
Conf2014_SplunkSearchOptimization
 
Splunk Cloud
Splunk CloudSplunk Cloud
Splunk Cloud
 
Splunk for IT Operations
Splunk for IT OperationsSplunk for IT Operations
Splunk for IT Operations
 
Getting Data into Splunk
Getting Data into SplunkGetting Data into Splunk
Getting Data into Splunk
 
Splunk Overview
Splunk OverviewSplunk Overview
Splunk Overview
 
Splunk HTTP Event Collector
Splunk HTTP Event CollectorSplunk HTTP Event Collector
Splunk HTTP Event Collector
 
Getting Started with Splunk (Hands-On)
Getting Started with Splunk (Hands-On) Getting Started with Splunk (Hands-On)
Getting Started with Splunk (Hands-On)
 
Splunk Overview
Splunk OverviewSplunk Overview
Splunk Overview
 
Worst Splunk practices...and how to fix them
Worst Splunk practices...and how to fix themWorst Splunk practices...and how to fix them
Worst Splunk practices...and how to fix them
 
Splunk Architecture overview
Splunk Architecture overviewSplunk Architecture overview
Splunk Architecture overview
 
Splunk-Presentation
Splunk-Presentation Splunk-Presentation
Splunk-Presentation
 
Splunk Architecture
Splunk ArchitectureSplunk Architecture
Splunk Architecture
 
Splunk conf2014 - Onboarding Data Into Splunk
Splunk conf2014 - Onboarding Data Into SplunkSplunk conf2014 - Onboarding Data Into Splunk
Splunk conf2014 - Onboarding Data Into Splunk
 
Getting started with Splunk - Break out Session
Getting started with Splunk - Break out SessionGetting started with Splunk - Break out Session
Getting started with Splunk - Break out Session
 
Observability & Datadog
Observability & DatadogObservability & Datadog
Observability & Datadog
 
Taking Splunk to the Next Level - Architecture
Taking Splunk to the Next Level - ArchitectureTaking Splunk to the Next Level - Architecture
Taking Splunk to the Next Level - Architecture
 
Taking Splunk to the Next Level - Architecture Breakout Session
Taking Splunk to the Next Level - Architecture Breakout SessionTaking Splunk to the Next Level - Architecture Breakout Session
Taking Splunk to the Next Level - Architecture Breakout Session
 
Splunk 101
Splunk 101Splunk 101
Splunk 101
 
How to justify the economic value of your data investment
How to justify the economic value of your data investmentHow to justify the economic value of your data investment
How to justify the economic value of your data investment
 
Observability
Observability Observability
Observability
 

Andere mochten auch

ASGARD Splunk Conf 2016
ASGARD Splunk Conf 2016ASGARD Splunk Conf 2016
ASGARD Splunk Conf 2016Keith Kraus
 
Data Onboarding Breakout Session
Data Onboarding Breakout SessionData Onboarding Breakout Session
Data Onboarding Breakout SessionSplunk
 
SplunkLive! Customer Presentation - University of Alabama at Birmingham
SplunkLive! Customer Presentation - University of Alabama at BirminghamSplunkLive! Customer Presentation - University of Alabama at Birmingham
SplunkLive! Customer Presentation - University of Alabama at BirminghamSplunk
 
EMC RecoverPoint Screenshots
EMC RecoverPoint ScreenshotsEMC RecoverPoint Screenshots
EMC RecoverPoint Screenshotsbhenderson
 
What's New in 6.3 + Data On-Boarding
What's New in 6.3 + Data On-BoardingWhat's New in 6.3 + Data On-Boarding
What's New in 6.3 + Data On-BoardingSplunk
 
EMC VPLEX Continuous availability and non disruptive
EMC VPLEX Continuous availability and non disruptiveEMC VPLEX Continuous availability and non disruptive
EMC VPLEX Continuous availability and non disruptivesolarisyougood
 
Modeling and Solving Resource-Constrained Project Scheduling Problems with IB...
Modeling and Solving Resource-Constrained Project Scheduling Problems with IB...Modeling and Solving Resource-Constrained Project Scheduling Problems with IB...
Modeling and Solving Resource-Constrained Project Scheduling Problems with IB...Philippe Laborie
 
Taking Splunk to the Next Level – Architecture
Taking Splunk to the Next Level – ArchitectureTaking Splunk to the Next Level – Architecture
Taking Splunk to the Next Level – ArchitectureSplunk
 
Splunk Dashboarding & Universal Vs. Heavy Forwarders
Splunk Dashboarding & Universal Vs. Heavy ForwardersSplunk Dashboarding & Universal Vs. Heavy Forwarders
Splunk Dashboarding & Universal Vs. Heavy ForwardersHarry McLaren
 
Splunk for DevOps - Faster Insights - Better Code
Splunk for DevOps - Faster Insights - Better CodeSplunk for DevOps - Faster Insights - Better Code
Splunk for DevOps - Faster Insights - Better CodePhilipp Drieger
 
What's New in Splunk Cloud and Enterprise 6.5
What's New in Splunk Cloud and Enterprise 6.5What's New in Splunk Cloud and Enterprise 6.5
What's New in Splunk Cloud and Enterprise 6.5Splunk
 
Emc recoverpoint technical
Emc recoverpoint technicalEmc recoverpoint technical
Emc recoverpoint technicalsolarisyougood
 
Field Extractions: Making Regex Your Buddy
Field Extractions: Making Regex Your BuddyField Extractions: Making Regex Your Buddy
Field Extractions: Making Regex Your BuddyMichael Wilde
 
Splunk資安智慧分析平台
Splunk資安智慧分析平台Splunk資安智慧分析平台
Splunk資安智慧分析平台Ching-Lin Tao
 
Big Data Taiwan 2014 Track1-3: Big Data, Big Challenge — Splunk 幫你解決 Big Data...
Big Data Taiwan 2014 Track1-3: Big Data, Big Challenge — Splunk 幫你解決 Big Data...Big Data Taiwan 2014 Track1-3: Big Data, Big Challenge — Splunk 幫你解決 Big Data...
Big Data Taiwan 2014 Track1-3: Big Data, Big Challenge — Splunk 幫你解決 Big Data...Etu Solution
 

Andere mochten auch (15)

ASGARD Splunk Conf 2016
ASGARD Splunk Conf 2016ASGARD Splunk Conf 2016
ASGARD Splunk Conf 2016
 
Data Onboarding Breakout Session
Data Onboarding Breakout SessionData Onboarding Breakout Session
Data Onboarding Breakout Session
 
SplunkLive! Customer Presentation - University of Alabama at Birmingham
SplunkLive! Customer Presentation - University of Alabama at BirminghamSplunkLive! Customer Presentation - University of Alabama at Birmingham
SplunkLive! Customer Presentation - University of Alabama at Birmingham
 
EMC RecoverPoint Screenshots
EMC RecoverPoint ScreenshotsEMC RecoverPoint Screenshots
EMC RecoverPoint Screenshots
 
What's New in 6.3 + Data On-Boarding
What's New in 6.3 + Data On-BoardingWhat's New in 6.3 + Data On-Boarding
What's New in 6.3 + Data On-Boarding
 
EMC VPLEX Continuous availability and non disruptive
EMC VPLEX Continuous availability and non disruptiveEMC VPLEX Continuous availability and non disruptive
EMC VPLEX Continuous availability and non disruptive
 
Modeling and Solving Resource-Constrained Project Scheduling Problems with IB...
Modeling and Solving Resource-Constrained Project Scheduling Problems with IB...Modeling and Solving Resource-Constrained Project Scheduling Problems with IB...
Modeling and Solving Resource-Constrained Project Scheduling Problems with IB...
 
Taking Splunk to the Next Level – Architecture
Taking Splunk to the Next Level – ArchitectureTaking Splunk to the Next Level – Architecture
Taking Splunk to the Next Level – Architecture
 
Splunk Dashboarding & Universal Vs. Heavy Forwarders
Splunk Dashboarding & Universal Vs. Heavy ForwardersSplunk Dashboarding & Universal Vs. Heavy Forwarders
Splunk Dashboarding & Universal Vs. Heavy Forwarders
 
Splunk for DevOps - Faster Insights - Better Code
Splunk for DevOps - Faster Insights - Better CodeSplunk for DevOps - Faster Insights - Better Code
Splunk for DevOps - Faster Insights - Better Code
 
What's New in Splunk Cloud and Enterprise 6.5
What's New in Splunk Cloud and Enterprise 6.5What's New in Splunk Cloud and Enterprise 6.5
What's New in Splunk Cloud and Enterprise 6.5
 
Emc recoverpoint technical
Emc recoverpoint technicalEmc recoverpoint technical
Emc recoverpoint technical
 
Field Extractions: Making Regex Your Buddy
Field Extractions: Making Regex Your BuddyField Extractions: Making Regex Your Buddy
Field Extractions: Making Regex Your Buddy
 
Splunk資安智慧分析平台
Splunk資安智慧分析平台Splunk資安智慧分析平台
Splunk資安智慧分析平台
 
Big Data Taiwan 2014 Track1-3: Big Data, Big Challenge — Splunk 幫你解決 Big Data...
Big Data Taiwan 2014 Track1-3: Big Data, Big Challenge — Splunk 幫你解決 Big Data...Big Data Taiwan 2014 Track1-3: Big Data, Big Challenge — Splunk 幫你解決 Big Data...
Big Data Taiwan 2014 Track1-3: Big Data, Big Challenge — Splunk 幫你解決 Big Data...
 

Ähnlich wie SplunkLive! Presentation - Data Onboarding with Splunk

SplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunk
 
SplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunk
 
Importance of ‘Centralized Event collection’ and BigData platform for Analysis !
Importance of ‘Centralized Event collection’ and BigData platform for Analysis !Importance of ‘Centralized Event collection’ and BigData platform for Analysis !
Importance of ‘Centralized Event collection’ and BigData platform for Analysis !Piyush Kumar
 
SplunkLive! Zurich 2018: Integrating Metrics and Logs
SplunkLive! Zurich 2018: Integrating Metrics and LogsSplunkLive! Zurich 2018: Integrating Metrics and Logs
SplunkLive! Zurich 2018: Integrating Metrics and LogsSplunk
 
SplunkLive! Frankfurt 2018 - Integrating Metrics & Logs
SplunkLive! Frankfurt 2018 - Integrating Metrics & LogsSplunkLive! Frankfurt 2018 - Integrating Metrics & Logs
SplunkLive! Frankfurt 2018 - Integrating Metrics & LogsSplunk
 
Machine Data Is EVERYWHERE: Use It for Testing
Machine Data Is EVERYWHERE: Use It for TestingMachine Data Is EVERYWHERE: Use It for Testing
Machine Data Is EVERYWHERE: Use It for TestingTechWell
 
Actionable Insights - Thompson
Actionable Insights - ThompsonActionable Insights - Thompson
Actionable Insights - ThompsonProlifics
 
Splunk MINT for Mobile Intelligence and Splunk App for Stream for Enhanced Op...
Splunk MINT for Mobile Intelligence and Splunk App for Stream for Enhanced Op...Splunk MINT for Mobile Intelligence and Splunk App for Stream for Enhanced Op...
Splunk MINT for Mobile Intelligence and Splunk App for Stream for Enhanced Op...Splunk
 
rough-work.pptx
rough-work.pptxrough-work.pptx
rough-work.pptxsharpan
 
SplunkLive! - Getting started with Splunk
SplunkLive! - Getting started with SplunkSplunkLive! - Getting started with Splunk
SplunkLive! - Getting started with SplunkSplunk
 
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...Databricks
 
SplunkLive! Munich 2018: Integrating Metrics and Logs
SplunkLive! Munich 2018: Integrating Metrics and LogsSplunkLive! Munich 2018: Integrating Metrics and Logs
SplunkLive! Munich 2018: Integrating Metrics and LogsSplunk
 
Monitoring and Instrumentation Strategies: Tips and Best Practices - AppSphere16
Monitoring and Instrumentation Strategies: Tips and Best Practices - AppSphere16Monitoring and Instrumentation Strategies: Tips and Best Practices - AppSphere16
Monitoring and Instrumentation Strategies: Tips and Best Practices - AppSphere16AppDynamics
 
SplunkLive! Frankfurt 2018 - Legacy SIEM to Splunk, How to Conquer Migration ...
SplunkLive! Frankfurt 2018 - Legacy SIEM to Splunk, How to Conquer Migration ...SplunkLive! Frankfurt 2018 - Legacy SIEM to Splunk, How to Conquer Migration ...
SplunkLive! Frankfurt 2018 - Legacy SIEM to Splunk, How to Conquer Migration ...Splunk
 
What’s New: Splunk App for Stream and Splunk MINT
What’s New: Splunk App for Stream and Splunk MINTWhat’s New: Splunk App for Stream and Splunk MINT
What’s New: Splunk App for Stream and Splunk MINTSplunk
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseSplunk
 
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...Amazon Web Services
 
SplunkLive! London - Splunk App for Stream & MINT Breakout
SplunkLive! London - Splunk App for Stream & MINT BreakoutSplunkLive! London - Splunk App for Stream & MINT Breakout
SplunkLive! London - Splunk App for Stream & MINT BreakoutSplunk
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream ProcessingGuido Schmutz
 
StreamCentral for the IT Professional
StreamCentral for the IT ProfessionalStreamCentral for the IT Professional
StreamCentral for the IT ProfessionalRaheel Retiwalla
 

Ähnlich wie SplunkLive! Presentation - Data Onboarding with Splunk (20)

SplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding OverviewSplunkLive! Frankfurt 2018 - Data Onboarding Overview
SplunkLive! Frankfurt 2018 - Data Onboarding Overview
 
SplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding OverviewSplunkLive! Munich 2018: Data Onboarding Overview
SplunkLive! Munich 2018: Data Onboarding Overview
 
Importance of ‘Centralized Event collection’ and BigData platform for Analysis !
Importance of ‘Centralized Event collection’ and BigData platform for Analysis !Importance of ‘Centralized Event collection’ and BigData platform for Analysis !
Importance of ‘Centralized Event collection’ and BigData platform for Analysis !
 
SplunkLive! Zurich 2018: Integrating Metrics and Logs
SplunkLive! Zurich 2018: Integrating Metrics and LogsSplunkLive! Zurich 2018: Integrating Metrics and Logs
SplunkLive! Zurich 2018: Integrating Metrics and Logs
 
SplunkLive! Frankfurt 2018 - Integrating Metrics & Logs
SplunkLive! Frankfurt 2018 - Integrating Metrics & LogsSplunkLive! Frankfurt 2018 - Integrating Metrics & Logs
SplunkLive! Frankfurt 2018 - Integrating Metrics & Logs
 
Machine Data Is EVERYWHERE: Use It for Testing
Machine Data Is EVERYWHERE: Use It for TestingMachine Data Is EVERYWHERE: Use It for Testing
Machine Data Is EVERYWHERE: Use It for Testing
 
Actionable Insights - Thompson
Actionable Insights - ThompsonActionable Insights - Thompson
Actionable Insights - Thompson
 
Splunk MINT for Mobile Intelligence and Splunk App for Stream for Enhanced Op...
Splunk MINT for Mobile Intelligence and Splunk App for Stream for Enhanced Op...Splunk MINT for Mobile Intelligence and Splunk App for Stream for Enhanced Op...
Splunk MINT for Mobile Intelligence and Splunk App for Stream for Enhanced Op...
 
rough-work.pptx
rough-work.pptxrough-work.pptx
rough-work.pptx
 
SplunkLive! - Getting started with Splunk
SplunkLive! - Getting started with SplunkSplunkLive! - Getting started with Splunk
SplunkLive! - Getting started with Splunk
 
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
 
SplunkLive! Munich 2018: Integrating Metrics and Logs
SplunkLive! Munich 2018: Integrating Metrics and LogsSplunkLive! Munich 2018: Integrating Metrics and Logs
SplunkLive! Munich 2018: Integrating Metrics and Logs
 
Monitoring and Instrumentation Strategies: Tips and Best Practices - AppSphere16
Monitoring and Instrumentation Strategies: Tips and Best Practices - AppSphere16Monitoring and Instrumentation Strategies: Tips and Best Practices - AppSphere16
Monitoring and Instrumentation Strategies: Tips and Best Practices - AppSphere16
 
SplunkLive! Frankfurt 2018 - Legacy SIEM to Splunk, How to Conquer Migration ...
SplunkLive! Frankfurt 2018 - Legacy SIEM to Splunk, How to Conquer Migration ...SplunkLive! Frankfurt 2018 - Legacy SIEM to Splunk, How to Conquer Migration ...
SplunkLive! Frankfurt 2018 - Legacy SIEM to Splunk, How to Conquer Migration ...
 
What’s New: Splunk App for Stream and Splunk MINT
What’s New: Splunk App for Stream and Splunk MINTWhat’s New: Splunk App for Stream and Splunk MINT
What’s New: Splunk App for Stream and Splunk MINT
 
Getting Started with Splunk Enterprise
Getting Started with Splunk EnterpriseGetting Started with Splunk Enterprise
Getting Started with Splunk Enterprise
 
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
Using AWS to Build a Scalable Big Data Management & Processing Service (BDT40...
 
SplunkLive! London - Splunk App for Stream & MINT Breakout
SplunkLive! London - Splunk App for Stream & MINT BreakoutSplunkLive! London - Splunk App for Stream & MINT Breakout
SplunkLive! London - Splunk App for Stream & MINT Breakout
 
Introduction to Stream Processing
Introduction to Stream ProcessingIntroduction to Stream Processing
Introduction to Stream Processing
 
StreamCentral for the IT Professional
StreamCentral for the IT ProfessionalStreamCentral for the IT Professional
StreamCentral for the IT Professional
 

Mehr von Splunk

.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routineSplunk
 
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTVSplunk
 
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Navegando la normativa SOX (Telefónica).conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Navegando la normativa SOX (Telefónica)Splunk
 
.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - Raiffeisen Bank International.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - Raiffeisen Bank InternationalSplunk
 
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett .conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett Splunk
 
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär).conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)Splunk
 
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu....conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...Splunk
 
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever....conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...Splunk
 
.conf go 2023 - De NOC a CSIRT (Cellnex)
.conf go 2023 - De NOC a CSIRT (Cellnex).conf go 2023 - De NOC a CSIRT (Cellnex)
.conf go 2023 - De NOC a CSIRT (Cellnex)Splunk
 
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)Splunk
 
Splunk - BMW connects business and IT with data driven operations SRE and O11y
Splunk - BMW connects business and IT with data driven operations SRE and O11ySplunk - BMW connects business and IT with data driven operations SRE and O11y
Splunk - BMW connects business and IT with data driven operations SRE and O11ySplunk
 
Splunk x Freenet - .conf Go Köln
Splunk x Freenet - .conf Go KölnSplunk x Freenet - .conf Go Köln
Splunk x Freenet - .conf Go KölnSplunk
 
Splunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go KölnSplunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go KölnSplunk
 
Data foundations building success, at city scale – Imperial College London
 Data foundations building success, at city scale – Imperial College London Data foundations building success, at city scale – Imperial College London
Data foundations building success, at city scale – Imperial College LondonSplunk
 
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...Splunk
 
SOC, Amore Mio! | Security Webinar
SOC, Amore Mio! | Security WebinarSOC, Amore Mio! | Security Webinar
SOC, Amore Mio! | Security WebinarSplunk
 
.conf Go 2022 - Observability Session
.conf Go 2022 - Observability Session.conf Go 2022 - Observability Session
.conf Go 2022 - Observability SessionSplunk
 
.conf Go Zurich 2022 - Keynote
.conf Go Zurich 2022 - Keynote.conf Go Zurich 2022 - Keynote
.conf Go Zurich 2022 - KeynoteSplunk
 
.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform Session.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform SessionSplunk
 
.conf Go Zurich 2022 - Security Session
.conf Go Zurich 2022 - Security Session.conf Go Zurich 2022 - Security Session
.conf Go Zurich 2022 - Security SessionSplunk
 

Mehr von Splunk (20)

.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine
 
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
 
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Navegando la normativa SOX (Telefónica).conf Go 2023 - Navegando la normativa SOX (Telefónica)
.conf Go 2023 - Navegando la normativa SOX (Telefónica)
 
.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - Raiffeisen Bank International.conf Go 2023 - Raiffeisen Bank International
.conf Go 2023 - Raiffeisen Bank International
 
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett .conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
.conf Go 2023 - På liv og død Om sikkerhetsarbeid i Norsk helsenett
 
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär).conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
.conf Go 2023 - Many roads lead to Rome - this was our journey (Julius Bär)
 
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu....conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
.conf Go 2023 - Das passende Rezept für die digitale (Security) Revolution zu...
 
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever....conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
.conf go 2023 - Cyber Resilienz – Herausforderungen und Ansatz für Energiever...
 
.conf go 2023 - De NOC a CSIRT (Cellnex)
.conf go 2023 - De NOC a CSIRT (Cellnex).conf go 2023 - De NOC a CSIRT (Cellnex)
.conf go 2023 - De NOC a CSIRT (Cellnex)
 
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
conf go 2023 - El camino hacia la ciberseguridad (ABANCA)
 
Splunk - BMW connects business and IT with data driven operations SRE and O11y
Splunk - BMW connects business and IT with data driven operations SRE and O11ySplunk - BMW connects business and IT with data driven operations SRE and O11y
Splunk - BMW connects business and IT with data driven operations SRE and O11y
 
Splunk x Freenet - .conf Go Köln
Splunk x Freenet - .conf Go KölnSplunk x Freenet - .conf Go Köln
Splunk x Freenet - .conf Go Köln
 
Splunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go KölnSplunk Security Session - .conf Go Köln
Splunk Security Session - .conf Go Köln
 
Data foundations building success, at city scale – Imperial College London
 Data foundations building success, at city scale – Imperial College London Data foundations building success, at city scale – Imperial College London
Data foundations building success, at city scale – Imperial College London
 
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
Splunk: How Vodafone established Operational Analytics in a Hybrid Environmen...
 
SOC, Amore Mio! | Security Webinar
SOC, Amore Mio! | Security WebinarSOC, Amore Mio! | Security Webinar
SOC, Amore Mio! | Security Webinar
 
.conf Go 2022 - Observability Session
.conf Go 2022 - Observability Session.conf Go 2022 - Observability Session
.conf Go 2022 - Observability Session
 
.conf Go Zurich 2022 - Keynote
.conf Go Zurich 2022 - Keynote.conf Go Zurich 2022 - Keynote
.conf Go Zurich 2022 - Keynote
 
.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform Session.conf Go Zurich 2022 - Platform Session
.conf Go Zurich 2022 - Platform Session
 
.conf Go Zurich 2022 - Security Session
.conf Go Zurich 2022 - Security Session.conf Go Zurich 2022 - Security Session
.conf Go Zurich 2022 - Security Session
 

Kürzlich hochgeladen

Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesMd Hossain Ali
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXTarek Kalaji
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarPrecisely
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7DianaGray10
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URLRuncy Oommen
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?IES VE
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsSafe Software
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfAijun Zhang
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDELiveplex
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Commit University
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1DianaGray10
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdfPedro Manuel
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 

Kürzlich hochgeladen (20)

Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just MinutesAI Fame Rush Review – Virtual Influencer Creation In Just Minutes
AI Fame Rush Review – Virtual Influencer Creation In Just Minutes
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
VoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBXVoIP Service and Marketing using Odoo and Asterisk PBX
VoIP Service and Marketing using Odoo and Asterisk PBX
 
AI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity WebinarAI You Can Trust - Ensuring Success with Data Integrity Webinar
AI You Can Trust - Ensuring Success with Data Integrity Webinar
 
UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7UiPath Studio Web workshop series - Day 7
UiPath Studio Web workshop series - Day 7
 
Designing A Time bound resource download URL
Designing A Time bound resource download URLDesigning A Time bound resource download URL
Designing A Time bound resource download URL
 
How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?How Accurate are Carbon Emissions Projections?
How Accurate are Carbon Emissions Projections?
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
20150722 - AGV
20150722 - AGV20150722 - AGV
20150722 - AGV
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration WorkflowsIgniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
 
Machine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdfMachine Learning Model Validation (Aijun Zhang 2024).pdf
Machine Learning Model Validation (Aijun Zhang 2024).pdf
 
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDEADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
ADOPTING WEB 3 FOR YOUR BUSINESS: A STEP-BY-STEP GUIDE
 
Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)Crea il tuo assistente AI con lo Stregatto (open source python framework)
Crea il tuo assistente AI con lo Stregatto (open source python framework)
 
Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1Secure your environment with UiPath and CyberArk technologies - Session 1
Secure your environment with UiPath and CyberArk technologies - Session 1
 
Nanopower In Semiconductor Industry.pdf
Nanopower  In Semiconductor Industry.pdfNanopower  In Semiconductor Industry.pdf
Nanopower In Semiconductor Industry.pdf
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 

SplunkLive! Presentation - Data Onboarding with Splunk

  • 1. Copyright © 2014 Splunk Inc. Data Onboarding Ingestion Without the Indigestion David Millis Client Architect
  • 2. 2 Legal Notice During the course of this presentation, we may make forward looking statements regarding future events or the expected performance of the company. We caution you that such statements reflect our current expectations and estimates based on factors currently known to us and that actual events or results could differ materially. For important factors that may cause actual results to differ from those contained in our forward-looking statements, please review our filings with the SEC. The forward-looking statements made in this presentation are being made as of the time and date of its live presentation. If reviewed after its live presentation, this presentation may not contain current or accurate information. We do not assume any obligation to update any forward looking statements we may make. In addition, any information about our roadmap outlines our general product direction and is subject to change at any time without notice. It is for informational purposes only and shall not be incorporated into any contract or other commitment. Splunk undertakes no obligation either to develop the features or functionality described or to include any such feature or functionality in a future release. Splunk®, Splunk>®, Listen to Your Data®, The Engine for Machine Data®, Hunk™, Splunk Cloud™, Splunk Storm® and SPL™ are registered trademarks or trademarks of Splunk Inc. in the United States and/or other countries. All other brand names, product names or trademarks belong to their respective owners. © 2014 Splunk Inc. All rights reserved.
  • 3. • Systematic way to bring new data sources into Splunk • Make sure that new data is instantly usable & has maximum value for users • Goes hand-in-hand with the User Onboarding process (sold separately) What is the Data Onboarding Process?
  • 4. Know Your (Data) Pipeline
  • 8. • Input Processors: Monitor, FIFO, UDP, TCP, Scripted • No events yet-- just a stream of bytes • Break data stream into 64KB blocks • Annotate stream with metadata keys (host, source, sourcetype, index, etc.) • Can happen on UF, HF or indexer Inputs– Where it all starts
  • 9. • Check character set • Break lines • Process headers • Can happen on HF or indexer Parsing Queue
  • 10. • Merge lines for multi-line events • Identify events (finally!) • Extract timestamps • Exclude events based on timestamp (MAX_DAYS_AGO, ..) • Can happen on HF or indexer Aggregation/Merging Queue
  • 11. • Do regex replacement (field extraction, punctuation extraction, event routing, host/source/sourcetype overrides) • Annotate events with metadata keys (host, source, sourcetype, ..) • Can happen on HF or indexer Typing Queue
  • 12. • Output processors: TCP, syslog, HTTP • Indexandforward • Sign blocks • Calculate license volume and throughput metrics • Index • Write to disk • Can happen on HF or indexer Indexing Queue
  • 14. Data Pipeline: UF & Indexer
  • 15. Data Pipeline: HF & Indexer
  • 16. Data Pipeline: UF, IF & Indexer
  • 18. • Pre-board • Build the index-time configs • Build the search-time configs • Create data models • Document • Test • Get ready to deploy • Bring it! • Test & Validate Process Overview
  • 19. • Identify the specific sourcetype(s) - onboard each separately • Check for pre-existing app/TA on splunk.com-- don't reinvent the wheel! • Gather info • Where does this data originate/reside? How will Splunk collect it? • Which users/groups will need access to this data? Access controls? • Determine the indexing volume and data retention requirements • Will this data need to drive existing dashboards (ES, PCI, etc.)? • Who is the SME for this data? • Map it out • Get a "big enough" sample of the event data • Identify and map out fields • Assign sourcetype and TA names according to CIM conventions Pre-Board
  • 20. • The Common Information Model (CIM) defines relationships in the underlying data, while leaving the raw machine data intact • A naming convention for fields, eventtypes & tags • More advanced reporting and correlation requires that the data be normalized, categorized, and parsed • CIM-compliant data sources can drive CIM-based dashboards (ES, PCI, others) Tangent: What is the CIM and why should I care?
  • 21. • Identify necessary configs (inputs, props and transforms) to properly handle: • timestamp extraction, timezone, event breaking, sourcetype/host/source assignments • Do events contain sensitive data (i.e., PII, PAN, etc.)? Create masking transforms if necessary • Package all index-time configs into the TA Build the Index-time configs
  • 22. • Assign sourcetype according to event format; events with similar format should have the same sourcetype • When do I need a separate index? • When the data volume will be very large, or when it will be searched exclusively a lot • When access to the data needs to be controlled • When the data requires a specific data retention policy • Resist the temptation to create lots of indexes Tangent: Best & Worst Practices
  • 23. • Always specify a sourcetype and index • Be as specific as possible: use /var/log/fubar.log, not /var/log/ • Arrange your monitored filesystems to minimize unnecessary monitored logfiles • Use a scratch index while testing new inputs Best & Worst Practices – [monitor]
  • 24. • Lookout for inadvertent, runaway monitor clauses • Don’t monitor thousands of files unnecessarily– that’s the NSA’s job • From the CLI: splunk show monitor • From your browser: https://your_splunkd:8089/services/admin/inputstatus/ TailingProcessor:FileStatus Best & Worst Practices – [monitor]
  • 25. • Find & fix index-time problems BEFORE polluting your index • A try-it-before-you-fry-it interface for figuring out • Event breaking • Timestamp recognition • Timezone assignment • Provides the necessary props.conf parameter settings Your friend, the Data PreviewerAnother Tangent!
  • 27. • Identify "interesting" events which should be tagged with an existing CIM tag (http://docs.splunk.com/Documentation/CIM/latest/User/Alerts) • Get a list of all current tags: | rest splunk_server=local /services/admin/tags | rename tag_name as tag, field_name_value AS definition, eai:acl.app AS app | eval definition_and_app=definition . " (" . app . ")" | stats values(definition_and_app) as "definitions (app)" by tag | sort +tag • Get a list of all eventtypes (with associated tags): | rest splunk_server=local /services/admin/eventtypes | rename title as eventtype, search AS definition, eai:acl.app AS app | table eventtype definition app tags | sort +eventtype • Examine the current list of CIM tags: for each "interesting" event, identify which tags should be applied to each. A particular event may have multiple tags • Are there new tags which should be created, beyond those in the current CIM tag library? If so, add them to the CIM library Build the Search-time Configs: eventtypes & tags
  • 28. • Extract "interesting" fields • If already in your CIM library, name or alias appropriately • If not already in your CIM library, name according to CIM conventions • Add lookups for missing/desirable fields • Lookups may be required to supply CIM-compliant fields/field values (for example, to convert 'sev=42' to 'severity=medium' • Make the values more readable for humans • Put everything into the TA package Build the Search-time Configs: extractions & lookups
  • 29. • Create data models. What will be interesting for end users? • Document! (Especially the fields, eventtypes & tags) • Test • Does this data drive relevant existing dashboards correctly? • Do the data models work properly / produce correct results? • Is the TA packaged properly? • Check with originating user/group; is it OK? Keep Going
  • 30. • Determine additional Splunk infrastructure required; can existing infrastructure & license support this? • Will new forwarders be required? If so, initiate CR process(es) • Will firewall changes be required? If so, initiate CR process(es) • Will new Splunk roles be required? Create & map to AD roles • Will new app contexts be required? Create app(s) as necessary • Will new users be added? Create the accounts Get Ready to Deploy
  • 31. • Deploy new search heads & indexers as needed • Install new forwarders as needed • Deploy new app & TA to search heads & indexers • Deploy new TA to relevant forwarders Bring it!
  • 32. • All sources reporting? • Event breaking, timestamp, timezone, host, source, sourcetype? • Field extractions, aliases, lookups? • Eventtypes, tags? • Data model(s)? • User access? • Confirm with original requesting user/group: looks OK? Test & Validate
  • 33. Done!
  • 34. • Bring new data sources in correctly the first time • Reduce the amount of “bad” data in your indexes– and the time spent dealing with it • Make the new data immediately useful to ALL users– not just the ones who originally requested it • Allow the data to drive all sorts of dashboards without extra modifications Gee, This Seems Like a Lot of Work…
  • 35. • http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Datapipelin e • http://wiki.splunk.com/Community:HowIndexingWorks • http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings • http://docs.splunk.com/Documentation/CIM/latest/User/Overview • http://docs.splunk.com/Documentation/CIM/latest/User/Alerts • http://splunk-base.splunk.com/apps/29008/sos-splunk-on-splunk Reference
  • 36. Copyright © 2014 Splunk Inc. Thank You! David Millis dmillis@splunk.com

Hinweis der Redaktion

  1. Splunk safe harbor statement.
  2. This section should take ~10 minutes
  3. This section should take ~10 minutes
  4. This section should take ~10 minutes
  5. This section should take ~10 minutes