ELK Solutions Enablement Session - 17th March'2020

Confidentialinformation,forinternaluseonly
Elastic Stack
Webinar On
17th March 2020

Agenda
• Elastic Stack Product.
• Use Cases.
• Ashnik Use Cases.
• Demo
• Q & A

Store, Search, &
Analyze
Visualize &
Manage
Ingest
Elastic Stack
Kibana
Elasticsearch
Beats Logstash
Elastic Stack

Solutions
Store, Search, &
Analyze
Visualize &
Manage
Ingest
Kibana
Elasticsearch
Beats Logstash
Elastic Stack
SaaS SELF-MANAGED
App Search Site Search Enterprise Search
FUTURE
Metrics APM
Business Analytics
Logging Security AnalyticsUptime
Maps

● Scalable
● Real-time
● Highly available
● Developer-friendly
● Versatile storage
● Query & aggregations
Elasticsearch
Heart of the Elastic Stack

MACHINE LEARNING
GRAP
H
TEMPORAL
QUERY
GEOSPATIA
L
AGGREGATIO
N

● Visualize and explore
● Manage and monitor
● Share and report
● Developer tools
● Time-series analysis
● Geospatial exploration
Kibana
Window into the Elastic Stack

8
All the visualizations you expect, and then some more

Explore your time series data using specialized UIs

10
Create live pixel-perfect presentations using Canvas

Explore geo data on maps, powered by Elastic Maps Service

Search. Scroll. Discover. Make sense of your data.

13
OOTB dashboards for 50+ (and growing) data sources

Manage your data, and the stack

● Ship from any source
● Transform at the edge
● Docker and k8s ready
● Cloud metadata enrichment
● 70+ community Beats
● 50+ modules
Beats
Lightweight data shippers

FILEBEAT
Log Files
METRICBEAT
Metrics
PACKETBEAT
Network Data
WINLOGBEAT
Window Events
HEARTBEAT
Uptime Monitoring
AUDITBEAT
Audit Data
FUNCTIONBEAT
Serverless Monitoring
Plus a growing set of community Beats

● Flexible ETL engine
● Parse & transform data
● Many inputs & outputs
● Horizontally scalable
● 200+ plugins
Logstash
Data processing pipeline

Modules
Data to dashboards in 5 minutes
Turnkey for many formats
Automated data parsing
Out of the box dashboards
Preconfigured ML jobs

● Two-way connector
● Backup ES data to Hadoop
● See Hadoop data in Kibana
● Search on your Hive data
● Spark / Storm support
ES-Hadoop
Elasticsearch Hadoop connector

Elasticsearch Kibana
ES-Hadoop
Backup Elasticsearch
data to HDFS
Move data between
Elasticsearch & Hadoop

21
Stack Features
Powerful feature that apply across use cases

Security
Granular and tightly integrated
Authentication
Native (built-in)
3rd Party (LDAP and AD)
SSO (SAML & Kerberos)
Custom (add your own)
Granular Controls
Document & field level permissions
Integrated with Kibana Spaces
Encryption
In transit (TLS & SSL)
At rest (using dmcrypt)
And more (audit logs, IP filters,...)

Alerting
Alert on anything you can query
Powered by Elasticsearch
Alert on any Elasticsearch query
Distributed execution
Highly available
Notifications
Email, Slack, PagerDuty.
Custom (webhook)
Stack Integrations
Machine learning, Monitoring, and
Reporting

Monitoring
Elastic Stack health at a glance
Full Stack Monitoring
Kibana UIs & dashboards
Track multiple clusters
Vital stats at all levels
Performance Optimization
Optimize performance
Capacity planning
Root cause analysis
Automated Health Alerts
Use with alerting & ML

Assets:
- STATIC showing the generate report option in
Canvas
Reporting
Share the Kibana <3
Export to PDF or CSV
Dashboards & visuals
Canvas workpads
Saved searches
Automate Reporting
Generate on a schedule
Trigger on a condition
Delivered to your inbox

Assets:
- Single UI Screenshot
Graph
Find meaningful connections
Same data. New views.
Uses Elasticsearch relevance features
Includes an API & UI
Use Cases
Recommendations
Fraud discovery
Threat hunting
Behavior analysis

Machine Learning
Detect the unusual in your data
Automated Anomaly Detection
Unsupervised algorithms
Continuous (online) model
Single & multiple time series
Population outliers
Forecasting
Many Use Cases
IT Operations
Security Analytics
Business KPIs
APM

Elasticsearch SQL
SQL with the Elasticsearch twist
Power of Elasticsearch
Full text search
Relevance scoring
Scale & speed
On Ramp to Full Query DSL
Explain & Translate APIs
Connect to 3rd Party Tools
JDBC client
ODBC client

Data Rollups
You know, for saving space
Rollup Data into Coarser Buckets
Save on disk space
Automate via a rollup job
Query just like regular data
Great for metrics use cases
Kibana Support
Rollups Management UI
Visualize rolled up data

30
Logs. Metrics. APM.
The operations trifecta in one place

31
Operational Monitoring
Unify Logs + Metrics + APM
Ingest
Rich ecosystem of connectors
Extensible ingest pipelines
Developer friendly APIs
Exploration
Turnkey solution UIs
OOTB dashboards
Live presentations
Analytics
Anomaly detection
Trending & forecasting
Flexible alerting tools

32
Ingest
Exploration
OOTB dashboards
Live presentations
Analytics
Anomaly detection

33
Ingest
Exploration
OOTB dashboards
Live presentations
Analytics
Anomaly detection

34
APM
Open Source
Language & Agents
Java, Go, RUM, Node, Python, Ruby,
and more on the way.
Dedicated UIs
Streamline APM workflows
Distributed tracing
Just Another Index
Correlate with other data
Leverage all stack features

35
APM
Open Source
Language & Agents
Dedicated UIs
Distributed tracing
Just Another Index

36
APM
Open Source
Language & Agents
Dedicated UIs
Distributed tracing
Just Another Index

37
Security Analytics
Same Data. Different Questions

38
Security Analytics
Same data. Different questions.
Ingest
Ecosystem of connectors
Elastic Common Schema (ECS)
Analytics
Ad hoc queries @ scale
Graph analytics
Machine learning
Detect, Hunt, Investigate
Automated attack detection
Interactive threat hunting
Investigation at speed of thought

39
App Search
Powered by Swiftype, backed by Elasticsearch

Elastic App Search Service
Powered by Swiftype, built on the Elastic Stack
A powerful set of APIs and developer
tools designed for developers building
rich, user-facing search applications
Out-of-the box features include:
Optimized relevance for search use cases
Typo-tolerance
Relevance tuning
First-party API Clients and robust APIs
Detailed API Logs & Analytics
Automatic Scaling & Operational Support

Elastic App Search Service
Analytics
Understand search behavior across your platform, for any cohort,
and take action using Curations and Relevance Tuning
Relevance Tuning and Curations
Customize the relevance model for any engine using an intuitive
interface. Get full control on the key relevance signals

42
Site Search
The search experience your site deserves

Elastic Site Search Service
Create and manage a tailored search
experience for your website with world-class
relevance, intuitive customization, and rich
analytics
Out-of-the box features include:
Optimized relevance for search use cases
Fully automated and managed web crawler
Algorithmic Customizations (Weights, Pinnings)
UI Libraries for easy website integrations
Advanced Analytics and Behavioral Insights
Automatic Scaling & Operational Support

Elastic Site Search Service
Web-Based Crawler
Index data from any website by simply adding its domain to an
engine and letting the crawler discover content automatically
Weights and Result Rankings
Curate and fine-tune the search output for any engine using a set
of intuitive tool directly from a cloud-based dashboard

Confidentialinformation,forinternaluse
only
Copyright Elasticsearch BV 2015-2018 Copying, publishing and/or
distributing without written permission is strict ly prohibited
The Elastic Journey of an Event
!30
Beats Elasticsearch
Logstash
Kibana
Log
Files
Metrics
Wire
Data
your{beat}
Nodes
Instances
Kafka
Distributed
Message
Queue
NotificationQueues Storage Metrics
Data
Store
Web
APIs
Social Sensors
Nodes
AjitG
adge
-27-M
ay-2018
-Ashnik
Elastic Stack Architecture Overview

Tech
Finance
Telco
Consumer
4
6
Enterprise Customers in Every Industry

APAC Customer Base
• Body Level One
‒ Body Level Two
‒ Body Level Three
‒ Body Level Four
» Body Level Five

only
Infrastructure monitoring :
Who need this ? Most of the domain who are using IT and electronics products need to monitor in real-time.
What data sources needed ? : system logs and application log , metric data , device logs , beats ?
What is business use case / output : ? Infrastructure up time , performance , response , user or system activity
monitoring.
Examples : Monitor IT operation using beats , Monitor Application performance , up time and use behavior ,
Monitor network device or electronic devices and their performance such as ATM monitoring , Device monitoring ,
Network bandwidth monitoring etc., Container and Kubernetes monitoring , Database monitoring. .
Examples : Ebay monitoring their use behavior and infra , OTTO for remote robot monitoring for self driving
company using http/json based protocol, John Deer for farming tools and machine for further agriculture based
analysis , JPL ( NASA ) for location based searching
Hint : Does your enterprises do have real-time monitoring that provide not only monitoring but actionable insight
usingAnomalies detection ? Expensive tools for different devices and objects for different tools , Is it real-time
customize monitoring ? FIS,Allianz.

only
Search
Application Search ( Enterprise Search ), Site Search.
Who need this ? : Most of the Enterprises / SMB / Government / Retails / BFSI /Telecom etc. who have digital
business on internet or intranet .
What Data sources needed ? : For Site Search - Data which is on your web site that need to search and analyze
effectively.
For Enterprise Search : Business data that store in sql/nosql databases or file format like excel , csv, text that
need to effectively search and analyze .
What is Business Use Case / Output. ? : Fast , Relevant search with user behavioral information. Ex : DisplayVs
Monitor , Auto complete suggestion , irrelevant results , faster search ,
Examples : Grab ,Shopify,ebay,just EAT, BBC, Facebook ,TTK Cigna , zendesk , Groupon , Github ,Uber , Kaidee,
AIA
Some Hints : Site search ? Google Appliance ? Enterprise Search on sql queries ? New age startup like online food
delivery , retails, mobile app search etc ?

only
Real Time Business Analytics / Business Intelligent :
Who need this ? Most of the Domains who like to use real time analytics to use for business decision online batch
jobs or weekly/monthly data.
What data sources needed ? Business data that store in sql/nosql databases or file format like excel , csv, text that
need to effectively search and analyze . External data plugin like Hadoop or may be data sources from social media
media , web etc.
What is business use case / output ?: Real-time decision making visualization and dashboards , machine learning
anomalies detection , recommendation engine , complex aggregator visualization such as significant term ,
percentile , derivatives etc.
Examples : Car2Go for real time data processing for car details and price to find customers , Insurance Fraud
Detection , Goldman Search for tracking and analyzing stock trade for guidance to traders and users, Swisslife for
customer data visualization for customers , agent and corporates
HINT : RDBMS like specially open source RDBMS and like to build real-time decision makingng BI ( DW ) to
visualize data and build machine learning pattern ( See EDB customers who like to build DW/ BI as extension ).,
Some customer has Data lake or Hadoop but not sure how to utilize same in real time ,
Use Cases : Insurance Fraud Detection, AIA , ACL

only
Security Analytics :
Who need this ? Most of the domain who care about their internal and external Cyber Security.
What data sources needed ? Metric data, system and network log devices,TCP packet data , SIEM or other
security and n/w devices log data.
What is business use case / output ?: Cyber Security use cases like failed login attempts, threat hunting, DNS
exfiltration , PCI DSS compliance rule breach etc.Very well Augmented with existing SIEM tool like arch sight .
Cyber attack detection by location
Examples : USAA for entire , Credit Suisse, wire card ( payment transaction securing online payment ), Symantec
Slack , NetApp.
HINT : Try to find if customer using any SIEM tool. Can replace or argument SIEM tool. Use ML for threat hunting
and analomolies detection

only
Log analytics
Who need this ? Most of the domain who like to centralize their IT and device logs
What data sources needed ? Machine and devices logs , application logs etc.
What is business use case / output ? Fast and relevant search for log data for any diagnosis , automate issue by
alerting , dashboards and visualization for performance metrics etc., Business alert by finding
Examples : FIS ( OTP sms , ngnix logs for application monitoring for banks ), Citibank, Xoom ( mobile money
transfer ), ITV, Kaidee , Walgreen, Allianz
HINT: Cenreliazing log can give many use cases ,

only
Demo
Dashboards and Visualization.
SIEM
Uptime
Log Search
APM
ML

only
Q & A

only
Thank You

ELK Solutions Enablement Session - 17th March'2020

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie ELK Solutions Enablement Session - 17th March'2020

Ähnlich wie ELK Solutions Enablement Session - 17th March'2020 (20)

Mehr von Ashnikbiz

Mehr von Ashnikbiz (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

ELK Solutions Enablement Session - 17th March'2020