- Business goal
- What is Fast Data for us
- What is Fast & Big Data solution
- Reference Architecture
- Data Science for Big Data
- Technology Stack
- Solution Architecture
- Identity & Telemetry Data Processing Facts
- Continuous Deployment
- Quality Control
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Дмитрий Лавриненко "Big & Fast Data for Identity & Telemetry services"
1. Big & Fast Data
for Identity & Telemetry services
2. Business Goal
Deliver a vendor-agnostic and open-source ready Big Data as a Service platform
by using most up to date automation technologies and utilizing partnership
with major Big Data software vendors and independent developers.
3. What is Fast Data for us?
• Continuous data loading
• Massively parallel processing
• Data consolidation
• Dimensional processing
• Data normalization & denormalization (depends on tech stack)
• Structured & dimensional data models
• Hybrid distributed warehouses
4. What is Fast & Big Data Solution
• Data WH
• Processing
• Analytics
• Visualisation
• Machine Learning
• Data Virtualization
• Data Ingestion
As a Service
6. Reference Architecture
All Data
Real-time Data Processing
Data Acquisition and Storing
DataIntegration
Data Warehousing
Data Management
(Governance, Security, Quality, MDM)
Analytics
Reporting and
Analysis
Predictive
Modeling
Data Mining
Data Lake
(Landing, Exploration
and Archiving)
UX and
Visualization
Applications
Application data
Media data
Social data
Enterprise
content data
Telemetry
Other data
Customer
Analytics
Marketing
Analytics
Web/Mobile/Social
Analytics
IT Operational
Analytics
Fraud and Risk
Analytics
Complex Event
Processing
Real-time Query and
Search
7. Data Science for Big Data
Artificial
Intelligence
Machine Learning
High-Dimensional Data
Big Data
Apache Hadoop
Infrastructure
Data Collection
Data Augmentation
Real-time
Data Processing
Predictive
Analytics
Risk Analysis
Direct Marketing
Decision Support
Systems
Learning and
Intelligent Optimization
Data
Analysis
Data Exploration
Data Visualization
Business Intelligence