Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Trivadis Azure Data Lake

114 Aufrufe

Veröffentlicht am

This is a presentation by Patrik Borosch about the challenges of todays on premise and in the cloud held at the azure data lake event by trivadis.

Veröffentlicht in: Technologie
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Trivadis Azure Data Lake

  1. 1. The world is changing
  2. 2. Agenda - New challenges and ways - On-premises or cloud (or both?) - The union of all - Making sense of it - How to get there...
  3. 3. Today, 80% of organizations adopt cloud-first strategies AI investment increased by 300% in 2017 Data will grow to 44 ZB in 2020
  4. 4. Today, 80% of organizations adopt cloud-first strategies AI investment increased by 300% in 2017 Data will grow to 44 ZB in 2020 C LO U D A IDATA
  5. 5. C LO U D DATA A I Organizations that harness data, cloud, and AI outperform
  6. 6. Rely on a modern data estate
  7. 7. Patrik Borosch TSP / DP • 02.07.1971/married/Daughter(19) • Music(Squared Circle/The Midcrise Liars)/climbing/skiing/cycling/Tango • EDV-Kaufmann (german IHK 1995) = Data Processing with Cobol and Databases = the early BI guys ;) • Reporting and data processing in controlling departments • BI Consultant/Senior Consultant: ASTECH Solutions/T-Systems/Trivadis/Avanade •  Discipline Manager Microsoft BI and Power Pivot Trainer @ Trivadis • Head of BI: Allianz Global Assistance • TSP DP: Microsoft • SQL Server/SSIS/SSAS/SSRS/MDM • PowerBI/PowerPivot/PowerQuery • Azure SQL DB/DW/Data Lake/Azure Stream Analytics/Data Factory/AAS • SQL/DAX/MDX/(PowerShell)/(C#) • Informatica/Microstrategy/Essbase/Enterprise Architect/UML/Perl/Unix/Linux  had that... been there... • First «Big Data»-Project in 2006: Teradata/Informatica/Microstrategy, 1.4TB = eight weeks for init load = lots of fun :D
  8. 8. 8
  9. 9. 9
  10. 10. Dr. John Snow (1854) One of the first visual investigations of collected data helped to solve a cholera epidemic in Soho… - You need the facts - BUT you also need to make sense of it... Therefore you need to have the right tools and methods...
  11. 11. The many sources and rapid growth of data requires a new approach • Sentiment Analysis • Social Media / Sales Connection • Customer Segmentation
  12. 12. Data lake From Wikipedia, the free encyclopedia A data lake is a method of storing data within a system or repository, in its natural format,[1] that facilitates the collocation of data in various schemata and structural forms, usually object blobs or files. The idea of data lake is to have a single store of all data in the enterprise ranging from raw data (which implies exact copy of source system data) to transformed data which is used for various tasks including reporting, visualization, analytics and machine learning. The data lake includes structured data from relational databases (rows and columns), semi-structured data (CSV, logs, XML, JSON), unstructured data (emails, documents, PDFs) and even binary data (images, audio, video) thus creating a centralized data store accommodating all forms of data. James Dixon / Pentaho (2010) Data Lakes
  13. 13. BUSINESS APPS CUSTOM APPS ANALYTICAL DASHBOARDS AZURE SQL DATA WAREHOUSE AZURE CLI AZURE DATA FACTORY BCP COMMAND LINE UTILITY SQL SERVER INTEGRATION SERVICES AZURE ANALYSIS SERVICES
  14. 14. BUSINESS APPS CUSTOM APPS ANALYTICAL DASHBOARDS DATA FACTORY ANALYTICAL DASHBOARDS Polybase AZURE SQL DATA WAREHOUSE DATA FACTORY AZURE ANALYSIS SERVICES AZURE MACHINE LEARNING & MACHINE LEARNING SERVER AZURE COSMOS DB AZURE STORAGE
  15. 15. BUSINESS APPS CUSTOM APPS ANALYTICAL DASHBOARDS DATA FACTORY ANALYTICAL DASHBOARDS Polybase AZURE SQL DATA WAREHOUSE DATA FACTORY AZURE ANALYSIS SERVICES AZURE MACHINE LEARNING & MACHINE LEARNING SERVER AZURE COSMOS DB AZURE HDINSIGHT (Hadoop)AZURE STORAGE
  16. 16. BUSINESS APPS CUSTOM APPS ANALYTICAL DASHBOARDS DATA FACTORY ANALYTICAL DASHBOARDS Polybase AZURE SQL DATA WAREHOUSE DATA FACTORY AZURE ANALYSIS SERVICES AZURE MACHINE LEARNING & MACHINE LEARNING SERVER AZURE COSMOS DB AZURE STORAGE AZURE DATABRICKS (SPARK)
  17. 17. BUSINESS APPS CUSTOM APPS ANALYTICAL DASHBOARDS DATA FACTORY AZURE DATA LAKE STORE AZURE DATA LAKE ANALYTICS ANALYTICAL DASHBOARDS Polybase AZURE SQL DATA WAREHOUSE DATA FACTORY AZURE ANALYSIS SERVICES AZURE MACHINE LEARNING & MACHINE LEARNING SERVER AZURE COSMOS DB
  18. 18. CONTROL EASE OF USE Azure Data Lake Analytics Azure Data Lake Store Azure Storage Any Hadoop technology, any distribution Workload optimized, managed clusters Data Engineering in a Job-as-a-service model Azure Marketplace HDP | CDH | MapR Azure Data Lake Analytics IaaS Clusters Managed Clusters Big Data as-a-service Azure HDInsight Frictionless & Optimized Spark clusters Azure Databricks BIGDATA STORAGE BIGDATA ANALYTICS ReducedAdministration K N O W I N G T H E V A R I O U S B I G D A T A S O L U T I O N S Drag & Drop Azure ML
  19. 19. Big Data is driving transformative changes Cost Culture Data Characteristics Traditional Big Data Relational (with highly modeled schema) All Data (with schema agility) Expensive (storage and compute capacity) Cloud (storage and compute capacity) Rear-view reporting (using relational algebra) Intelligent action (using relational algebra AND ML, graph, streaming, image processing)
  20. 20. Cognitive Services • Faces, images, emotion recognition and video intelligence • Spoken language processing, speaker recognition, custom speech recognition • Natural language processing, sentiment and topics analysis, spelling errors • Complex tasks processing, knowledge exploration, intelligent recommendations • Bing engine capabilities for Web, Autosuggest, Image, Video and News Intelligence Cortana Bot Framework Cognitive Services
  21. 21. Microsoft BI, the agile way… Azure Analysis Services
  22. 22. Data Sources Ingest Prepare Analyze Publish Consume Sensors and devices Stream Analytics Diagnostic Streaming Power BI Sources - Oralce HFS - SAP BW - … Azure Data Lake Store Data Factory: Move data, orchestrate, schedule and monitor Azure Data LakeIoT Hubs Machine Learning HDInsight Data Science Workbench Stream Analytics Power BI Report Server Architecture Blueprint SSIS SQL Server 2017: Security, Performance, Polybase, ML Services, Analytics SQL Server 2017 SSAS BI Bot Apps Lab- and other Apps AzureDataPlatformSQLserver2017
  23. 23. AI built-in | Most secure | Lowest TCO M I C R O S O F T F O R Y O U R M O D E R N D A T A E S T A T E Data warehouses Data lakes Operational databases Data warehouses Data lakes Operational databases SQL Server Azure Data Services Industry leader 4 years in a row #1 TPC-H performance T-SQL query over any data 70% faster 2x the global reach 99.9% SLA HYBRID Easiest lift and shift with no code changes SocialLOB Graph IoTImageCRM Security and performance Flexibility of choice Reason over any data, anywhere
  24. 24. Tools for your migration journey SQL Server Migration Assistant (SSMA) Automates database migration to SQL Server from Microsoft Access, DB2, MySQL, Oracle, and SAP ASE. Data Migration Assistant (DMA) Enables upgrade to SQL Server and Azure SQL Database. Database Experimentation Assistant (DEA) Assists in evaluating a targeted version of SQL for a given workload. Azure Hybrid Benefit for SQL Server Maximizes current on-premises license investments to facilitate migration. Azure SQL Database Managed Instance Facilitates lift and shift migration from on-premises SQL Server to PaaS. Azure Database Migration Service (Azure DMS) SQL Server Migration Assistant (SSMA) Data Migration Assistant (DMA) Database Experimentation Assistant (DEA) SQL Database Managed Instance Azure Hybrid Benefit for SQL Server
  25. 25. Empower today’s innovators to unleash the power of data and reimagine possibilities that will improve our world

×