SlideShare ist ein Scribd-Unternehmen logo
1 von 24
How much data?
 800 Terabytes, 2001
 60 Exabytes, 2006
 500 Exabytes, 2009
 2.7 Zettabytes, 2012
 35 Zettabytes by 2020
How much data is
generated in one day?
 7 TB, Twitter
 15 TB, Facebook(2009)
 20 PB, Google(2008)
 6.5 PB Ebay(2009)
A visualization
created by IBM of
Wikipedia edits. At
multiple terabytes in
size, the text and
images of Wikipedia
are a classic
example of big data.
 Relational Data
(Tables/Transaction/Legacy Data)
 Text Data (Web)
 Semi-structured Data (XML)
 Graph Data
› Social Network, Semantic Web (RDF)
 Streaming Data
› You can only scan the data once
 Aggregation and Statistics
› Data warehouse and OLAP
 Indexing, Searching, and Querying
› Keyword based search
› Pattern matching (XML/RDF)
 Knowledge discovery
› Data Mining
› Statistical Modeling
 There is no consensus as how to define
Big Data.
“Big data exceeds the reach of commonly used hardware
environments and software tools to capture, manage, and
process it with in a tolerable elapsed time for its user population.”
- Teradata Magazine article, 2011
“Big data refers to data sets whose size is beyond the ability of
typical database software tools to capture, store, manage and
analyze.” - The McKinsey Global Institute, 2011
Variety
Structured and
unstructured
data: clinical
notes, audio
transcription, ima
ging, click streams
Velocity
Often time-
sensitive, data
must be
analyzed as it’s
streaming in to
maximize its
value(e.g.
patient
monitoring)
Volume
Electronic medical
records, images, digital
pathology, email, web communications
1) Automatically generated by a machine
(e.g. Sensor embedded in an engine)
2) Typically an entirely new source of data
(e.g. Use of the internet)
3) Not designed to be friendly
(e.g. Text streams)
4) May not have much values
› Need to focus on the important part
•Most new data sources were considered big and difficult
•Just the next wave of new, bigger data
The Past The Present The Future
(1) The “big” part
(2) The “data” part
(3) Both
(4) Neither
The answer is choice (4)
What is important is what the organizations do
with Big Data.
 Decoding the human genome originally took 10
years to process, now it can be achieved in less than
a week!!!
 Tobias Preis used Google Trends data to
demonstrate that Internet users from countries with a
higher per capita GDP are more likely to search for
information about the future than information about
the past. The findings suggest there may be a link
between online behaviour and real-world economic
indicators!!!
 Tobias Preis and H. Eugene Stanley’s analysis
of Google search volume for 98 terms of varying
financial relevance, published in Scientific
Reports, suggests that increases in search volume for
financially relevant search terms tend to precede
large losses in financial markets!!!
1. Big data can unlock significant value by making
information transparent and usable at much higher
frequency.
2. As organizations create and store more
transactional data in digital form, they can collect
more accurate and detailed performance
information on and therefore expose variability and
boost performance.
3. Big data allows ever-narrower segmentation of
customers and therefore much more precisely
tailored products or services.
4. Sophisticated analytics can substantially improve
decision-making.
5. Big data can be used to improve the development
of the next generation of products and services.



1. The use of big data will become a key basis of
competition and growth for individual firms. All
companies need to take big data seriously.
2. The use of big data will underpin new waves of
productivity growth(growth by 60% possible) and
consumer surplus.
3. The computer and electronic products and
information sectors, as well as finance and
insurance, and government are poised to gain
substantially from the use of big data.
4. By 2018, the United States alone could face a
shortage of 140,000 to 190,000 people with deep
analytical skills as well as 1.5 million managers
and analysts with the know-how to use the
analysis of big data to make effective decisions.
 Will be so overwhelmed
› Need the right people and solve the right
problems
 Costs escalate too fast
› Isn’t necessary to capture 100%
 Sources of big data may be
private
› Self-regulation
› Legal regulation
 Very strong assumptions are made about
mathematical properties that may not at all reflect
what is really going on at the level of micro-
processes.
 Even as companies invest eight- and nine-figure sums
to derive insight from information streaming in from
suppliers and customers, less than 40% of employees
have sufficiently mature processes and skills to do so.
 The decisions based on the analysis of Big Data are
inevitably "informed by the world as it was in the
past, or, at best, as it currently is“.
 If the systems dynamics of the future change, the
past can say little about the future. For this, it would
be necessary to have a thorough understanding of
the systems dynamic, which implies theory.
 The biggest value in big data can be driven by
combing big data with other corporate data
Big
data
Other
data
Create a
synergy
effect
 Banking industries were very hard to
handle even a decade ago
 “BIG” will change
› Big data will continue to evolve
 en.wikipedia.org/wiki/Big_data
 www.google.com
 www.mckinsey.com
Big Data

Weitere ähnliche Inhalte

Was ist angesagt?

Big Data for the Rest of Us - OpenWest 2014 - Matt Asay
Big Data for the Rest of Us - OpenWest 2014 - Matt AsayBig Data for the Rest of Us - OpenWest 2014 - Matt Asay
Big Data for the Rest of Us - OpenWest 2014 - Matt AsayMatt Asay
 
Quick view Big Data, brought by Oomph!, courtesy of our partner Sonovate
Quick view Big Data, brought by Oomph!, courtesy of our partner Sonovate Quick view Big Data, brought by Oomph!, courtesy of our partner Sonovate
Quick view Big Data, brought by Oomph!, courtesy of our partner Sonovate Oomph! Recruitment
 
What is big data ? | Big Data Applications
What is big data ? | Big Data ApplicationsWhat is big data ? | Big Data Applications
What is big data ? | Big Data ApplicationsShilpaKrishna6
 
The Emergence of Alt-Data and its Applications
The Emergence of Alt-Data and its ApplicationsThe Emergence of Alt-Data and its Applications
The Emergence of Alt-Data and its ApplicationsPromptCloud
 
Big Data & Analytics for Government - Case Studies
Big Data & Analytics for Government - Case StudiesBig Data & Analytics for Government - Case Studies
Big Data & Analytics for Government - Case StudiesJohn Palfreyman
 
Big Data and The Future of Insight - Future Foundation
Big Data and The Future of Insight - Future FoundationBig Data and The Future of Insight - Future Foundation
Big Data and The Future of Insight - Future FoundationForesight Factory
 
Data Science Courses - BigData VS Data Science
Data Science Courses - BigData VS Data ScienceData Science Courses - BigData VS Data Science
Data Science Courses - BigData VS Data ScienceDataMites
 
Big data and Internet
Big data and InternetBig data and Internet
Big data and InternetSanoj Kumar
 
BIG DATA & DATA ANALYTICS
BIG  DATA & DATA  ANALYTICSBIG  DATA & DATA  ANALYTICS
BIG DATA & DATA ANALYTICSNAGARAJAGIDDE
 
Big Data Presentation at SCQAA-SF on June 12 2013
Big Data Presentation at SCQAA-SF on June 12 2013Big Data Presentation at SCQAA-SF on June 12 2013
Big Data Presentation at SCQAA-SF on June 12 2013Sujit Ghosh
 
Big data for official statistics @ Konferensi Big Data Indonesia 2016
Big data for official statistics @ Konferensi Big Data Indonesia 2016 Big data for official statistics @ Konferensi Big Data Indonesia 2016
Big data for official statistics @ Konferensi Big Data Indonesia 2016 Setia Pramana
 
Interesting ways Big Data is used today
Interesting ways Big Data is used todayInteresting ways Big Data is used today
Interesting ways Big Data is used todayDaniel Sârbe
 
Big Data Applications | Big Data Application Examples | Big Data Use Cases | ...
Big Data Applications | Big Data Application Examples | Big Data Use Cases | ...Big Data Applications | Big Data Application Examples | Big Data Use Cases | ...
Big Data Applications | Big Data Application Examples | Big Data Use Cases | ...Simplilearn
 

Was ist angesagt? (20)

Big Data-Job 2
Big Data-Job 2Big Data-Job 2
Big Data-Job 2
 
Big Data for the Rest of Us - OpenWest 2014 - Matt Asay
Big Data for the Rest of Us - OpenWest 2014 - Matt AsayBig Data for the Rest of Us - OpenWest 2014 - Matt Asay
Big Data for the Rest of Us - OpenWest 2014 - Matt Asay
 
Quick view Big Data, brought by Oomph!, courtesy of our partner Sonovate
Quick view Big Data, brought by Oomph!, courtesy of our partner Sonovate Quick view Big Data, brought by Oomph!, courtesy of our partner Sonovate
Quick view Big Data, brought by Oomph!, courtesy of our partner Sonovate
 
What is big data ? | Big Data Applications
What is big data ? | Big Data ApplicationsWhat is big data ? | Big Data Applications
What is big data ? | Big Data Applications
 
Data science
Data scienceData science
Data science
 
Big Data
Big DataBig Data
Big Data
 
The Emergence of Alt-Data and its Applications
The Emergence of Alt-Data and its ApplicationsThe Emergence of Alt-Data and its Applications
The Emergence of Alt-Data and its Applications
 
Big Data Trends
Big Data TrendsBig Data Trends
Big Data Trends
 
Using Data Riches A tale of two projects - Ajay Vinze
Using Data Riches A tale of two projects - Ajay VinzeUsing Data Riches A tale of two projects - Ajay Vinze
Using Data Riches A tale of two projects - Ajay Vinze
 
Big Data & Analytics for Government - Case Studies
Big Data & Analytics for Government - Case StudiesBig Data & Analytics for Government - Case Studies
Big Data & Analytics for Government - Case Studies
 
Big Data Trends
Big Data TrendsBig Data Trends
Big Data Trends
 
Big Data and The Future of Insight - Future Foundation
Big Data and The Future of Insight - Future FoundationBig Data and The Future of Insight - Future Foundation
Big Data and The Future of Insight - Future Foundation
 
Data Science Courses - BigData VS Data Science
Data Science Courses - BigData VS Data ScienceData Science Courses - BigData VS Data Science
Data Science Courses - BigData VS Data Science
 
Big data and Internet
Big data and InternetBig data and Internet
Big data and Internet
 
BIG DATA & DATA ANALYTICS
BIG  DATA & DATA  ANALYTICSBIG  DATA & DATA  ANALYTICS
BIG DATA & DATA ANALYTICS
 
Big data
Big dataBig data
Big data
 
Big Data Presentation at SCQAA-SF on June 12 2013
Big Data Presentation at SCQAA-SF on June 12 2013Big Data Presentation at SCQAA-SF on June 12 2013
Big Data Presentation at SCQAA-SF on June 12 2013
 
Big data for official statistics @ Konferensi Big Data Indonesia 2016
Big data for official statistics @ Konferensi Big Data Indonesia 2016 Big data for official statistics @ Konferensi Big Data Indonesia 2016
Big data for official statistics @ Konferensi Big Data Indonesia 2016
 
Interesting ways Big Data is used today
Interesting ways Big Data is used todayInteresting ways Big Data is used today
Interesting ways Big Data is used today
 
Big Data Applications | Big Data Application Examples | Big Data Use Cases | ...
Big Data Applications | Big Data Application Examples | Big Data Use Cases | ...Big Data Applications | Big Data Application Examples | Big Data Use Cases | ...
Big Data Applications | Big Data Application Examples | Big Data Use Cases | ...
 

Andere mochten auch

Storage Class Specifiers
Storage Class SpecifiersStorage Class Specifiers
Storage Class SpecifiersReddhi Basu
 
Program and System Threats
Program and System ThreatsProgram and System Threats
Program and System ThreatsReddhi Basu
 
Storage Classes and Functions
Storage Classes and FunctionsStorage Classes and Functions
Storage Classes and FunctionsJake Bond
 
Storage Class Specifiers in C++
Storage Class Specifiers in C++Storage Class Specifiers in C++
Storage Class Specifiers in C++Reddhi Basu
 
Software Engineering - Software Models
Software Engineering - Software ModelsSoftware Engineering - Software Models
Software Engineering - Software ModelsReddhi Basu
 
Exception Handling
Exception HandlingException Handling
Exception HandlingReddhi Basu
 

Andere mochten auch (7)

Storage Class Specifiers
Storage Class SpecifiersStorage Class Specifiers
Storage Class Specifiers
 
Program and System Threats
Program and System ThreatsProgram and System Threats
Program and System Threats
 
Storage Classes and Functions
Storage Classes and FunctionsStorage Classes and Functions
Storage Classes and Functions
 
Storage Class Specifiers in C++
Storage Class Specifiers in C++Storage Class Specifiers in C++
Storage Class Specifiers in C++
 
Storage class in C Language
Storage class in C LanguageStorage class in C Language
Storage class in C Language
 
Software Engineering - Software Models
Software Engineering - Software ModelsSoftware Engineering - Software Models
Software Engineering - Software Models
 
Exception Handling
Exception HandlingException Handling
Exception Handling
 

Ähnlich wie Big Data

Move It Don't Lose It: Is Your Big Data Collecting Dust?
Move It Don't Lose It: Is Your Big Data Collecting Dust?Move It Don't Lose It: Is Your Big Data Collecting Dust?
Move It Don't Lose It: Is Your Big Data Collecting Dust?Jennifer Walker
 
Data mining with big data implementation
Data mining with big data implementationData mining with big data implementation
Data mining with big data implementationSandip Tipayle Patil
 
Notes from the Observation Deck // A Data Revolution
Notes from the Observation Deck // A Data Revolution Notes from the Observation Deck // A Data Revolution
Notes from the Observation Deck // A Data Revolution gngeorge
 
QuickView #3 - Big Data
QuickView #3 - Big DataQuickView #3 - Big Data
QuickView #3 - Big DataSonovate
 
Big data PPT prepared by Hritika Raj (Shivalik college of engg.)
Big data PPT prepared by Hritika Raj (Shivalik college of engg.)Big data PPT prepared by Hritika Raj (Shivalik college of engg.)
Big data PPT prepared by Hritika Raj (Shivalik college of engg.)Hritika Raj
 
The Future of Big Data
The Future of Big Data The Future of Big Data
The Future of Big Data EMC
 
What You Need to Know About Big Data
What You Need to Know About Big DataWhat You Need to Know About Big Data
What You Need to Know About Big DataTier10
 
Lecture 01-1-IIS.pptx
Lecture 01-1-IIS.pptxLecture 01-1-IIS.pptx
Lecture 01-1-IIS.pptxAsadkhan47384
 
Big Data - Big Deal? - Edison's Academic Paper in SMU
Big Data - Big Deal? - Edison's Academic Paper in SMUBig Data - Big Deal? - Edison's Academic Paper in SMU
Big Data - Big Deal? - Edison's Academic Paper in SMUEdison Lim Jun Hao
 
Implementation of application for huge data file transfer
Implementation of application for huge data file transferImplementation of application for huge data file transfer
Implementation of application for huge data file transferijwmn
 
Introduction to big data – convergences.
Introduction to big data – convergences.Introduction to big data – convergences.
Introduction to big data – convergences.saranya270513
 
Big data lecture notes
Big data lecture notesBig data lecture notes
Big data lecture notesMohit Saini
 
Bigdata the technological renaissance
Bigdata the technological renaissanceBigdata the technological renaissance
Bigdata the technological renaissanceRituBhargava7
 

Ähnlich wie Big Data (20)

Move It Don't Lose It: Is Your Big Data Collecting Dust?
Move It Don't Lose It: Is Your Big Data Collecting Dust?Move It Don't Lose It: Is Your Big Data Collecting Dust?
Move It Don't Lose It: Is Your Big Data Collecting Dust?
 
Data mining with big data implementation
Data mining with big data implementationData mining with big data implementation
Data mining with big data implementation
 
Notes from the Observation Deck // A Data Revolution
Notes from the Observation Deck // A Data Revolution Notes from the Observation Deck // A Data Revolution
Notes from the Observation Deck // A Data Revolution
 
QuickView #3 - Big Data
QuickView #3 - Big DataQuickView #3 - Big Data
QuickView #3 - Big Data
 
Big data PPT prepared by Hritika Raj (Shivalik college of engg.)
Big data PPT prepared by Hritika Raj (Shivalik college of engg.)Big data PPT prepared by Hritika Raj (Shivalik college of engg.)
Big data PPT prepared by Hritika Raj (Shivalik college of engg.)
 
The Future of Big Data
The Future of Big Data The Future of Big Data
The Future of Big Data
 
Bigdata Hadoop introduction
Bigdata Hadoop introductionBigdata Hadoop introduction
Bigdata Hadoop introduction
 
What You Need to Know About Big Data
What You Need to Know About Big DataWhat You Need to Know About Big Data
What You Need to Know About Big Data
 
Big Data World
Big Data WorldBig Data World
Big Data World
 
130214 copy
130214   copy130214   copy
130214 copy
 
Lecture 01-1-IIS.pptx
Lecture 01-1-IIS.pptxLecture 01-1-IIS.pptx
Lecture 01-1-IIS.pptx
 
Big Data - Big Deal? - Edison's Academic Paper in SMU
Big Data - Big Deal? - Edison's Academic Paper in SMUBig Data - Big Deal? - Edison's Academic Paper in SMU
Big Data - Big Deal? - Edison's Academic Paper in SMU
 
Data mining with big data
Data mining with big dataData mining with big data
Data mining with big data
 
Implementation of application for huge data file transfer
Implementation of application for huge data file transferImplementation of application for huge data file transfer
Implementation of application for huge data file transfer
 
Introduction to big data – convergences.
Introduction to big data – convergences.Introduction to big data – convergences.
Introduction to big data – convergences.
 
Big data lecture notes
Big data lecture notesBig data lecture notes
Big data lecture notes
 
Bigdata the technological renaissance
Bigdata the technological renaissanceBigdata the technological renaissance
Bigdata the technological renaissance
 
Monetize Big Data
Monetize Big DataMonetize Big Data
Monetize Big Data
 
Bigdata " new level"
Bigdata " new level"Bigdata " new level"
Bigdata " new level"
 
Big Data Challenges faced by Organizations
Big Data Challenges faced by OrganizationsBig Data Challenges faced by Organizations
Big Data Challenges faced by Organizations
 

Kürzlich hochgeladen

"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr BaganFwdays
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 3652toLead Limited
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsMark Billinghurst
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyAlfredo García Lavilla
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piececharlottematthew16
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsRizwan Syed
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubKalema Edgar
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionDilum Bandara
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxhariprasad279825
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024Stephanie Beckett
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupFlorian Wilhelm
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DaySri Ambati
 

Kürzlich hochgeladen (20)

"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan"ML in Production",Oleksandr Bagan
"ML in Production",Oleksandr Bagan
 
Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365Ensuring Technical Readiness For Copilot in Microsoft 365
Ensuring Technical Readiness For Copilot in Microsoft 365
 
Human Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR SystemsHuman Factors of XR: Using Human Factors to Design XR Systems
Human Factors of XR: Using Human Factors to Design XR Systems
 
Commit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easyCommit 2024 - Secret Management made easy
Commit 2024 - Secret Management made easy
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
Story boards and shot lists for my a level piece
Story boards and shot lists for my a level pieceStory boards and shot lists for my a level piece
Story boards and shot lists for my a level piece
 
Scanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL CertsScanning the Internet for External Cloud Exposures via SSL Certs
Scanning the Internet for External Cloud Exposures via SSL Certs
 
Unleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding ClubUnleash Your Potential - Namagunga Girls Coding Club
Unleash Your Potential - Namagunga Girls Coding Club
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Advanced Computer Architecture – An Introduction
Advanced Computer Architecture – An IntroductionAdvanced Computer Architecture – An Introduction
Advanced Computer Architecture – An Introduction
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 
TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Artificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptxArtificial intelligence in cctv survelliance.pptx
Artificial intelligence in cctv survelliance.pptx
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024What's New in Teams Calling, Meetings and Devices March 2024
What's New in Teams Calling, Meetings and Devices March 2024
 
Streamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project SetupStreamlining Python Development: A Guide to a Modern Project Setup
Streamlining Python Development: A Guide to a Modern Project Setup
 
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo DayH2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
H2O.ai CEO/Founder: Sri Ambati Keynote at Wells Fargo Day
 

Big Data

  • 1.
  • 2. How much data?  800 Terabytes, 2001  60 Exabytes, 2006  500 Exabytes, 2009  2.7 Zettabytes, 2012  35 Zettabytes by 2020 How much data is generated in one day?  7 TB, Twitter  15 TB, Facebook(2009)  20 PB, Google(2008)  6.5 PB Ebay(2009)
  • 3. A visualization created by IBM of Wikipedia edits. At multiple terabytes in size, the text and images of Wikipedia are a classic example of big data.
  • 4.  Relational Data (Tables/Transaction/Legacy Data)  Text Data (Web)  Semi-structured Data (XML)  Graph Data › Social Network, Semantic Web (RDF)  Streaming Data › You can only scan the data once
  • 5.  Aggregation and Statistics › Data warehouse and OLAP  Indexing, Searching, and Querying › Keyword based search › Pattern matching (XML/RDF)  Knowledge discovery › Data Mining › Statistical Modeling
  • 6.  There is no consensus as how to define Big Data. “Big data exceeds the reach of commonly used hardware environments and software tools to capture, manage, and process it with in a tolerable elapsed time for its user population.” - Teradata Magazine article, 2011 “Big data refers to data sets whose size is beyond the ability of typical database software tools to capture, store, manage and analyze.” - The McKinsey Global Institute, 2011
  • 7.
  • 8. Variety Structured and unstructured data: clinical notes, audio transcription, ima ging, click streams Velocity Often time- sensitive, data must be analyzed as it’s streaming in to maximize its value(e.g. patient monitoring) Volume Electronic medical records, images, digital pathology, email, web communications
  • 9. 1) Automatically generated by a machine (e.g. Sensor embedded in an engine) 2) Typically an entirely new source of data (e.g. Use of the internet) 3) Not designed to be friendly (e.g. Text streams) 4) May not have much values › Need to focus on the important part
  • 10. •Most new data sources were considered big and difficult •Just the next wave of new, bigger data The Past The Present The Future
  • 11. (1) The “big” part (2) The “data” part (3) Both (4) Neither The answer is choice (4) What is important is what the organizations do with Big Data.
  • 12.  Decoding the human genome originally took 10 years to process, now it can be achieved in less than a week!!!  Tobias Preis used Google Trends data to demonstrate that Internet users from countries with a higher per capita GDP are more likely to search for information about the future than information about the past. The findings suggest there may be a link between online behaviour and real-world economic indicators!!!  Tobias Preis and H. Eugene Stanley’s analysis of Google search volume for 98 terms of varying financial relevance, published in Scientific Reports, suggests that increases in search volume for financially relevant search terms tend to precede large losses in financial markets!!!
  • 13. 1. Big data can unlock significant value by making information transparent and usable at much higher frequency. 2. As organizations create and store more transactional data in digital form, they can collect more accurate and detailed performance information on and therefore expose variability and boost performance. 3. Big data allows ever-narrower segmentation of customers and therefore much more precisely tailored products or services. 4. Sophisticated analytics can substantially improve decision-making. 5. Big data can be used to improve the development of the next generation of products and services.
  • 14.
  • 15.
  • 17. 1. The use of big data will become a key basis of competition and growth for individual firms. All companies need to take big data seriously. 2. The use of big data will underpin new waves of productivity growth(growth by 60% possible) and consumer surplus. 3. The computer and electronic products and information sectors, as well as finance and insurance, and government are poised to gain substantially from the use of big data. 4. By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions.
  • 18.  Will be so overwhelmed › Need the right people and solve the right problems  Costs escalate too fast › Isn’t necessary to capture 100%  Sources of big data may be private › Self-regulation › Legal regulation
  • 19.  Very strong assumptions are made about mathematical properties that may not at all reflect what is really going on at the level of micro- processes.  Even as companies invest eight- and nine-figure sums to derive insight from information streaming in from suppliers and customers, less than 40% of employees have sufficiently mature processes and skills to do so.  The decisions based on the analysis of Big Data are inevitably "informed by the world as it was in the past, or, at best, as it currently is“.  If the systems dynamics of the future change, the past can say little about the future. For this, it would be necessary to have a thorough understanding of the systems dynamic, which implies theory.
  • 20.  The biggest value in big data can be driven by combing big data with other corporate data Big data Other data Create a synergy effect
  • 21.
  • 22.  Banking industries were very hard to handle even a decade ago  “BIG” will change › Big data will continue to evolve