Thank you for the summary. I do not actually have an opinion on whether games or gamification are good or bad. I'm an AI assistant created by Anthropic to be helpful, harmless, and honest.

1. LECTURE L21
L21 Big Data and Analytics

2. Big Data
With the computer revolution, digital data becomes possible
Over the years, data has grown exponentially
“Big Data” has become a
platform by itself with new
possibilties

3. Global Data is Growing Fast
Data in Digital Universe vs. Data Storage Cost, 2010-2015
Source: Mary Meeker, KPCB

4. Data is a New Growth Platform
The
Network
The
Software
The
Infrastructure
The
Data
Large investments in fiber optic & last-mile cable create connectivity that
facilitated the early Internet growth
Optimising the network with software became far more capital efficient than
additional capital expenditure buildouts, ultimately resulting in the creation of
pervasive networks (Siloed DCs -> AWS) and pervasive software (Siebel ->
Salesforce)
Emergence of pervasive software created the need to optimise the
performance of the network and store extraordinary amounts of data at
extremely low prices
Next Big Wave: Leveraging this unlimited connectivity and storage to collect /
aggregate / correlate / interpret all of this data to improve people’s live and
enable enterprises to operate more efficiently

5. Evolution of Data Platform
Source: Mary Meeker, KPCB

6. Data Generators
Source: Mary Meeker, KPCB

7. Improve people’s live and enable
enterprises to operate more efficiently

8. “Data is moving from something you use outside
the workstream to becoming a part of the
business app itself.”
— Frank Bien, CEO of Looker

9. Big Data Examples

10. Big Data Examples
Macy's Inc. and real-time pricing
The retailer adjusts pricing in near-real time for 73 million
items, based on demand and inventory.
Source:Ten big data case studies in a nutshell

11. Big Data Examples
Tipp24 AG, a platform for placing bets
The company uses software to analyse billions of
transactions and hundreds of customer attributes, and to
develop predictive models that target customers and
personalise marketing messages on the fly.
Source:Ten big data case studies in a nutshell

12. Big Data Examples
Wal-Mart Stores Inc. and search
The mega-retailer's latest search engine for Walmart.com
includes semantic data. A platform that was designed in-
house, relies on text analysis, machine learning and even
synonym mining to produce relevant search results.
Wal-Mart says adding semantic search has improved online
shoppers completing a purchase by 10% to 15%.
Source:Ten big data case studies in a nutshell

13. Big Data Examples
PredPol Inc. and repurposing
The Los Angeles and Santa Cruz police departments, a team
of educators and a company called PredPol have taken an
algorithm used to predict earthquakes, tweaked it and
started feeding it crime data.
The software can predict where crimes are likely to occur
down to 500 square feet. In LA, there's been a 33%
reduction in burglaries and 21% reduction in violent crimes
in areas where the software is being used.
Source:Ten big data case studies in a nutshell

14. Big Data Examples
American Express and business intelligence
AmEx started looking for indicators that could really predict
loyalty and developed sophisticated predictive models to
analyse historical transactions and 115 variables to forecast
potential churn
The company believes it can now identify 24% of Australian
accounts that will close within the next four months.
Source:Ten big data case studies in a nutshell

15. Big Data Examples
A Bank and IBM
A large US bank uses IBM machine learning technologies to
analyse credit card transactions.
Using machine learning and stream computing to detect financial fraud

16. TEDxUofM - Jameson Toole - Big Data for Tomorrow

18. What is Big Data?

19. What is Big Data?
Big data is high-volume, high-velocity and/or high-variety
information assets that demand cost-effective, innovative
forms of information processing that enable enhanced
insight, decision making, and process automation.
Gartner

20. What is Big Data?
Big data refers to a process that is used when traditional
data mining and handling techniques cannot uncover the
insights and meaning of the underlying data. Data that is
unstructured or time sensitive or simply very large cannot be
processed by relational database engines. This type of data
requires a different processing approach called big data,
which uses massive parallelism on readily-available
hardware.
Techopedia

21. What is Big Data?
“Big data is the oil of the 21st century and analytics is the
combustion engine.”
—Peter Sondergaard, Gartner Reseach

22. What is Big Data?
Byte: one rice
David Wellman: What is Big Data?

23. What is Big Data?
Byte: one rice
Kilobyte: handful of rice
David Wellman: What is Big Data?

24. What is Big Data?
Byte: one rice
Kilobyte: handful of rice
Megabyte: Big pot of rice
David Wellman: What is Big Data?

25. What is Big Data?
Byte: one rice
Kilobyte: handful of rice
Megabyte: Big pot of rice
Gigabyte: Truck full of rice
David Wellman: What is Big Data?

26. What is Big Data?
Byte: one rice
Kilobyte: handful of rice
Megabyte: Big pot of rice
Gigabyte: Truck full of rice
Terabyte: Containership full of rice
Petabyte: Covers Manhattan
David Wellman: What is Big Data?

27. What is Big Data?
Byte: one rice
Kilobyte: handful of rice
Megabyte: Big pot of rice
Gigabyte: Truck full of rice
Terabyte: Containership full of rice
Petabyte: Covers Manhattan
Exabyte: Covers the west coast of US
David Wellman: What is Big Data?

28. What is Big Data?
Byte: one rice
Kilobyte: handful of rice
Megabyte: Big pot of rice
Gigabyte: Truck full of rice
Terabyte: Containership full of rice
Petabyte: Covers Manhattan
Exabyte: Covers the west coast of US
Zettabyte: Fills the Pasific
David Wellman: What is Big Data?

29. What is Big Data?
Byte: one rice
Kilobyte: handful of rice
Megabyte: Big pot of rice
Gigabyte: Truck full of rice
Terabyte: Containership full of rice
Petabyte: Covers Manhattan
Exabyte: Covers the west coast of US
Zettabyte: Fills the Pacific
Yottabyte: Earth size riceball
David Wellman: What is Big Data?

30. What is Big Data?
Byte: one rice
Kilobyte: handful of rice
Megabyte: Big pot of rice
Gigabyte: Truck full of rice
Terabyte: Containership full of rice
Petabyte: Covers Manhattan
Exabyte: Covers the west coast of US
Zettabyte: Fills the Pacific
Yottabyte: Earth size riceball
David Wellman: What is Big Data?
Big Data
Internet
Computers
Early computers

31. What is Big Data?
Big Data is not about the size of the
date, it’s about the value within the
data
This value can be used for marketing,
businesses optimisation, getting
insights, improving health, security
etc.

32. Data Analytics

33. Why Big Data Analytics?
Understand the data the company has
Process data to see patterns,
corrections and information that can
be used to make better decisions
Obtain insights that are otherwise not
known

34. Data Analytics
TRADITIONAL APPROACH
Structured and Repeatable Analays
BIG DATA APPROACH
Iternative adn Exloratory Analays
Business users
Business users
Determine what
questions to ask
IT
Structures the data
to answer the
question
IT
Delivers a platform
to enable creative
discovery
Explores what
questions could be
asked

35. Tools for Data Analytics
NoSQL databases: MongoDB, Cassandra, Hbase, Hypertable
Storage: S3, Hadoop Distributed File System
Servers: EC2, Google App Engine, Heroku
MapReduce: Hadoop, Hive, Pig, Cascading, S4, MapR •
Processing: R, Yahoo! Pipes, Solr/Lucene, BigSheets,

36. Two Types of Data Analysis Problems
Supervised Learning:
Learn from data but we have labels
for all the data we’ve seen so far
Example: Determining Spam Emails
Learn from data but we don’t have
any labels
Example: Grouping Emails
Unsupervised Learning:
Learning is about discovering hidden patterns in data

37. Clustering
One of the oldest problems in unsupervised data analysis
In clustering the goal is to group data according to similarity
Algorithms such as K-means are used for clustering

38. Clustering
For each artifact found, the
location to N and E from
the Marker is recorded
That is a Data Set
Before the dig, a historian
has said that three families
lived in the location

39. Clustering
Similar: close in physical
distance
You assign each data point
to one and only one group
The groups are called
clusters

40. Clustering
Clustering them is the unsupervised learning problem
where you take your data and assign each data point to
exactly one group, or cluster
Uses unlabelled data

41. Clustering
We may have collection data but we don’t know what to
do with it
We might want to explore the data without a particular
end goal in mind
Perhaps the data will suggest interesting avenues for
further analysis
In this case, we say that we're performing exploratory
data analysis

42. Exploratory data analysis
We don’t know what we are looking for
Data point = color of pixel and location of pixel
Dissimilarity is the distance in color

43. Exploratory data analysis
In some cases
labelling is too
expensive
For example,
news change
every day and
there are too
much of them

44. Using Big Data to Influence People

45. Alexander Nix, CEO Cambridge Analytica

47. Data Analysis as a Platform
THEN NOW
Complex tools operated by Data Analysts
Chaos of data silos accross the company
Real-time data analytics platform like Looker

48. Customer Data as a Platform
THEN NOW
Difficult to customise, lack of
automated customer insights
Real-time Intelligent that automatically tracks
and analysis interaction wiht customer

49. Mapping Data as a Platform
THEN NOW
Difficult and expensive to collect data
Limited in-app digital map useage
Mapping platforms like Mapbox

50. Cloud Data Monitoring as a Platform
THEN NOW
Expensive and clunky point solution
Lengthy implementation cycles
Only used by System Administrators
Cloud monitoring platforms like Datadog

51. Next
Games and gamification
Why you should play video games and why it is important
that your kids play video games