8. Data Analytics:
● handling data collected by systems
● generate the insights
● improve decision making
○ with facts based on data.
What is Data Analytics?🤔
Copyright 2021. Genese Cloud Academy
9. Types of Data Analytics
● Descriptive Analytics
● Diagnostic Analytics
● Predictive Analytics
● Prescriptive Analytics
What Are the Types of Data Analytics?🤔
10. Benefits Of Data Analytics
Credit card Fraud Detection Customer Personalization Security Threats Detection
Real time Alerting User Behaviour
Financial Modeling
11. Use Cases:
● Social Media
● Ecommerce
● Information Security
● Logistics
● Factory operations
● Internet of Things
What Might be the Use
Cases of Data Analytics?🤔
18. High Speed computing:
● implemented in
○ Super Computers for Scientific Research.
Main Area Of Discipline:
● Developing Parallel Processing Algorithm
● Developing Software
shifted from supercomputers to computing Clusters.
Application Areas of HPC
20. When do you actually need an HPC ?🤔
● Complete time consuming operations in less time
● Complete an operation under a tight schedule
● Perform a high number of operations per second
22. Streaming Data:
● generated continuously by thousands of data sources
Data Stream:
● Continuous
● Ordered
● Changing
● Fast
● huge amount
Traditional DBMS:
Data stored:
● Finite data sets
● Persistent data sets
Streaming data includes a wide variety of data
● log files
● ecommerce purchases,
● in-game player activity,
● information from social networks,
● financial trading floors,
● geospatial services,
● telemetry from connected devices or instrumentation in data centers.
23. Core Banking
● Improved Scalability
● Met HA and SHA needs
Online Gaming
● Increased reliability
● Accurate and real time data
● Ability to process data at scale
● Faster ramp time
24. Government Services
● Near real time events and
better data quality
● Increased efficiency
● Produce and store data
● Better privacy
Financial Services
● Enhanced Customer experience
● Improved fraud detection engine
25. Real-Time Fraud Detection
● Act in real time
● Detect Fraud
● Minimize risk
● Improve customer experience
Real-Time E-Commerce
● OnBoarding New Merchants Faster
● Enabled 360 view of customers
● Enhanced Performance & Monitoring
● Projected saving of Millions of dollars
Do You know About Amazon GO?
26.
27. Benefits of streaming data
● Improve operational efficiencies
● Reduce infrastructure cost
● Provide faster insights and actions
29. Challenges in working with streaming data
Streaming data processing requires two layers:
● a storage layer
○ record ordering
○ strong consistency
■ Fast
■ Inexpensive
■ replayable
● Reads
● Writes
● a processing layer.
○ consuming data from the storage layer
○ running computations on that data
○ notifying the storage layer to delete data that is no longer needed.
● Scalability
● data durability
● fault tolerance
30. Infrastructure to build streaming data applications:
● Amazon Kinesis Data Streams,
● Amazon Kinesis Data Firehose,
● Amazon Managed Streaming for Apache Kafka (Amazon MSK),
● Apache Flume,
● Apache Spark Streaming,
● Apache Storm.
31.
32. Working with streaming data
on AWS
Amazon Kinesis is a platform for streaming data on AWS
● load and analyze streaming data
● custom streaming data applications
offers three services:
● Amazon Managed Streaming for Apache Kafka
(Amazon MSK).
● Amazon Kinesis Data Firehose,
● Amazon Kinesis Data Streams
run other streaming data platforms
● Apache Flume,
● Apache Spark Streaming, and
● Apache Storm
○ on Amazon EC2
33. What is Visualization?🤔🤔🤔
any technique for creating images, diagrams, or animations to communicate a message.
Copyright 2021. Genese Cloud Academy
34.
35. Why is data visualization important?
Data visualization is a:
● Quick & easy way
○ to convey concepts in a universal manner
○ can experiment with different scenarios by making slight adjustments.
● Identify areas that need attention or improvement.
● Clarify which factors influence customer behavior.
● Help you understand which products to place where.
● Predict sales volumes.
36. Data visualization tools
business intelligence (BI) reporting tool.
set up visualization tools to:
● generate automatic dashboards that track company performance across key performance indicators
(KPIs)
● visually interpret the results.
Kibana Tableau Grafana QuickSight Power BI
37. What is Big Data?
Copyright 2021. Genese Cloud Academy
38. BIG DATA:
● collection of data
huge in volume
growing exponentially
with time.
Types Of Big Data
● Structured
● Unstructured
● Semi-structured
39. Why Learn Big Data?
● Gartner – Big Data is the new Oil.
● IDC – Its market will be growing 7 times faster than the overall IT market.
● IBM – It is not just a technology – it’s a Business Strategy for capitalizing on information resources.
● IBM – Big Data is the biggest buzz word because technology makes it possible to analyze all the
available data.
42. Your First Big Data
Application on AWS
Word Count example
Copyright 2021. Genese Cloud Academy
Hinweis der Redaktion
Data Analytics is Vital to every Business.
-helps decision makers (based on analytics and data)
-critical tasks (launching New Product,Offering Discounts,marketing New Areas) requires time sensitive decision and experience.
Organizations spend millions of dollars on data storage. The problem isn’t finding the data — the problem is failing to do anything with it, AWS
Volume : total amount of data that is coming in and will be ingested into the system.
Velocity : speed at which data is flowing in, the challenge consists of processing the data in near real-time and return results as quickly as possible. validating a credit card transaction must instantaneous (near real-time)
Variety : Data to be ingested in the system can have different formats,.
Veracity : accuracy of incoming data.
Value :Decisions makers seek to extract meaningful information and insights from systems to have a competitive edge.
Area of discipline can be divided into small independent parts and can be executed simultaneously by separate processors.
Bank
Pubg
Amazon Go
LInk to video: https://youtu.be/NrmMk1Myrxc
applying machine learning algorithms,
and extract deeper insights from the data.
Over time, complex, stream and event processing algorithms, like decaying time windows
to find the most recent popular movies, are applied, further enriching the insights.
applying machine learning algorithms,
and extract deeper insights from the data.
Over time, complex, stream and event processing algorithms, like decaying time windows
to find the most recent popular movies, are applied, further enriching the insights.
Drift : The ability to detect and adapt to changes in the distribution of examples is paramount for data stream mining algorithms
One pass:a one-pass or single-pass is a streaming algorithm which reads its input exactly once. It does so by processing items in order, without unbounded buffering; it reads a block into an input buffer, processes it, and moves the result into an output buffer for each step in the process.
Real-time:
Streaming data is data that is continuously generated and delivered rather than processed in batches or micro-batches. ... The terms “real-time” and “stream” converge in “real-time stream processing” to describe streams of real-time data that are gathered and processed as they are generated.
Bound data is finite and unchanging data, where everything is known about the set of data. Typically Bound data has a known ending point and is relatively fixed.
applying machine learning algorithms,
and extract deeper insights from the data.
Over time, complex, stream and event processing algorithms, like decaying time windows
to find the most recent popular movies, are applied, further enriching the insights.
deploy and manage your own streaming data solution in the cloud on Amazon EC2.
Because of the way the human brain processes information,
using charts or graphs to visualize large amounts of complex data is easier than poring over spreadsheets or reports.
Data visualization is a quick, easy way to convey concepts in a universal manner – and
can experiment with different scenarios by making slight adjustments.
data with so large size and complexity that none of traditional data management tools can store it or process it efficiently. Big data is also a data but with huge size.