A brief about Relational Database Management System (RDBMS), Apache Spark, and Hadoop and a comparison between them.

1. vs
vs
Submitted by:
Aastha Joshi
Aishwarya Singh
Joel Keith Pais
Laxmi
P Vignesh

2. RDBMS
Hadoop
Apache Spark
Comparison between them
Contents

3. RDBMS
Stands for ‘Relational Database Management
System’
It is a database that stores data in a structured
format using rows and columns.
One can execute queries on the data like
adding, updating, and searching for values.
It also provides a visual representation of the
data.
It is "relational" because the values within each
table are related to each other.
The relational structure makes it possible to run
queries across multiple tables at once.
Structured Query Language is the standard
programming language used to access the
database.
ADVANTAGES
Addresses the need for integrating,
managing and analysing data from
multiple sources across on-
premises and cloud environments
Ease to locate and access specific
values within the database
High flexibility due to storage,
retrieval and publishing of JSON data
within a relational database
EXAMPLES

4. It is a matter of the past when data were limited.
Now, the world has already experienced the power
of Big Data, and the same is used to analyze to
frame different business strategies and others.
Apache Hadoop is one of the kinds of open-
source platforms that we can use to store and
process relatively large datasets amounting from
gigabytes to petabytes. This open-source allows
multiple computers to make clusters and analyze
the large datasets in parallel and effectively.

5. Four main
components
of
the Hadoop
ecosystem:
HADOOP DISTRIBUTED FILE SYSTEM (HDFS)
A primary data storage system that runs on commodity
hardware and manages enormous data collections. It
also has a high data throughput and a high fault
tolerance.
YET ANOTHER RESOURCE NEGOTIATOR (YARN)
YARN is a cluster resource manager that schedules
tasks and assigns resources (such as CPU and memory)
to applications.
1
2
3
HADOOP MAPREDUCE
Breaks down the big data processing tasks into smaller
ones, distributes them across different nodes, and then
runs each one.
4
HADOOP COMMON (HADOOP CORE):
A collection of common libraries and utilities on which
the other three modules rely.

6. Importance
of
Hadoop
Ability to quickly store and handle large amounts of any type of data
That's an important concern as data volumes and varieties continue to grow, notably from social
media and the Internet of Things (IoT).
Computer processing power.
Hadoop's distributed computing model efficiently processes large amounts of data. The more
computing nodes you use, the more processing power you have.
Fault tolerance
Data and application processing are protected against hardware failure. If a node fails, jobs are
automatically transferred to other nodes, ensuring that the distributed computing does not fail.
Multiple copies of all data are stored automatically.
Flexibility
Unlike traditional relational databases, we don’t have to preprocess data before storing it. We can
store as much data as we want and decide how to use it later. It includes unstructured data like text,
pictures, and videos.
Low cost
The open-source framework is free and stores large amounts of data by using commodity
hardware.
Scalability
By simply adding nodes, we can easily expand our system to handle more data. A little administrative
is required.

7. Challenges in using Hadoop
1 MAPREDUCE PROGRAMMING ISN'T SUITED FOR EVERY PROBLEM
It performs well for simple information requests and problems that can be broken down into independent units, but it is
inefficient for iterative and interactive analytic operations. MapReduce is file-intensive. Iterative algorithms require
multiple map-shuffle/sort-reduce phases to complete because the nodes mainly communicate through sorts and
shuffles. This results in so many files being created between MapReduce phases, which is inefficient for advanced
analytics computing.
2
It can be difficult to find entry-level programmers who have adequate Java expertise to be productive with MapReduce.
That's one reason distribution providers are racing to put relational (SQL) technology on top of Hadoop. Programmers
with SQL skills are easy to find than MapReduce skills. And, Hadoop administration seems a mix of art and science,
requiring a basic understanding of operating systems, hardware, and Hadoop kernel settings.
THERE’S A WIDELY ACKNOWLEDGED TALENT GAP
3
Another concern is the fragmented data protection challenges, which are being handled by new tools and technology.
The Kerberos authentication protocol is a significant step toward securing Hadoop environments.
DATA SECURITY
4
Hadoop lacks user-friendly, full-featured tools for data management, data cleansing, governance, and metadata services.
FULL-FLEDGED DATA GOVERNANCE AND MANAGEMENT

8. Apache Spark began in 2009 as a research project at UC Berkeley's AMPLab focused on data-
intensive application areas.
Apache Spark is an open-source, distributed processing system used for big data workloads.
For rapid analytic queries against any quantity of data, it uses in-memory caching and efficient
query execution.
It allows code reuse across different workloads—batch processing, interactive queries, real-
time analytics, machine learning, and graph processing—and provides development APIs in
Java, Scala, Python, and R.
Spark's objective was to build a new framework that was optimised for quick iterative
processing, such as machine learning and interactive data analysis, while preserving Hadoop
MapReduce's scalability and fault tolerance.
The primary importance of Apache Spark in the Big data industry is because of its in-memory
data processing that makes it a high-speed data processing engine compared to MapReduce.
Apache Spark delivers a better-integrated framework which supports all ranges of Big data
formats like batch data, text data, real-time streaming data, graphical data, etc.
Apache Spark

9. Core Components
Spark SQL and Data Frames: Spark SQL
allows users to run SQL and HQL queries in
order to process structured and semi-
structured data.
Spark Streaming: Spark streaming facilitates
the processing of live stream data i.e. log files.
It also contains APIs to manipulate data
streams.
MLib Machine Learning: MLib is the Spark
library with machine learning functionality. It
contains various machine learning algorithms
such as regressions, clustering, collaborative
filtering, classification, etc.
GraphX: The library that supports graph
computation is known as GraphX. It
enables users to perform graph
manipulation. It also provides graph
computation algorithms.
Apache Spark Core API: It provides a
platform to execute Spark applications.
Apache Spark framework consists of the main five components that are responsible
for the functioning of the Spark.

10. Advantages
Speed: For large-scale data processing, Spark is 100 times quicker than Hadoop. Apache Spark
utilizes an in-memory (RAM) processing architecture.
Ease of Use: Apache Spark provides simple APIs for working with big datasets. It has over 80
high-level operators that make creating parallel programs a breeze.
Advanced Analytics: Spark does more than only support 'MAP' and 'reduce'. Machine learning
(ML), graph algorithms, streaming data, SQL queries, and other features are also supported.
Apache Spark is faster than most data warehouses.
Dynamic: Apache Spark allows simple creation of parallel apps. Over 80 high-level operators
are available through Spark.
Multilingual: Python, Java, Scala, and more programming languages are supported by Apache
Spark.
Powerful: Because of its low-latency in-memory data processing capacity, Apache Spark can
handle a wide range of analytics problems. It has well-developed libraries for graph analytics
and machine learning techniques.
Open-source: The best thing about Apache Spark is, it has a massive Open-source community
behind it.

12. THANK YOU