Big Data has shaped much of the tech innovation happening around the world today giving people immense power to make sense of large blobs of structured and unstructured data. Join Riju Saha, Digital Excellence Head, Oracle COE at Tata Consultancy Services to decode the fundamentals of Big data and how can you build a career in this fascinating field.

1. Big Data : An Introduction

2. About the
Speaker

3. Agenda
❑ About Data
❑ Big Data
❑ Concepts
❑ Tools
❑ Architecture Patterns
❑ Trends
❑ Opportunities

4. The Data Life
❑ 3.7 million Google search queries
❑ 4.3 million YouTube views
❑ 38 million text sent on WhatsApp
❑ 266 000 hours watched on Netflix
❑ 990,000 Tinder swipes
❑ Uber riders take 45,788 trips
❑ 25.9 million real time payments
❑ 4 million data points per engine per
flight
Source: Data Never Sleeps 6.0

5. Digital Universe
❑ Growing 40% a year into the next
decade
❑ It is doubling in size every 2 years
❑ Data generated by People,
Enterprise and Smart Devices
❑ Forecast for 2020 : 44 zettabytes
or 44 trillion gigabytes
Source: www.i-scoop.eu :Order From Chaos

6. Big Data and Its Dimensions
! Volume
! Velocity
! Variety
Emergence of new Dimensions
! Value
! Veracity
! Visualisation
! Viscosity
! Virality

7. Big Data Categories
! Structured
! Human- or machine- generated and highly organized information that are stored in relational format
! Structured data represent only 5 to 10% of all data
! Semi Structured
! Has internal semantic tags and markings that identify separate elements, but lacks the structure required
to fit in a relational database
! Represents 5-10% of Data
! Unstructured
! Data that has no identifiable internal structure or that doesn’t fit neatly in a relational database format
! 85% of the data around the world is considered unstructured data (Digital Reasoning website, IBM)

8. Revisiting the
Definition
! ERP/CRM processing highly
structured data
! Human interaction via Web/
Clickstream/Social - Semi
Structured/Unstructured
! Observational Data from IoT
(Sensors, RFID, GPS) -
Unstructured

9. The Value Funnel

10. The Big Data Tools
! Infrastructure
! Cloudera, AWS
! Databases
! Neo4j (Graphing DB)
! Snowflake (Cloud EDW)
! Framework
! Hadoop, Spark
! Streaming
! Kafka, Storm
! AI/Deep Learning
! Mahout, Tensorflow
! Search
! Solr, Elastic Search
! Visualization
! Tableau, Qlik view, Zeppelin

11. Architecture Blocks

12. Sample Architecture
Source: https://www.irisidea.com

13. Trends
❑ IoT network proliferation : Smart Device networks
❑ Increased adoption of AI in daily life of people and Enteprises
❑ Predictive analytics and prescriptive analytics to gauge demand and service requests
❑ Augmented Reality : More Immersive experience
❑ Access to Dark Data : Old Manuscripts, Historical Archives
❑ Smart chatbots : Augmented intelligence
❑ Rise of Quantum Computing : Increased computer power
❑ Elasticity and fluidity in infrastructure on Cloud

14. Opportunities
❑ Data Officers : Defines the enterprise wise data strategy, Defines the Who, What, Where of Data
❑ Big Data Architects : Responsible for defining the Big Data foundation and orchestration of existing
setup with Big Data
❑ Data Engineers : Responsible for Design and processing of Big Data
❑ Business Intelligence Analysts : Responsible for analyzing data and recommend business
❑ System Administrators : Crucial role. Maintains and secures the systems.
❑ Trivia ~ Yahoo has 4500 node Hadoop cluster storing 455PB pf data
❑ Data Scientists : In Emerging phase, responsible for analyzing data and creating intelligent models
for business to add value to their operations.

15. Convergence

16. Basic Skills Reqd
❑ Operations Research
❑ Statistical Modeling
❑ Analytics Methods
❑ Social Sciences
❑ Design Thinking

17. Questions !