SlideShare ist ein Scribd-Unternehmen logo
1 von 38
Downloaden Sie, um offline zu lesen
AI / ML Technologies and
Application for Product
Development
Chetan Khatri
FOSSASIA Summit, 2018
@khatri_chetan
Accionlabs Artificial Intelligence | Machine Learning
WHO AM I ?
Lead - Data Science, Technology Evangelist @ Accion labs India Pvt. Ltd.
Committer @ Apache Spark, Apache HBase, Elixir Lang.
Co-Authored University Curriculum @ University of Kachchh.
Software Engineering @: Nazara Games, Eccella Corporation.
M.Sc. - Computer Science from University of Kachchh.
Accionlabs Artificial Intelligence | Machine Learning
What is Artificial Intelligence ?
“Artificial intelligence is intelligence demonstrated by machines, in contrast to the natural intelligence
displayed by humans and other animals”. - Wikipedia
The Original definition of A.I:
“Every aspect of learning or any other feature of intelligence can in principle be so precisely described that a
machine can be made it simulate it. An attempt will be made to find how to make machines use language,
form abstractions and concepts, solve kinds of problems now reserved for humans, and improve
themselves”.
- John McCarthy at Dartmouth Conference. 1955
AI is new electricity!
Accionlabs Artificial Intelligence | Machine Learning
“Software is eating the world, and A.I is eating
software !”
- GPUs / TPUs are eating Linear Algebra.
- Linear Algebra is eating Deep Learning.
- Deep Learning is eating Machine Learning.
- Machine Learning is eating Artificial Intelligence (AI).
- AI is eating Software.
- Software is eating the world.
Accionlabs Artificial Intelligence | Machine Learning
Machine learning
It is a subfield of AI concerned with algorithms that allow computer to learn from examples/data and
experience. Machine
Learning Machine
Learning
Supervised
Learning
Unsupervised
Learning
Reinforcement
Learning
Deep Neural
Network / Deep
learning
Accionlabs Artificial Intelligence | Machine Learning
How deep learning is different ?
Data
Feature
engineering
Statistical
Machine learning
Model
Features
Unseen Sample
Data
Deep Learning kind of
Machine learning
Model
Automatically identifies
features !
Prediction
Prediction
Unseen Sample
Accionlabs Artificial Intelligence | Machine Learning
What is an Artificial Neural Network ?
BEACH
SEA
SKY
...
Accionlabs Artificial Intelligence | Machine Learning
Neural Networks have been around for a while ...
Accionlabs Artificial Intelligence | Machine Learning
… But then this happened
1
WEB-SCALE
DATA
Data volumes double
every year
Accionlabs Artificial Intelligence | Machine Learning
… But then this happened
2
WEB-SCALE
DATA
Massive Adoption of GPU
and TPU.
Accionlabs Artificial Intelligence | Machine Learning
… But then this happened
3
ADVANCED NEURAL
NETWORK DESIGNS
Accionlabs Artificial Intelligence | Machine Learning
Supervised Learning !
A (INPUT) B (RESPONSE)
EMAIL SPAM ? (0/1)
IMAGE
OBJECT(1,...,10
00)
AUDIO TEXT
ENGLISH FRENCH
Accionlabs Artificial Intelligence | Machine Learning
Machine learning / AI: Use Cases
Regression &
Classification
Real-time Data
Analysis
Character
Recognition (OCR)
ML Workload
Automation
Recommendation
& Personalization
Natural Language
Processing
Computer Vision
Data
Visualization
Information
Extraction
Deep Neural
Network
Conversational
Dialog Agent
(Bots)
Speech
Recognition
Forecasting
Predictions
Reinforcement
Learning
Scaling Prediction
services
Federated
Learning
Accionlabs Artificial Intelligence | Machine Learning
TOOLS &
TECHNOLOGIES
Languages
Data Quality
Accionlabs Artificial Intelligence | Machine Learning
Predictive
Modelling
Deep
Learning / AI
TOOLS &
TECHNOLOGIES
Accionlabs Artificial Intelligence | Machine Learning
Data
Visualization
Containerized Scheduling
TOOLS &
TECHNOLOGIES
Accionlabs Artificial Intelligence | Machine Learning
Scheduling
GPU Enabled
TOOLS &
TECHNOLOGIES
Accionlabs Artificial Intelligence | Machine Learning
Artificial Intelligence and Machine learning
Automating the organization
Improving Decision making &
Reducing Inefficiencies
Accionlabs Artificial Intelligence | Machine Learning
Machine learning process
Collect and
prepare
training data
Choose and
optimize your
ML algorithm
Setup and manage
environments for
training
Train and tune model
(trial and error)
Deploy model
in production
Scale and manage the
production environment
Accionlabs Artificial Intelligence | Machine Learning
High level Architecture
BUILD TRAIN
Machine learning as
Service
ALGORITHMS
FRAMEWORKS
K-Means Clustering
Principal Component Analysis
Neural Topic Modelling
Factorization Machines
Linear Learner - Regression
Deep Reinforcement Learner
Convolutional Neural Network
XGBoost
Latent Dirichlet Allocation
Seq2Seq , LSTM
Recurrent Neural Network
Linear Learner - Classification
TensorFlow
PyTorch
Torch
Caffe2
CNTK
Caffe
Apache Spark MLlib
Apache Flink
Setup and manage
environments for
training
Train and tune
model
(trial and error)
Deploy model
in production
Scale & manage
the production
environment
Accionlabs Artificial Intelligence | Machine Learning
Applications of AI : Computer Vision
Self-Driving
Cars
Convolution Neural
Network
Recurrent Neural
network
Vision i.e Camera Object recognition and identification
Real time, Per pixel Object Segmentation
Centimeter-accurate positioning
Accionlabs Artificial Intelligence | Machine Learning
Applications of AI : Computer Vision
Visual Search &
Recommendation
Convolution Neural
Network
Recurrent Neural
network
Camera
Accionlabs Artificial Intelligence | Machine Learning
Applications of AI : Computer Vision
Accionlabs Artificial Intelligence | Machine Learning
Applications of AI : Computer Vision
Ad / User Click ? (0/1) Whether user will click to adv. or not ?
UI Wireframe UI Screen
Possible ???
Screenshot Source Code
Accionlabs Artificial Intelligence | Machine Learning
Applications of AI : Computer Vision
Ad / User Click ? (0/1) Whether user will click to adv. or not ?
UI Wireframe UI Screen
Possible ???
Screenshot Source Code
DEMO
Accionlabs Artificial Intelligence | Machine Learning
Applications of AI : Computer Vision - DEMO
UI Wireframe UI Screen
Accionlabs Artificial Intelligence | Machine Learning
Applications of AI : Computer Vision - DEMO
Screenshot Source Code
Accionlabs Artificial Intelligence | Machine Learning
Natural Language Processing
Entities
Key Phrases
Language
Sentiment
Topics
Accionlabs Artificial Intelligence | Machine Learning
Application of AI: Natural Language Processing
Business Applications
Entity Recognition
Sentiment Analysis
Content Classification
Language Translation, Speech Recognition
Conversational Dialog Agent / Chatbots
Syntax Analysis (Key Phrases, Language
Understanding)
Techniques
Recurrent Neural Network
Latent Dirichlet Allocation (LDA)
Neural Topic Modeling
Neural Recursive Network / Attention Network
Transfer Learning
Seq-to-Seq + LSTM
Accionlabs Artificial Intelligence | Machine Learning
Challenges
- Interoperability of Deep Learning frameworks.
- Scale Inference at Production.
- Distribution and Parallelism of ML / AI Models.
- Build reusable components that work well together (across frameworks) - UNIX Philosophy
- Train at GPU / CPU - Inference @ Mobile, Raspberry PI, Drone, Streaming Application
Accionlabs Artificial Intelligence | Machine Learning
Facial Emotion Recognition: DEMO
1.
Facial
Emotion
Recognition
Convolution Neural
Network
Recurrent Neural
network
Vision i.e Camera
Accionlabs Artificial Intelligence | Machine Learning
Real time object recognition : DEMO
Real time
Object
recognition
Convolution Neural
Network
Recurrent Neural
network
Vision i.e Camera
Everyone has own choice of Deep Learning
Frameworks
Research to Production
Everyone has own choice of Deep Learning
Frameworks
Reimplementation takes
Weeks or months
Accionlabs Artificial Intelligence | Machine Learning
Deep Learning Framework Zoo
...
Framework
backends
Vendor and numeric libraries
...
O (n2
) pairs
Apple CoreML Nvidia TensorRT
Intel/Nervana
ngraph
Qualcomm SNPE
Accionlabs Artificial Intelligence | Machine Learning
Open Neural Network Exchange (ONNX)
...
Framework
backends
Vendor and numeric libraries
...
Apple CoreML Nvidia TensorRT
Intel/Nervana
ngraph
Qualcomm SNPE
Shared model and operator representation
From O(n2
) to O(n) pairs
Accionlabs Artificial Intelligence | Machine Learning
Train at GPU, Inference at Mobile App !
...
Apple CoreML TensorFlow Light
Accionlabs Artificial Intelligence | Machine Learning
Thanks !
@khatri_chetan chetan.khatri@accionlabs.com

Weitere ähnliche Inhalte

Ähnlich wie Fossasia ai-ml technologies and application for product development-chetan khatri

HKOSCon18 - Chetan Khatri - Open Source AI / ML Technologies and Application ...
HKOSCon18 - Chetan Khatri - Open Source AI / ML Technologies and Application ...HKOSCon18 - Chetan Khatri - Open Source AI / ML Technologies and Application ...
HKOSCon18 - Chetan Khatri - Open Source AI / ML Technologies and Application ...Chetan Khatri
 
Artificial Intelligence and Machine Learning
Artificial Intelligence and Machine LearningArtificial Intelligence and Machine Learning
Artificial Intelligence and Machine LearningMykola Dobrochynskyy
 
artificial intelligence ppt.pptx
artificial intelligence ppt.pptxartificial intelligence ppt.pptx
artificial intelligence ppt.pptxBrijithaGokula
 
Machine learning in startup
Machine learning in startupMachine learning in startup
Machine learning in startupMasas Dani
 
Artificial Intelligence
Artificial IntelligenceArtificial Intelligence
Artificial IntelligencePrakhyath Rai
 
Smart Data Webinar: Machine Learning Update
Smart Data Webinar: Machine Learning UpdateSmart Data Webinar: Machine Learning Update
Smart Data Webinar: Machine Learning UpdateDATAVERSITY
 
Facial expression recognition projc 2 (3) (1)
Facial expression recognition projc 2 (3) (1)Facial expression recognition projc 2 (3) (1)
Facial expression recognition projc 2 (3) (1)AbhiAchalla
 
AI in Finance: Moving forward!
AI in Finance: Moving forward!AI in Finance: Moving forward!
AI in Finance: Moving forward!Adrian Hornsby
 
unleshing the the Power Azure Open AI - MCT Summit middle east 2024 Riyhad.pptx
unleshing the the Power Azure Open AI - MCT Summit middle east 2024 Riyhad.pptxunleshing the the Power Azure Open AI - MCT Summit middle east 2024 Riyhad.pptx
unleshing the the Power Azure Open AI - MCT Summit middle east 2024 Riyhad.pptxUsama Wahab Khan Cloud, Data and AI
 
Introduction to Amazon AI & Deep Learning
Introduction to Amazon AI & Deep LearningIntroduction to Amazon AI & Deep Learning
Introduction to Amazon AI & Deep LearningAmazon Web Services
 
Moving Forward with AI
Moving Forward with AIMoving Forward with AI
Moving Forward with AIAdrian Hornsby
 
Tensorflow IoT - 1 Wk coding challenge
Tensorflow IoT - 1 Wk coding challengeTensorflow IoT - 1 Wk coding challenge
Tensorflow IoT - 1 Wk coding challengegeetachauhan
 
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062Wael Alawsey
 
大會主題演說 2: AI x 機器學習,無所不在!Ubiquity of AI and Machine Learning in our Everyday ...
大會主題演說 2: AI x 機器學習,無所不在!Ubiquity of AI and Machine Learning in our Everyday ...大會主題演說 2: AI x 機器學習,無所不在!Ubiquity of AI and Machine Learning in our Everyday ...
大會主題演說 2: AI x 機器學習,無所不在!Ubiquity of AI and Machine Learning in our Everyday ...Amazon Web Services
 
"Imaging + AI: Opportunities Inside the Car and Beyond," a Presentation from ...
"Imaging + AI: Opportunities Inside the Car and Beyond," a Presentation from ..."Imaging + AI: Opportunities Inside the Car and Beyond," a Presentation from ...
"Imaging + AI: Opportunities Inside the Car and Beyond," a Presentation from ...Edge AI and Vision Alliance
 
[Srijan Wednesday Webinars] Artificial Intelligence & the Future of Business
[Srijan Wednesday Webinars] Artificial Intelligence & the Future of Business[Srijan Wednesday Webinars] Artificial Intelligence & the Future of Business
[Srijan Wednesday Webinars] Artificial Intelligence & the Future of BusinessSrijan Technologies
 
a-i-presentation-121229232307-phpapp02 (2) (2).pptx
a-i-presentation-121229232307-phpapp02 (2) (2).pptxa-i-presentation-121229232307-phpapp02 (2) (2).pptx
a-i-presentation-121229232307-phpapp02 (2) (2).pptxAYESHASIDDIQA702386
 
a-i-presentation-121229232307-phpapp02 (2) (2).pptx
a-i-presentation-121229232307-phpapp02 (2) (2).pptxa-i-presentation-121229232307-phpapp02 (2) (2).pptx
a-i-presentation-121229232307-phpapp02 (2) (2).pptxAYESHASIDDIQA702386
 

Ähnlich wie Fossasia ai-ml technologies and application for product development-chetan khatri (20)

HKOSCon18 - Chetan Khatri - Open Source AI / ML Technologies and Application ...
HKOSCon18 - Chetan Khatri - Open Source AI / ML Technologies and Application ...HKOSCon18 - Chetan Khatri - Open Source AI / ML Technologies and Application ...
HKOSCon18 - Chetan Khatri - Open Source AI / ML Technologies and Application ...
 
Artificial Intelligence and Machine Learning
Artificial Intelligence and Machine LearningArtificial Intelligence and Machine Learning
Artificial Intelligence and Machine Learning
 
artificial intelligence ppt.pptx
artificial intelligence ppt.pptxartificial intelligence ppt.pptx
artificial intelligence ppt.pptx
 
Machine learning in startup
Machine learning in startupMachine learning in startup
Machine learning in startup
 
Artificial Intelligence
Artificial IntelligenceArtificial Intelligence
Artificial Intelligence
 
Smart Data Webinar: Machine Learning Update
Smart Data Webinar: Machine Learning UpdateSmart Data Webinar: Machine Learning Update
Smart Data Webinar: Machine Learning Update
 
Artificial Intelligence
Artificial IntelligenceArtificial Intelligence
Artificial Intelligence
 
Facial expression recognition projc 2 (3) (1)
Facial expression recognition projc 2 (3) (1)Facial expression recognition projc 2 (3) (1)
Facial expression recognition projc 2 (3) (1)
 
AI in Finance: Moving forward!
AI in Finance: Moving forward!AI in Finance: Moving forward!
AI in Finance: Moving forward!
 
unleshing the the Power Azure Open AI - MCT Summit middle east 2024 Riyhad.pptx
unleshing the the Power Azure Open AI - MCT Summit middle east 2024 Riyhad.pptxunleshing the the Power Azure Open AI - MCT Summit middle east 2024 Riyhad.pptx
unleshing the the Power Azure Open AI - MCT Summit middle east 2024 Riyhad.pptx
 
Introduction to Amazon AI & Deep Learning
Introduction to Amazon AI & Deep LearningIntroduction to Amazon AI & Deep Learning
Introduction to Amazon AI & Deep Learning
 
Moving Forward with AI
Moving Forward with AIMoving Forward with AI
Moving Forward with AI
 
Tensorflow IoT - 1 Wk coding challenge
Tensorflow IoT - 1 Wk coding challengeTensorflow IoT - 1 Wk coding challenge
Tensorflow IoT - 1 Wk coding challenge
 
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062
ARTIFICIAL INTELLIGENT ( ITS / TASK 6 ) done by Wael Saad Hameedi / P71062
 
大會主題演說 2: AI x 機器學習,無所不在!Ubiquity of AI and Machine Learning in our Everyday ...
大會主題演說 2: AI x 機器學習,無所不在!Ubiquity of AI and Machine Learning in our Everyday ...大會主題演說 2: AI x 機器學習,無所不在!Ubiquity of AI and Machine Learning in our Everyday ...
大會主題演說 2: AI x 機器學習,無所不在!Ubiquity of AI and Machine Learning in our Everyday ...
 
"Imaging + AI: Opportunities Inside the Car and Beyond," a Presentation from ...
"Imaging + AI: Opportunities Inside the Car and Beyond," a Presentation from ..."Imaging + AI: Opportunities Inside the Car and Beyond," a Presentation from ...
"Imaging + AI: Opportunities Inside the Car and Beyond," a Presentation from ...
 
AI at Google (30 min)
AI at Google (30 min)AI at Google (30 min)
AI at Google (30 min)
 
[Srijan Wednesday Webinars] Artificial Intelligence & the Future of Business
[Srijan Wednesday Webinars] Artificial Intelligence & the Future of Business[Srijan Wednesday Webinars] Artificial Intelligence & the Future of Business
[Srijan Wednesday Webinars] Artificial Intelligence & the Future of Business
 
a-i-presentation-121229232307-phpapp02 (2) (2).pptx
a-i-presentation-121229232307-phpapp02 (2) (2).pptxa-i-presentation-121229232307-phpapp02 (2) (2).pptx
a-i-presentation-121229232307-phpapp02 (2) (2).pptx
 
a-i-presentation-121229232307-phpapp02 (2) (2).pptx
a-i-presentation-121229232307-phpapp02 (2) (2).pptxa-i-presentation-121229232307-phpapp02 (2) (2).pptx
a-i-presentation-121229232307-phpapp02 (2) (2).pptx
 

Mehr von Chetan Khatri

Data Science for Beginner by Chetan Khatri and Deptt. of Computer Science, Ka...
Data Science for Beginner by Chetan Khatri and Deptt. of Computer Science, Ka...Data Science for Beginner by Chetan Khatri and Deptt. of Computer Science, Ka...
Data Science for Beginner by Chetan Khatri and Deptt. of Computer Science, Ka...Chetan Khatri
 
Demystify Information Security & Threats for Data-Driven Platforms With Cheta...
Demystify Information Security & Threats for Data-Driven Platforms With Cheta...Demystify Information Security & Threats for Data-Driven Platforms With Cheta...
Demystify Information Security & Threats for Data-Driven Platforms With Cheta...Chetan Khatri
 
PyconZA19-Distributed-workloads-challenges-with-PySpark-and-Airflow
PyconZA19-Distributed-workloads-challenges-with-PySpark-and-AirflowPyconZA19-Distributed-workloads-challenges-with-PySpark-and-Airflow
PyconZA19-Distributed-workloads-challenges-with-PySpark-and-AirflowChetan Khatri
 
ScalaTo July 2019 - No more struggles with Apache Spark workloads in production
ScalaTo July 2019 - No more struggles with Apache Spark workloads in productionScalaTo July 2019 - No more struggles with Apache Spark workloads in production
ScalaTo July 2019 - No more struggles with Apache Spark workloads in productionChetan Khatri
 
No more struggles with Apache Spark workloads in production
No more struggles with Apache Spark workloads in productionNo more struggles with Apache Spark workloads in production
No more struggles with Apache Spark workloads in productionChetan Khatri
 
PyConLT19-No_more_struggles_with_Apache_Spark_(PySpark)_workloads_in_production
PyConLT19-No_more_struggles_with_Apache_Spark_(PySpark)_workloads_in_productionPyConLT19-No_more_struggles_with_Apache_Spark_(PySpark)_workloads_in_production
PyConLT19-No_more_struggles_with_Apache_Spark_(PySpark)_workloads_in_productionChetan Khatri
 
Automate ml workflow_transmogrif_ai-_chetan_khatri_berlin-scala
Automate ml workflow_transmogrif_ai-_chetan_khatri_berlin-scalaAutomate ml workflow_transmogrif_ai-_chetan_khatri_berlin-scala
Automate ml workflow_transmogrif_ai-_chetan_khatri_berlin-scalaChetan Khatri
 
HBaseConAsia 2018 - Scaling 30 TB's of Data lake with Apache HBase and Scala ...
HBaseConAsia 2018 - Scaling 30 TB's of Data lake with Apache HBase and Scala ...HBaseConAsia 2018 - Scaling 30 TB's of Data lake with Apache HBase and Scala ...
HBaseConAsia 2018 - Scaling 30 TB's of Data lake with Apache HBase and Scala ...Chetan Khatri
 
TransmogrifAI - Automate Machine Learning Workflow with the power of Scala an...
TransmogrifAI - Automate Machine Learning Workflow with the power of Scala an...TransmogrifAI - Automate Machine Learning Workflow with the power of Scala an...
TransmogrifAI - Automate Machine Learning Workflow with the power of Scala an...Chetan Khatri
 
An Introduction to Spark with Scala
An Introduction to Spark with ScalaAn Introduction to Spark with Scala
An Introduction to Spark with ScalaChetan Khatri
 
HBase with Apache Spark POC Demo
HBase with Apache Spark POC DemoHBase with Apache Spark POC Demo
HBase with Apache Spark POC DemoChetan Khatri
 
HKOSCon18 - Chetan Khatri - Scaling TB's of Data with Apache Spark and Scala ...
HKOSCon18 - Chetan Khatri - Scaling TB's of Data with Apache Spark and Scala ...HKOSCon18 - Chetan Khatri - Scaling TB's of Data with Apache Spark and Scala ...
HKOSCon18 - Chetan Khatri - Scaling TB's of Data with Apache Spark and Scala ...Chetan Khatri
 
Fossasia 2018-chetan-khatri
Fossasia 2018-chetan-khatriFossasia 2018-chetan-khatri
Fossasia 2018-chetan-khatriChetan Khatri
 
An Introduction Linear Algebra for Neural Networks and Deep learning
An Introduction Linear Algebra for Neural Networks and Deep learningAn Introduction Linear Algebra for Neural Networks and Deep learning
An Introduction Linear Algebra for Neural Networks and Deep learningChetan Khatri
 
Introduction to Computer Science
Introduction to Computer ScienceIntroduction to Computer Science
Introduction to Computer ScienceChetan Khatri
 
An introduction to Git with Atlassian Suite
An introduction to Git with Atlassian SuiteAn introduction to Git with Atlassian Suite
An introduction to Git with Atlassian SuiteChetan Khatri
 
Think machine-learning-with-scikit-learn-chetan
Think machine-learning-with-scikit-learn-chetanThink machine-learning-with-scikit-learn-chetan
Think machine-learning-with-scikit-learn-chetanChetan Khatri
 
A step towards machine learning at accionlabs
A step towards machine learning at accionlabsA step towards machine learning at accionlabs
A step towards machine learning at accionlabsChetan Khatri
 
Voltage measurement using arduino
Voltage measurement using arduinoVoltage measurement using arduino
Voltage measurement using arduinoChetan Khatri
 
Design & Building Smart Energy Meter
Design & Building Smart Energy MeterDesign & Building Smart Energy Meter
Design & Building Smart Energy MeterChetan Khatri
 

Mehr von Chetan Khatri (20)

Data Science for Beginner by Chetan Khatri and Deptt. of Computer Science, Ka...
Data Science for Beginner by Chetan Khatri and Deptt. of Computer Science, Ka...Data Science for Beginner by Chetan Khatri and Deptt. of Computer Science, Ka...
Data Science for Beginner by Chetan Khatri and Deptt. of Computer Science, Ka...
 
Demystify Information Security & Threats for Data-Driven Platforms With Cheta...
Demystify Information Security & Threats for Data-Driven Platforms With Cheta...Demystify Information Security & Threats for Data-Driven Platforms With Cheta...
Demystify Information Security & Threats for Data-Driven Platforms With Cheta...
 
PyconZA19-Distributed-workloads-challenges-with-PySpark-and-Airflow
PyconZA19-Distributed-workloads-challenges-with-PySpark-and-AirflowPyconZA19-Distributed-workloads-challenges-with-PySpark-and-Airflow
PyconZA19-Distributed-workloads-challenges-with-PySpark-and-Airflow
 
ScalaTo July 2019 - No more struggles with Apache Spark workloads in production
ScalaTo July 2019 - No more struggles with Apache Spark workloads in productionScalaTo July 2019 - No more struggles with Apache Spark workloads in production
ScalaTo July 2019 - No more struggles with Apache Spark workloads in production
 
No more struggles with Apache Spark workloads in production
No more struggles with Apache Spark workloads in productionNo more struggles with Apache Spark workloads in production
No more struggles with Apache Spark workloads in production
 
PyConLT19-No_more_struggles_with_Apache_Spark_(PySpark)_workloads_in_production
PyConLT19-No_more_struggles_with_Apache_Spark_(PySpark)_workloads_in_productionPyConLT19-No_more_struggles_with_Apache_Spark_(PySpark)_workloads_in_production
PyConLT19-No_more_struggles_with_Apache_Spark_(PySpark)_workloads_in_production
 
Automate ml workflow_transmogrif_ai-_chetan_khatri_berlin-scala
Automate ml workflow_transmogrif_ai-_chetan_khatri_berlin-scalaAutomate ml workflow_transmogrif_ai-_chetan_khatri_berlin-scala
Automate ml workflow_transmogrif_ai-_chetan_khatri_berlin-scala
 
HBaseConAsia 2018 - Scaling 30 TB's of Data lake with Apache HBase and Scala ...
HBaseConAsia 2018 - Scaling 30 TB's of Data lake with Apache HBase and Scala ...HBaseConAsia 2018 - Scaling 30 TB's of Data lake with Apache HBase and Scala ...
HBaseConAsia 2018 - Scaling 30 TB's of Data lake with Apache HBase and Scala ...
 
TransmogrifAI - Automate Machine Learning Workflow with the power of Scala an...
TransmogrifAI - Automate Machine Learning Workflow with the power of Scala an...TransmogrifAI - Automate Machine Learning Workflow with the power of Scala an...
TransmogrifAI - Automate Machine Learning Workflow with the power of Scala an...
 
An Introduction to Spark with Scala
An Introduction to Spark with ScalaAn Introduction to Spark with Scala
An Introduction to Spark with Scala
 
HBase with Apache Spark POC Demo
HBase with Apache Spark POC DemoHBase with Apache Spark POC Demo
HBase with Apache Spark POC Demo
 
HKOSCon18 - Chetan Khatri - Scaling TB's of Data with Apache Spark and Scala ...
HKOSCon18 - Chetan Khatri - Scaling TB's of Data with Apache Spark and Scala ...HKOSCon18 - Chetan Khatri - Scaling TB's of Data with Apache Spark and Scala ...
HKOSCon18 - Chetan Khatri - Scaling TB's of Data with Apache Spark and Scala ...
 
Fossasia 2018-chetan-khatri
Fossasia 2018-chetan-khatriFossasia 2018-chetan-khatri
Fossasia 2018-chetan-khatri
 
An Introduction Linear Algebra for Neural Networks and Deep learning
An Introduction Linear Algebra for Neural Networks and Deep learningAn Introduction Linear Algebra for Neural Networks and Deep learning
An Introduction Linear Algebra for Neural Networks and Deep learning
 
Introduction to Computer Science
Introduction to Computer ScienceIntroduction to Computer Science
Introduction to Computer Science
 
An introduction to Git with Atlassian Suite
An introduction to Git with Atlassian SuiteAn introduction to Git with Atlassian Suite
An introduction to Git with Atlassian Suite
 
Think machine-learning-with-scikit-learn-chetan
Think machine-learning-with-scikit-learn-chetanThink machine-learning-with-scikit-learn-chetan
Think machine-learning-with-scikit-learn-chetan
 
A step towards machine learning at accionlabs
A step towards machine learning at accionlabsA step towards machine learning at accionlabs
A step towards machine learning at accionlabs
 
Voltage measurement using arduino
Voltage measurement using arduinoVoltage measurement using arduino
Voltage measurement using arduino
 
Design & Building Smart Energy Meter
Design & Building Smart Energy MeterDesign & Building Smart Energy Meter
Design & Building Smart Energy Meter
 

Kürzlich hochgeladen

Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksdeepakthakur548787
 
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming PipelinesConf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming PipelinesTimothy Spann
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Seán Kennedy
 
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Boston Institute of Analytics
 
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Boston Institute of Analytics
 
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxThe Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxTasha Penwell
 
What To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptxWhat To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptxSimranPal17
 
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024Susanna-Assunta Sansone
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfblazblazml
 
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxaleedritatuxx
 
Networking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxNetworking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxHimangsuNath
 
Advanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsAdvanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsVICTOR MAESTRE RAMIREZ
 
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default  Presentation : Data Analysis Project PPTPredictive Analysis for Loan Default  Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPTBoston Institute of Analytics
 
SMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxSMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxHaritikaChhatwal1
 
INTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processingINTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processingsocarem879
 
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...Boston Institute of Analytics
 
wepik-insightful-infographics-a-data-visualization-overview-20240401133220kwr...
wepik-insightful-infographics-a-data-visualization-overview-20240401133220kwr...wepik-insightful-infographics-a-data-visualization-overview-20240401133220kwr...
wepik-insightful-infographics-a-data-visualization-overview-20240401133220kwr...KarteekMane1
 
Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 2Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 217djon017
 

Kürzlich hochgeladen (20)

Digital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing worksDigital Marketing Plan, how digital marketing works
Digital Marketing Plan, how digital marketing works
 
Insurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis ProjectInsurance Churn Prediction Data Analysis Project
Insurance Churn Prediction Data Analysis Project
 
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming PipelinesConf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
Conf42-LLM_Adding Generative AI to Real-Time Streaming Pipelines
 
Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...Student profile product demonstration on grades, ability, well-being and mind...
Student profile product demonstration on grades, ability, well-being and mind...
 
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
Data Analysis Project : Targeting the Right Customers, Presentation on Bank M...
 
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
Data Analysis Project Presentation: Unveiling Your Ideal Customer, Bank Custo...
 
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptxThe Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
The Power of Data-Driven Storytelling_ Unveiling the Layers of Insight.pptx
 
What To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptxWhat To Do For World Nature Conservation Day by Slidesgo.pptx
What To Do For World Nature Conservation Day by Slidesgo.pptx
 
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
FAIR, FAIRsharing, FAIR Cookbook and ELIXIR - Sansone SA - Boston 2024
 
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdfEnglish-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
English-8-Q4-W3-Synthesizing-Essential-Information-From-Various-Sources-1.pdf
 
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
 
Networking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptxNetworking Case Study prepared by teacher.pptx
Networking Case Study prepared by teacher.pptx
 
Advanced Machine Learning for Business Professionals
Advanced Machine Learning for Business ProfessionalsAdvanced Machine Learning for Business Professionals
Advanced Machine Learning for Business Professionals
 
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default  Presentation : Data Analysis Project PPTPredictive Analysis for Loan Default  Presentation : Data Analysis Project PPT
Predictive Analysis for Loan Default Presentation : Data Analysis Project PPT
 
SMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptxSMOTE and K-Fold Cross Validation-Presentation.pptx
SMOTE and K-Fold Cross Validation-Presentation.pptx
 
INTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processingINTRODUCTION TO Natural language processing
INTRODUCTION TO Natural language processing
 
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
Decoding the Heart: Student Presentation on Heart Attack Prediction with Data...
 
Data Analysis Project: Stroke Prediction
Data Analysis Project: Stroke PredictionData Analysis Project: Stroke Prediction
Data Analysis Project: Stroke Prediction
 
wepik-insightful-infographics-a-data-visualization-overview-20240401133220kwr...
wepik-insightful-infographics-a-data-visualization-overview-20240401133220kwr...wepik-insightful-infographics-a-data-visualization-overview-20240401133220kwr...
wepik-insightful-infographics-a-data-visualization-overview-20240401133220kwr...
 
Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 2Easter Eggs From Star Wars and in cars 1 and 2
Easter Eggs From Star Wars and in cars 1 and 2
 

Fossasia ai-ml technologies and application for product development-chetan khatri

  • 1. AI / ML Technologies and Application for Product Development Chetan Khatri FOSSASIA Summit, 2018 @khatri_chetan
  • 2. Accionlabs Artificial Intelligence | Machine Learning WHO AM I ? Lead - Data Science, Technology Evangelist @ Accion labs India Pvt. Ltd. Committer @ Apache Spark, Apache HBase, Elixir Lang. Co-Authored University Curriculum @ University of Kachchh. Software Engineering @: Nazara Games, Eccella Corporation. M.Sc. - Computer Science from University of Kachchh.
  • 3. Accionlabs Artificial Intelligence | Machine Learning What is Artificial Intelligence ? “Artificial intelligence is intelligence demonstrated by machines, in contrast to the natural intelligence displayed by humans and other animals”. - Wikipedia The Original definition of A.I: “Every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made it simulate it. An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves”. - John McCarthy at Dartmouth Conference. 1955 AI is new electricity!
  • 4. Accionlabs Artificial Intelligence | Machine Learning “Software is eating the world, and A.I is eating software !” - GPUs / TPUs are eating Linear Algebra. - Linear Algebra is eating Deep Learning. - Deep Learning is eating Machine Learning. - Machine Learning is eating Artificial Intelligence (AI). - AI is eating Software. - Software is eating the world.
  • 5. Accionlabs Artificial Intelligence | Machine Learning Machine learning It is a subfield of AI concerned with algorithms that allow computer to learn from examples/data and experience. Machine Learning Machine Learning Supervised Learning Unsupervised Learning Reinforcement Learning Deep Neural Network / Deep learning
  • 6. Accionlabs Artificial Intelligence | Machine Learning How deep learning is different ? Data Feature engineering Statistical Machine learning Model Features Unseen Sample Data Deep Learning kind of Machine learning Model Automatically identifies features ! Prediction Prediction Unseen Sample
  • 7. Accionlabs Artificial Intelligence | Machine Learning What is an Artificial Neural Network ? BEACH SEA SKY ...
  • 8. Accionlabs Artificial Intelligence | Machine Learning Neural Networks have been around for a while ...
  • 9. Accionlabs Artificial Intelligence | Machine Learning … But then this happened 1 WEB-SCALE DATA Data volumes double every year
  • 10. Accionlabs Artificial Intelligence | Machine Learning … But then this happened 2 WEB-SCALE DATA Massive Adoption of GPU and TPU.
  • 11. Accionlabs Artificial Intelligence | Machine Learning … But then this happened 3 ADVANCED NEURAL NETWORK DESIGNS
  • 12. Accionlabs Artificial Intelligence | Machine Learning Supervised Learning ! A (INPUT) B (RESPONSE) EMAIL SPAM ? (0/1) IMAGE OBJECT(1,...,10 00) AUDIO TEXT ENGLISH FRENCH
  • 13. Accionlabs Artificial Intelligence | Machine Learning Machine learning / AI: Use Cases Regression & Classification Real-time Data Analysis Character Recognition (OCR) ML Workload Automation Recommendation & Personalization Natural Language Processing Computer Vision Data Visualization Information Extraction Deep Neural Network Conversational Dialog Agent (Bots) Speech Recognition Forecasting Predictions Reinforcement Learning Scaling Prediction services Federated Learning
  • 14. Accionlabs Artificial Intelligence | Machine Learning TOOLS & TECHNOLOGIES Languages Data Quality
  • 15. Accionlabs Artificial Intelligence | Machine Learning Predictive Modelling Deep Learning / AI TOOLS & TECHNOLOGIES
  • 16. Accionlabs Artificial Intelligence | Machine Learning Data Visualization Containerized Scheduling TOOLS & TECHNOLOGIES
  • 17. Accionlabs Artificial Intelligence | Machine Learning Scheduling GPU Enabled TOOLS & TECHNOLOGIES
  • 18. Accionlabs Artificial Intelligence | Machine Learning Artificial Intelligence and Machine learning Automating the organization Improving Decision making & Reducing Inefficiencies
  • 19. Accionlabs Artificial Intelligence | Machine Learning Machine learning process Collect and prepare training data Choose and optimize your ML algorithm Setup and manage environments for training Train and tune model (trial and error) Deploy model in production Scale and manage the production environment
  • 20. Accionlabs Artificial Intelligence | Machine Learning High level Architecture BUILD TRAIN Machine learning as Service ALGORITHMS FRAMEWORKS K-Means Clustering Principal Component Analysis Neural Topic Modelling Factorization Machines Linear Learner - Regression Deep Reinforcement Learner Convolutional Neural Network XGBoost Latent Dirichlet Allocation Seq2Seq , LSTM Recurrent Neural Network Linear Learner - Classification TensorFlow PyTorch Torch Caffe2 CNTK Caffe Apache Spark MLlib Apache Flink Setup and manage environments for training Train and tune model (trial and error) Deploy model in production Scale & manage the production environment
  • 21. Accionlabs Artificial Intelligence | Machine Learning Applications of AI : Computer Vision Self-Driving Cars Convolution Neural Network Recurrent Neural network Vision i.e Camera Object recognition and identification Real time, Per pixel Object Segmentation Centimeter-accurate positioning
  • 22. Accionlabs Artificial Intelligence | Machine Learning Applications of AI : Computer Vision Visual Search & Recommendation Convolution Neural Network Recurrent Neural network Camera
  • 23. Accionlabs Artificial Intelligence | Machine Learning Applications of AI : Computer Vision
  • 24. Accionlabs Artificial Intelligence | Machine Learning Applications of AI : Computer Vision Ad / User Click ? (0/1) Whether user will click to adv. or not ? UI Wireframe UI Screen Possible ??? Screenshot Source Code
  • 25. Accionlabs Artificial Intelligence | Machine Learning Applications of AI : Computer Vision Ad / User Click ? (0/1) Whether user will click to adv. or not ? UI Wireframe UI Screen Possible ??? Screenshot Source Code DEMO
  • 26. Accionlabs Artificial Intelligence | Machine Learning Applications of AI : Computer Vision - DEMO UI Wireframe UI Screen
  • 27. Accionlabs Artificial Intelligence | Machine Learning Applications of AI : Computer Vision - DEMO Screenshot Source Code
  • 28. Accionlabs Artificial Intelligence | Machine Learning Natural Language Processing Entities Key Phrases Language Sentiment Topics
  • 29. Accionlabs Artificial Intelligence | Machine Learning Application of AI: Natural Language Processing Business Applications Entity Recognition Sentiment Analysis Content Classification Language Translation, Speech Recognition Conversational Dialog Agent / Chatbots Syntax Analysis (Key Phrases, Language Understanding) Techniques Recurrent Neural Network Latent Dirichlet Allocation (LDA) Neural Topic Modeling Neural Recursive Network / Attention Network Transfer Learning Seq-to-Seq + LSTM
  • 30. Accionlabs Artificial Intelligence | Machine Learning Challenges - Interoperability of Deep Learning frameworks. - Scale Inference at Production. - Distribution and Parallelism of ML / AI Models. - Build reusable components that work well together (across frameworks) - UNIX Philosophy - Train at GPU / CPU - Inference @ Mobile, Raspberry PI, Drone, Streaming Application
  • 31. Accionlabs Artificial Intelligence | Machine Learning Facial Emotion Recognition: DEMO 1. Facial Emotion Recognition Convolution Neural Network Recurrent Neural network Vision i.e Camera
  • 32. Accionlabs Artificial Intelligence | Machine Learning Real time object recognition : DEMO Real time Object recognition Convolution Neural Network Recurrent Neural network Vision i.e Camera
  • 33. Everyone has own choice of Deep Learning Frameworks
  • 34. Research to Production Everyone has own choice of Deep Learning Frameworks Reimplementation takes Weeks or months
  • 35. Accionlabs Artificial Intelligence | Machine Learning Deep Learning Framework Zoo ... Framework backends Vendor and numeric libraries ... O (n2 ) pairs Apple CoreML Nvidia TensorRT Intel/Nervana ngraph Qualcomm SNPE
  • 36. Accionlabs Artificial Intelligence | Machine Learning Open Neural Network Exchange (ONNX) ... Framework backends Vendor and numeric libraries ... Apple CoreML Nvidia TensorRT Intel/Nervana ngraph Qualcomm SNPE Shared model and operator representation From O(n2 ) to O(n) pairs
  • 37. Accionlabs Artificial Intelligence | Machine Learning Train at GPU, Inference at Mobile App ! ... Apple CoreML TensorFlow Light
  • 38. Accionlabs Artificial Intelligence | Machine Learning Thanks ! @khatri_chetan chetan.khatri@accionlabs.com