This session was presented at the CFA Institute on May 6th 2020
This deep-dive session discusses core methods and applications to provide an understanding of supervised and unsupervised machine learning. Participants will be introduced to advanced topics that include time series analysis, reinforcement learning, anomaly detection, and natural language processing. Case studies will also examine how to predict interest rates and credit risk with alternative data sets and how to analyze earning calls from EDGAR using Natural Language Processing Techniques.
Aspirational Block Program Block Syaldey District - Almora
Machine Learning and AI: Core Methods and Applications
1. MACHINE LEARNING AND AI:
CORE METHODS AND
APPLICATIONS
06 May 2020, 9:00 am - 11:00 am EDT
Sri Krishnamurthy, CFA
President
QuantUniversity
Richard Fernand, Moderator
Senior Director, Global Content, Professional Learning
CFA Institute
2. Use Q&A to submit questions for the presenters
Use CHAT to share comments and to see
what others are saying. You can also select
who you would like to send the message to
by clicking on the drop down next to To:.
3. Machine Learning and AI
Core methods and Applications
Part 2
2020 Copyright QuantUniversity LLC.
Presented By:
Sri Krishnamurthy, CFA, CAP
sri@quantuniversity.com
www.quantuniversity.com
05/06/2020
CFA Institute - Online
4. 4
Speaker bio
• Advisory and Consultancy for Financial
Analytics
• Prior Experience at MathWorks, Citigroup
and Endeca and 25+ financial services and
energy customers.
• Columnist for the Wilmott Magazine
• Author of forthcoming book
“The Model-Driven Enterprise”
• Teaches AI/ML and Fintech Related topics in
the MS and MBA programs at Northeastern
University, Boston
• Reviewer: Journal of Asset Management
Sri Krishnamurthy
Founder and CEO
QuantUniversity
5. 5
QuantUniversity
• Boston-based Data Science, Quant
Finance and Machine Learning
training and consulting advisory
• Trained more than 1000 students in
Quantitative methods, Data Science
and Big Data Technologies using
MATLAB, Python and R
• Building a platform for AI
and Machine Learning Exploration
and Experimentation
6. 1. Key trends in AI, Machine Learning & Fintech
2. An intuitive introduction to AI and ML
3. Case study
▫ Alternative investments: Interest rate predication for Peer-to-Peer
Market places using ML techniques
▫ Scenario analysis: Synthetic VIX data generation using Neural
Networks
Recap from Part 1
7. Machine Learning Workflow
Data Scraping/
Ingestion
Data
Exploration
Data Cleansing
and Processing
Feature
Engineering
Model
Evaluation
& Tuning
Model
Selection
Model
Deployment/
Inference
Supervised
Unsupervised
Modeling
Data Engineer, Dev Ops Engineer
Data Scientist/QuantsSoftware/Web Engineer
• AutoML
• Model Validation
• Interpretability
Robotic Process Automation (RPA) (Microservices, Pipelines )
• SW: Web/ Rest API
• HW: GPU, Cloud
• Monitoring
• Regression
• KNN
• Decision Trees
• Naive Bayes
• Neural Networks
• Ensembles
• Clustering
• PCA
• Autoencoder
• RMS
• MAPS
• MAE
• Confusion Matrix
• Precision/Recall
• ROC
• Hyper-parameter
tuning
• Parameter Grids
Risk Management/ Compliance(All stages)
Analysts&
DecisionMakers
12. 12
Demos, slides and video available on QuAcademy
Go to https://academy.qusandbox.com
Use code ‘CFAMasterclass’ for access
12
13.
14. • Machine learning
▫ Unsupervised learning – Clustering etc.
▫ Supervised machine learning - Classification
• Case studies
▫ Investments: Using Clustering for investment decision-support
▫ NLP: Building your own Sentiment Analysis Engine for EDGAR filings
• Frontier topics
▫ Anomaly detection
▫ Natural Language Processing
▫ Deep learning
▫ Risk in Machine Learning and AI
▫ Model governance, Interpretability and Model Management
Part 2: Machine Learning and AI :Core methods and
applications
15. Machine Learning Workflow
Data Scraping/
Ingestion
Data
Exploration
Data Cleansing
and Processing
Feature
Engineering
Model
Evaluation
& Tuning
Model
Selection
Model
Deployment/
Inference
Supervised
Unsupervised
Modeling
Data Engineer, Dev Ops Engineer
Data Scientist/QuantsSoftware/Web Engineer
• AutoML
• Model Validation
• Interpretability
Robotic Process Automation (RPA) (Microservices, Pipelines )
• SW: Web/ Rest API
• HW: GPU, Cloud
• Monitoring
• Regression
• KNN
• Decision Trees
• Naive Bayes
• Neural Networks
• Ensembles
• Clustering
• PCA
• Autoencoder
• RMS
• MAPS
• MAE
• Confusion Matrix
• Precision/Recall
• ROC
• Hyper-parameter
tuning
• Parameter Grids
Risk Management/ Compliance(All stages)
Analysts&
DecisionMakers
19. 19
Supervised Algorithms
▫ Given a set of variables 𝑥!, predict the value of another variable 𝑦 in
a given data set such that
▫ If y is numeric => Prediction
▫ If y is categorical => Classification
▫ Example: Given a review, is the author’s sentiment good or bad?
Machine Learning
19
x1,x2,x3… Model F(X) y
20. 20
• Parametric models
▫ Assume some functional form
▫ Fit coefficients
• Examples : Logistic Regression, Neural Networks
Supervised Learning models - Classification
20
Logistic Regression Model Neural network Model
21. 21
• Non-Parametric models
▫ No functional form assumed
• Examples : K-nearest Neighbors, Decision Trees
Supervised Learning models
21
K-nearest neighbor Model Decision tree Model
22. 22
Unsupervised Algorithms
▫ Given a dataset with variables 𝑥!, build a model that captures the
similarities in different observations and assigns them to different
buckets => Clustering
▫ Example: Given a list of emerging market stocks, can we segment them
into three buckets?
Machine Learning
22
Obs1,
Obs2,Obs3
etc.
Model
Obs1- Class 1
Obs2- Class 2
Obs3- Class 1
23. 23
• These methods partition the data into k clusters by assigning each data point to its
closest cluster centroid by minimizing the within-cluster sum of squares (WSS), which
is:
!
!"#
$
!
%∈'!
!
("#
)
(𝑥%( − 𝜇!()*
where 𝑆! is the set of observations in the kth cluster and 𝜇!( is the mean of jth
variable of the cluster center of the kth cluster.
• Then, they select the top n points that are the farthest away from their nearest
cluster centers as outliers.
K-means clustering
23
26. 26
Hierarchical Clustering
• Agglomerative: This is a "bottom-
up" approach: each observation
starts in its own cluster, and pairs
of clusters are merged as one
moves up the hierarchy.
• Divisive: This is a "top-down"
approach: all observations start in
one cluster, and splits are
performed recursively as one
moves down the hierarchy.
Source:
https://en.wikipedia.org/wiki/Hierarchical_cluster
ing
27. 27
Affinity propogation
• Affinity propagation (AP) is a clustering algorithm based on
the concept of "message passing" between data points
• Affinity propagation finds "exemplars," members of the
input set that are representative of clusters
• Affinity propagation does not require the number of clusters
to be determined or estimated before running the algorithm
• Source: https://en.wikipedia.org/wiki/Affinity_propagation
28. 28
Unsupervised Algorithms
▫ PCA->
– Dimension reduction using orthogonal linear transformations
▫ Manifold Learning ->
– Non-linear dimension reduction
▫ Association Rule Mining ->
– People who bought X also bought Y
▫ https://en.wikipedia.org/wiki/Principal_component_analysis
Machine Learning
28
36. 36
• Consider a two-class case with classes 𝐶! and 𝐶"
• Classification matrix:
Classification matrix
Predicted Class
Actual Class 𝐶! 𝐶"
𝐶!
𝑛!,!= number of 𝐶! cases
classified correctly
𝑛!,"= number of 𝐶! cases
classified incorrectly as 𝐶"
𝐶"
𝑛",!= number of 𝐶" cases
classified incorrectly as 𝐶!
𝑛","= number of 𝐶" cases
classified correctly
38. 38
• The ROC curve plots the pairs {sensitivity, 1-
specificity} as the cutoff value increases from 0
and 1
• Sensitivity (also called the true positive rate, or
recall in some fields) measures the proportion of
positives that are correctly identified
▫ e.g., the percentage of COVID19 diagnosed people who
are correctly identified as having COVID19
• Specificity (also called the true negative rate)
measures the proportion of negatives that are
correctly identified as such.
▫ e.g., the percentage of healthy people who are correctly
identified as not having COVID19
• Better performance is reflected by curves that are
closer to the top left corner
ROC Curve
39. 39
1. Data
2. Goals
3. Machine learning algorithms
4. Process
5. Performance Evaluation
Recap
40. Machine Learning Workflow
Data Scraping/
Ingestion
Data
Exploration
Data Cleansing
and Processing
Feature
Engineering
Model
Evaluation
& Tuning
Model
Selection
Model
Deployment/
Inference
Supervised
Unsupervised
Modeling
Data Engineer, Dev Ops Engineer
Data Scientist/QuantsSoftware/Web Engineer
• AutoML
• Model Validation
• Interpretability
Robotic Process Automation (RPA) (Microservices, Pipelines )
• SW: Web/ Rest API
• HW: GPU, Cloud
• Monitoring
• Regression
• KNN
• Decision Trees
• Naive Bayes
• Neural Networks
• Ensembles
• Clustering
• PCA
• Autoencoder
• RMS
• MAPS
• MAE
• Confusion Matrix
• Precision/Recall
• ROC
• Hyper-parameter
tuning
• Parameter Grids
Risk Management/ Compliance(All stages)
Analysts&
DecisionMakers
45. 45
Clustering stocks
• Which stocks are like each other?
• Are growth stocks behaving like growth stocks or value
stocks?
• Does the time series of prices & returns reveal which
stocks are close to each other?
63. 63
• If computers can understand language, opens huge possibilities
▫ Read and summarize
▫ Translate
▫ Describe what’s happening
▫ Understand commands
▫ Answer questions
▫ Respond in plain language
Language allows understanding
64. 64
• Describe rules of grammar
• Describe meanings of words and their
relationships
• …including all the special cases
• ...and idioms
• ...and special cases for the idioms
• ...
• ...understand language!
Traditional language AI
https://en.wikipedia.org/wiki/Formal_language
65. 65
What is NLP ?
Jumping NLP Curves
https://ieeexplore.ieee.org/document/6786458/
67. 67
• Ambiguity:
▫ “ground”
▫ “jaguar”
▫ “The car hit the pole while it was moving”
▫ “One morning I shot an elephant in my pajamas. How he got into my
pajamas, I’ll never know.”
▫ “The tank is full of soldiers.”
“The tank is full of nitrogen.”
Language is hard to deal with
69. 69
• Many ways to say the same thing
▫ “the same thing can be said in many ways”
▫ “language is versatile”
▫ “The same words can be arranged in many different ways to express
the same idea”
▫ …
Language is hard to deal with
70. 70
• APIs
• Human Insight
• Expert Knowledge
• Build your own
Options?
77. 77
What is AutoML?
Automated Machine Learning provides methods and processes to
make Machine Learning available for non-Machine Learning experts,
to improve efficiency of Machine Learning and to accelerate research
on Machine Learning.
• Incarnations: DataRobot, H20.ai, Azure ML, Amazon Sagemaker
AutoPilot, Google Automl
• Check https://www.automl.org/automl/ for more details
AutoML
http://www.asimovinstitute.org/neural-network-zoo/
78. 78
• Simple weighting (Crude but still being used!)
• Bayesian averaging
• Gradient Boosted Decision Trees, Random Forest etc.
Ensemble learning
79. 79
• Requires lots of data
• Labeling and annotation is a big problem
• ML as-a-service is the way to go unless very specific domain
expertise and data is available
Natural Language Processing
80. 80
• Fraud Detection
▫ Credit card fraud detection
– By owner or by operation
▫ Mobile phone fraud/anomaly detection
– Calling behavior, volume etc.
▫ Insurance claim fraud detection
– Medical malpractice
– Auto insurance
▫ Insider trading detection
▫ Pricing issues
▫ Network issues
Anomaly Detection
82. 82
• Components that needs to be tracked
WHAT CONSTITUTES AN ML MODEL?
82
• Programming environment
• Execution environment
• Hardware specs
• Cloud
• GPU
• Interdependencies
• Lineage/Provenance of
individual components
• Model params
• Hyper parameters
• Pipeline specifications
• Model specific
• Tests
• Data versions
Data Model
EnvironmentProcess
83. 83
ELEMENTS OF A MACHINE LEARNING SYSTEM
Source: Sculley et al., 2015 "Hidden Technical Debt in Machine Learning Systems"
88. 88
2. The Data questions
1. Do you know what data you need ?
2. Do you know if the data is available?
3. Do you have the data ?
4. Do you have the right data?
5. Will you continue to have the data?
10 Things to Remember
89. 89
3. Develop a data acquisition and data prep strategy
1. Do you know how to get the data ?
2. Who gets the data?
3. How do you process it?
4. How do you access it?
5. How do you version and govern the data?
10 Things to Remember
90. 90
4. Explore and evaluate your data and get it in the right format
10 Things to Remember
91. 91
5. Define your goal:
1. Summarization
2. Fact finding
3. Understanding relationships
4. Prediction
10 Things to Remember
92. 92
6. Shortlist (not “Choose” ) the
techniques/methodologies/algorithms
10 Things to Remember
93. 93
7. Evaluate/establish business constraints and narrow down your
choices of techniques/methodologies/algorithms
1. Cloud/Cost/Expertise/Cost-Value
2. Build/buy/access
10 Things to Remember
Outcomes
Time
Quality
Cost
94. 94
8. Establish criteria to know if the methodology/models/algorithms
work
1. Is the process replicable?
2. What performance metrics do we choose?
3. Can you evaluate the performance and validate if the models meet
the criteria?
4. Does it provide business value?
10 Things to Remember
95. 95
9. Fine tune your algorithms and algorithm selection
1. Hyper parameter tuning
2. Bias-variance tradeoff
3. Handling imbalanced class problems
4. Ensemble techniques
5. AutoML
10 Things to Remember
https://support.sas.com/resources/papers/proceedings17/SAS0514-2017.pdf
96. 96
10. How will this process reach decision makers
1. Deployment choices (On-prem/Cloud)
2. Frequency of data/model updates
3. Governance/Role/Responsibilities
4. Speed, Scale, Availability, Disaster recovery, Rollback, Pull-Plug
10 Things to Remember
97. 97
How do you monitor the efficacy of your solution?
1. Retuning
2. Monitoring
3. Model decay
4. Data augmentation
5. Newer innovations
10+1 Things to Remember
98. Machine Learning Workflow
Data Scraping/
Ingestion
Data
Exploration
Data Cleansing
and Processing
Feature
Engineering
Model
Evaluation
& Tuning
Model
Selection
Model
Deployment/
Inference
Supervised
Unsupervised
Modeling
Data Engineer, Dev Ops Engineer
Data Scientist/QuantsSoftware/Web Engineer
• AutoML
• Model Validation
• Interpretability
Robotic Process Automation (RPA) (Microservices, Pipelines )
• SW: Web/ Rest API
• HW: GPU, Cloud
• Monitoring
• Regression
• KNN
• Decision Trees
• Naive Bayes
• Neural Networks
• Ensembles
• Clustering
• PCA
• Autoencoder
• RMS
• MAPS
• MAE
• Confusion Matrix
• Precision/Recall
• ROC
• Hyper-parameter
tuning
• Parameter Grids
Risk Management/ Compliance(All stages)
Analysts&
DecisionMakers
99.
100. 1. Key trends in AI, Machine Learning & Fintech
2. An intuitive introduction to AI and ML
3. Case study
▫ Alternative investments: Interest rate predication for Peer-to-Peer
Market places using ML techniques
▫ Scenario analysis: Synthetic VIX data generation using Neural
Networks
Part 1
101. • Machine learning
▫ Unsupervised learning – Clustering etc.
▫ Supervised machine learning - Classification
• Case studies
▫ Investments: Using Clustering for investment decision-support
▫ NLP: Building your own Sentiment Analysis Engine for EDGAR filings
• Frontier topics
▫ Anomaly detection
▫ Natural Language Processing
▫ Deep learning
▫ Risk in Machine Learning and AI
▫ Model governance, Interpretability and Model Management
Part 2: Machine Learning and AI :Core methods and
applications
102. 102
Demos, slides and video available on QuAcademy
Go to https://academy.qusandbox.com
Use code ‘CFAMasterclass’ for access
102
106. Thank you!
Sri Krishnamurthy, CFA, CAP
Founder and CEO
QuantUniversity LLC.
srikrishnamurthy
www.QuantUniversity.com
Contact
Information, data and drawings embodied in this presentation are strictly a property of QuantUniversity LLC. and shall not be
distributed or used in any other publication without the prior written consent of QuantUniversity LLC.
106