This workshop will look into ways to create synthetic data from lending club loan record datasets alongside comparing characteristics and statistical properties of real and synthetic datasets. There will also be discussions into building machine learning models for predicting interest rates using real and synthetic datasets and evaluating the performance and discuss the advantages and disadvantages of using synthetic datasets as a proxy for real datasets

1. Synthetic Data Generation for Machine Learning in Finance
2020 Copyright QuantUniversity LLC.
Presented By:
Sri Krishnamurthy, CFA, CAP
QuantUniversity
9/25/2020
Powered by:

2. 2
Speaker
• Quant, Data Science & ML practitioner
• Prior Experience at MathWorks, Citigroup
and Endeca and 25+ financial services and
energy customers.
• Columnist for the Wilmott Magazine
• Author of forthcoming book
“Pragmatic Machine Learning in Finance ”
• Teaches Data Science/AI at Northeastern
University, Boston
• Reviewer: Journal of Asset Management
Sri Krishnamurthy
Founder and CEO
QuantUniversity

3. 3
About QuantUniversity
• Boston-based Data Science, Quant
Finance and Machine Learning
training and consulting advisory
• Trained more than 1000 students in
Quantitative methods, Data Science,
ML and Big Data Technologies
• Building a platform for
operationalizing AI and Machine
Learning in the Enterprise

4. 4
1. Challenges with Real Datasets
2. Synthetic Dataset generation tools
▫ Proprietary
▫ Open Source
– Faker
– Data Synthesizer
– SDV
– Synthpop
– GANs
3. Demos
▫ VIX Data Generator
Agenda

5. Challenges with Real Datasets

6. 6

7. 7
Not be feasible to get samples for all categories
• Lighting conditions
• Modifications (Glasses/No glasses,
Moustache/ No Moustache etc.)
• Positions
Coverage
Challenges with real datasets

8. 8
All scenarios haven’t
played out
• Stress scenarios
• What-if scenarios
Challenges with real datasets
Figure ref: http://www.actuaries.org/CTTEES_SOLV/Documents/StressTestingPaper.pdf

9. 9
Missing values
• Missing at random
• Missing sequences
• Need data to fill frames
Challenges with real datasets

10. 10
• Access
▫ Hard to find
▫ Rare class problems
▫ Privacy concerns
making it difficult to
share
Challenges with real datasets

11. 11
Imbalanced
• Need more samples of rare
class
• Need proxies for data points
that were not observed or
recorded
Challenges with real datasets

12. 12
Labels
• Human labeling is hard
• Synthetic label generators
Challenges with real datasets

13. Tools for Synthetic Data Generation

14. 14
Proprietary Tools
Company Core Technology
Tonic.ai
All-in-one platform for data anonymization, subsetting, and synthesis
integrated with databases (hadoop, oracle, mysql, MS sql server, mongo
db, amazon aurora/redshift, and google big query)
- Uses Condenser and Masquerade
Mostly.ai
Tablular data using generative deep neural networks (no image data)
CVEDIA
- Sensor modeling and algorithm training
- Handle image using SynCity as a custom pocket laboratory to generate
highly entropic scenes, conditions, and metadata. Enable real-time
Hardware-In-the-Loop (HWIL), Human-In-the-Loop (HITL) or Software-In-
the-Loop (SIL) simulations even with complex sensor configurations
Deep vision data
Image creation
Synthetic training data
Synthesis.ai The data generation platform for computer vision

15. 15
López de Prado, Marcos, Machine Learning for Asset Managers,
Cambridge University Press 2020

16. 16
Opensource tools

17. 17
SDV
https://www.computer.org/csdl/proceedings-
article/dsaa/2016/07796926/12OmNwx3Q7S

18. 18
Data Synthesizer
https://faculty.washington.edu/billhowe/publications/pdfs/pin
g17datasynthesizer.pdf

19. 19
Synthpop

20. 20
VAE
https://arxiv.org/pdf/1808.06444.pdf

21. 21
GAN
https://developers.google.com/machine-
learning/gan/gan_structure

22. 22
WGAN

23. 23

24. 24

25. 25

26. 26

28. 28

29. 29
Demo 1 – Loan Data Synthesizer

30. 30
Demo 2: Synthetic Sales data generation

31. 31
Demo 3 : Synthetic VIX generation

32. 32
If you want to be a part of QuSandbox private Beta
Contact us:
info@qusandbox

33. Sri Krishnamurthy, CFA, CAP
Founder and Chief Data Scientist
sri@quantuniversity.com
srikrishnamurthy
www.qu.academy
Information, data and drawings embodied in this presentation are strictly a property of QuantUniversity LLC. and shall not be
distributed or used in any other publication without the prior written consent of QuantUniversity LLC.
33