Machine Learning Use Cases in Finance (presentation at the Azure Machine Learning Hackathon 2015). Source & additional information: http://0xcode.in/azureml-hackathon-2015

1. Machine Learning in Finance
Dmitry Petukhov,
Researcher & Developer @ OpenWay
#AzureMLHackathon

2. Machine Learning in Finance. Machine Learning in our Life
Web
Social
Networks
Science
Healthcare
Finance
Telecom
Retail
Logistic
Security
Financial Markets
Investment Banking
Retail Banking
Insurance
Electronics
Proof: https://www.kaggle.com/wiki/DataScienceUseCases

3. Machine Learning in Finance. Machine Learning Use Cases in Finance
Financial Markets & etc. Retail Banking Insurance
Real-time Batch processingDuration
Market
Assets Price
Prediction
Social
Network
Analysis
Fraud
Detection
Risk Analysis
Compliance &
Regulatory
Reporting
Advertising
Campaign
Optimization
News
Analysis
Customer
Loyalty &
Marketing
Improving
operational
efficiencies
Credit
Scoring
Brand
Sentiment
Analysis
Personalized
Product
Offering
Customer
Segmentation
Reference: http://0xcode.in/big-data-in-banking

4. Machine Learning in Finance. Need & Opportunity
The Need for Banks.
The Opportunity for You!

5. Machine Learning in Finance. Opportunities for all
For FinTech Startups…
FinTech Incubators & Accelerators
Startupbootcamp Fintech
AlfaCamp
Barclays Accelerator
MasterCard Start Path
Visa Europe Collab
QIWI Universe 2.0
InspirAsia (Life.SREDA)
Future Fintech
For Researchers & Enthusiasts…
Competitions & Hackathons
Sberbank
Alfabank
Tinkoff
Beeline
…
Rusbase

6. Machine Learning in Finance. Concepts
Data
Hadoop 1.0
Infrastructure
ApplicationsClosed
Expensive
Big Volume,
Legal restrictions

7. Machine Learning in Finance. Implementation
Azure Storage, NoSQL & RDBS storage as a Service
Hadoop 1.0
IaaS: Azure VM, Cloudera/Hortonworks
PaaS: HDInsight, Web/Worker role, Azure Batch
Intelligent Systems

8. 1. Retrieve data / 4. Consume prediction
Azure
Transactions Log
Raw Transactions Data
Processing System
(Gate)
DMZ
Raw Transactions Data
Azure Blob Storage
Transactions Processing
Jobs Queue
Azure Service Bus
Transactions
Processing Nodes
Azure Worker Roles
Calculated Data Storage
NoSQL Storage
Prediction System
HDInsight
Fraud Prediction
Azure Machine Learning
Commands flow Data flow
Auxiliary
Services
CRMs Data
Transactions Batch Processing System
Machine Learning in Finance. Antifraud: Architecture
Bank
Antifraud System
h(θ0, θn)
Fraud prediction API
Azure ML Web Services
POST, https
REST API
JSON
Final Model
2. Pre-processing data 3. Create prediction model
Source: http://0xcode.in/antifraud-insights

9. Machine Learning in Finance. Antifraud: retrieve data
External Services: geolocation, currency exchange rate,
etc.
Support Service Data
Social Network & News aggregator
Plastic Card, Accounts, Merchants, IP-hosts, etc.
black/white lists
Number of customer grows fast…
Number of operations grows even faster…
Transactions Log
with request information
Banking CRM Data
Merchant CRM Data
Web-clicks Stream
Web/mobile-applications & Backend services Log
Data for Model
Join data
Problems

10. Machine Learning in Finance. Antifraud: processing data
Quality

11. Storage
Resource
Management
ML Framework
Execution
Engine
Local OS
Local Disc
PythonRuntime
YetAnother
Runtime
scikit
learn
HDFS
YARN
MapReduce
Mahout
HDFS / S3
YARN /
Apache Mesos
Spark
MLlib
HDFS / S3
YARN /
Apache Mesos
Python / R
on Spark
Python / R
tools
Spark
Local PC Hybrid Model Cluster (on-premises/on-demand)
some
library
Machine Learning in Finance. Infrastructure for Data Scientist
Low HighCost of deployment/ownership
Distributed
FS
Dark
Magic…
ML as a Service
Python / R
tools

12. Machine Learning in Finance. Antifraud: Create model
Demo

13. References
Machine Learning in Finance. References
Start for free from azure.com/ml
Read Microsoft Machine Learning Blog
Examine Azure ML documentation +free books
Take free MOOCs on MVA & EdX
Communicate on «Microsoft Azure Russia» group
Make the world better place with Azure for Researchers Award program

14. © 2015 Dmitry Petukhov All rights reserved. Microsoft Azure and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
Thank you!

15. Q&A
Now or later (send on email)
Ping me
Habr: @codezombie
Facebook: @code.zombi
LinkedIn: @dpetukhov
Read my tech code instinct blog (on http://0xCode.in/)
Machine Learning in Finance. Stay connected!
Download presentation from
http://0xcode.in/azureml-hackathon-2015 or