Real-time applications of predictive models must be able to generate predictions at the rate that transactions are generated. Previously, such applications of models trained using R needed to be converted to other languages like C++ or Java to achieve the required throughput. In this talk, I’ll describe how to use the in-database R processing capabilities of Microsoft R Server to detect fraud in a SQL Server database of loan records at a rate exceeding one million transactions per second. I will also show the process of training the underlying gradient-boosted tree model on a large training set using the out-of-memory algorithms of Microsoft R.

1. Predicting Loan Delinquency
at 1M Transactions per Second
David Smith @revodavid
R Community Lead, Microsoft

2. 2
It looks like you’ve created a
predictive model…
NOW
WHAT?

3. 3
http://hamiltonmusical.wikia.com/wiki/Right_Hand_Man

4. Generating Predictions
Batch Mode
• Create many (millions!) of predictions at once
• Time required proportional to number of predictions
Real Time
• Only a few (maybe only one!) data point available to predict
– There may be multiple requests in a short timeframe
• Latency the key metric here
– Many applications require sub-second latency at endpoint
4

5. Real-Time Operationalization Options
• Rewrite prediction code in some other language
– PMML / C++ / Java / …
• OR, use your R code:
– Deploy as a web service with Microsoft R Server
– Deploy as a stored procedure in SQL Server
5

6. Lending Club Loan Performance Data
• www.lendingclub.com/info/download-data.action
– Feature selection and generation: aka.ms/lendingclub
6
LoanStatNew Description
all_util Balance to credit limit on all trades
annual_inc_joint The combined self-reported annual income provided by the co-borrowers during
registration
dti_joint A ratio calculated using the co-borrowers' total monthly payments on the total debt
obligations, excluding mortgages and the requested LC loan, divided by the co-
borrowers' combined self-reported monthly income
int_rate Interest Rate on the loan
mths_since_last_record The number of months since the last public record.
revol_util Revolving line utilization rate, or the amount of credit the borrower is using relative
to all available revolving credit.
total_rec_prncp Principal received to date
is_bad (generated) Late > 16 days, Default, or Charged Off

7. Operationalization with Microsoft R Server
Data Scientist
Developer
Integration
Swagger API Service
Consume with any
programming language
Deployment
Publish R function into
web services
Configuration
 Data Science Virtual Machine
 Azure GS5 Instance
 32 cores
 448Gb RAM
Microsoft R Server
configured for
operationalizing R analytics
Microsoft R Client
(mrsdeploy package)
Quant
Consumption
Explore and consume
services in R directly
publishServiceMicrosoft R Client
(mrsdeploy package)
IT Administator

8. Flexible vs Real-Time Deployment
Flexible Deployment
Publish R as Web Service
• Any R function or package
• R interpreter runs on-demand in
Swagger via REST API
Real-Time Deployment
Publish R model object
• RevoScaleR or MicrosoftML models
• Prediction engine generates scores
from data via REST API
8
library(mrsdeploy)
publishService(
serviceType='Script',
Code=<<R script or function>>)
library(mrsdeploy)
publishService(
serviceType='RealTime',
model=<<R object>>)

9. Real-Time Deployment Models
Linear Regression (rxLinMod, rxFastLinear)
Logistic Regression (rxLogit, rxLogisticRegression)
Classification / Regression trees (rxDTree, rxFastTrees)
Classification / Regression forests (rxDForest, rxFastForest)
Stochastic gradient-boosted decision trees (rxBTrees)
One-class Support Vector Machines (rxOneClassSvm)
Convolutional Neural Networks (rxNeuralNet)
Also: pre-trained models for text sentiment and image featurization
9

10. FLEXIBLE AND REAL-TIME SCORING WITH
MICROSOFT R SERVER
Demonstration
Server: Azure Data Science Virtual Machine, Azure GS5 instance (32 cores, 448 GB memory)
Client: SurfaceBook / Microsoft R Client
10

11. 11

12. 12

13. 13

14. 14

15. 15

16. Flexible vs Real-Time
Performance Comparison
Server: Standard_D3_v2 (4 CPU core, 14GB RAM), Windows
16
Algos Real time
(ms)
Flexible (ms)
RxLogit
(model size 2K) 3.5 39.2
RxNeuralNet
(model size 8K) 2.5 122.0
Model Size Real time
(ms)
Flexible
(ms)
2 MB
(RxLogisticRegression)
5.0 9215.7
43 MB
(RxLogisticRegression)
5.4 20255.6

17. sp_execute_external_script
Flexible
Deployment in SQL Server 2016
17
SQL
SERVER
2016
Microsoft R Client
(RevoScaleR package)
rxSerializeObject
sp_rxPredict
Real-Time

18. 20
blog.revolutionanalytics.com/2016/09/fraud-detection.html
SQL Server 2017
8 sockets, 192 cores
6 TB RAM
Flexible operationalization
Flexible vs Real-Time
1M predictions/sec
Same benchmark
One-sixth the resources

19. Operationalization Overview
Platform Flexible Operationalization
• Any R Function / Package
Real-Time Operationalization
• Specific RevoScaleR / MicrosoftML models
SQL Server EXEC sp_execute_external_script
@language = N'R',
@script = N'<<R script>>'
EXEC sp_rxPredict
@model=<<serialized R object>>
@inputData=<<SQL query>>
Microsoft R
Server
library(mrsdeploy)
publishService(
serviceType='Script',
Code=<<R script or function>>)
library(mrsdeploy)
publishService(
serviceType='RealTime',
model=<<R object>>)
21
• Use Microsoft R Server 9+ or SQL Server 2016+ as the deployment server
• Flexible Operationalization supports any R code / package
• Real-Time Operationalization supports Microsoft R models with improved latency

20. Thank You!
David Smith @revodavid
R Community Lead, Microsoft
Special thanks:
Pratik Palnitkar, Microsoft
Arun Gurunathan, Microsoft
Download Microsoft R Client: aka.ms/rclient
Data Science Virtual Machine: aka.ms/dsvm