This document summarizes the typical layers of a machine learning application: the store layer collects and stores data from various sources in a big data store; the model layer uses machine learning algorithms to analyze the data, develop models for prediction, and test the models; the service layer exposes the models as APIs for external applications to consume and make predictions with new data.
2. APIs
Classification
KNN
…
ANN
DCT
Regression
Linear
ANN
Decisi
on
tree
Rando
m
Ensemble
..
…
..
..
Layer Architecture for Machine Learning Application
Logs
Store Layer Model Layer
Consumer Apps
Events
Audits
Reference data
External
Historical Data
Trainingdata
model
Testingdata
model
Pre-
train
model
ExploratoryAnalysisandVisualization
Service Layer
Web
Dashboard
Desktop
others
Algorithm tweaking
Data tweaking
Typically Machine learning application is consist of SMS layers - Store , Model and Service Layer as illustrated in below diagram
3. Store Layer
This layer is a pivotal part of the layer
architecture. It collect the data from different
applications / system / devices / sensors in
clustered environment. Each node has data lake
for specific business aspect. The detailed is
illustrated in the diagram -1. This layer is consist
of following key components.
Data Generation
Enterprise have different source to generate
data. It could be business apps, devices used for
business purpose , sensors and other systems like
- infra structure , communication or other
hardware system. They also get data from
business partners, franchises or agent that play a
vital role in business decisions.
Big Data Store
Enterprise need to store aforesaid generated
data in big store for analytic and machine
learning. The data could be structure or
unstructured format .
Apps
Device
Sensors
Social
Sites
Data Lake
Azure Streaming
Azure Data Factory
Azure Event hubs
CosmosDB
Blob
SQL Data Warehoues
Data Generation
Information Management Big Data Store
Systems
1
4. Model Layer
This layer is more about experiment and
observation for accurate perdition. Modell is
about the preparing training and testing data set
with combination of machine learning algorithms
to predict future value. There are several
possible options for model construct.
• Use pre-build model provided by cognitive
platform
• Develop specific model for particular
objective using data science language like
Python or R etc. This is most critical and
cumbersome activity that helps to identify
best fit machine learning algorithm for a
business case and keep varying with different
use cases.
• Or combination of both
Data
Sets
New
data
Sets
Machine Learning Cognitive Services
Analytics
1
2
5. Service Layer
Expose the data model as a service to consume outside. This service is accessible externally from different types of
applications to predict model with new data set.
Models
API
Gateway
Service
Service
Service
Service
Consumer
Apps
Service
2
Security
New
Data
set