Machine learning is a method of data analysis that automates the building of analytical models. By using algorithms that iteratively learn from data, computers are able to find hidden insights without the help of explicit programming. These insights bring tremendous benefits into many different domains. For business users, in particular, these insights help organizations improve customer experience, become more competitive, and respond much faster to opportunities or threats. The availability of very powerful in-memory computing platforms, such as Apache Ignite, means that more organizations can benefit from machine learning today. In this presentation, we will discuss how the Compute Grid, Data Grid, and Machine Learning Grid components of Apache Ignite work together to enable your business to start reaping the benefits of machine learning. Through examples, attendees will learn how Apache Ignite can be used for data analysis and be the in-memory hammer in your machine learning toolkit.

1. © 2017 GridGain Systems, Inc.
In-Memory Hammer for Your Data Science Toolkit
Apache Ignite
Denis Magda
Ignite PMC Chair
Apache Ignite PM

2. © 2017 GridGain Systems, Inc.
• Apache Ignite Overview
• Use Cases
• Data Science Toolkit Box
• Data Grid
• Durable Memory
• Distributed SQL
• Compute Grid
• Machine Learning Grid (Beta)
• Q&A
Agenda

3. © 2017 GridGain Systems, Inc.
Apache Ignite In-Memory Computing Platform
Memory-Centric Storage
Ignite Native Persistence
(Flash, SSD, Intel 3D XPoint)
Third-Party Persistence
(RDBMS, HDFS, NoSQL)
SQL Transactions Compute Services MLStreaming
Applications
Key/Value
IoTFinancial
Services
Pharma &
Healthcare
E-CommerceTravel &
Logistics
Telco

4. © 2017 GridGain Systems, Inc.
Apache Ignite Use Cases
FinTech
Financial Services Software Logistics & Travel
E-commerce
Telco
IoT
Pharma & Healthcare
Adtech

5. © 2017 GridGain Systems, Inc.
e-Therapeutics provides a computer-based drug
discovery platform and a specialized approach to
network biology.
Problem
• Analysis of a network of proteins influencing a disease and
drugs discovery could be measured in weeks
• Could not parallelize existing algorithms
Apache Ignite Solution
• 80x speed increase over the non-parallelized environment
• Analysis projects completion in hours and minutes
• Computational resources for abandoned research projects
- Drug Discovery and Network Biology
Cache &
Compute
API
e-Therapeutics
Platform
100x Cluster Nodes
5x Physical Nodes
Server Nodes
Clients
Nodes

6. © 2017 GridGain Systems, Inc.
Data Grid
JCache Transactions Compute SQL
Server Node
Distributed Key-Value Store
Dynamic
Scaling
Distributed
partitioned
hash map
ACID TransactionJCache & SQL
Server Node Server Node
DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY
RDBMS
NoSQL
HDFS
3rd party storage caching

7. © 2017 GridGain Systems, Inc.
Durable Memory
Off-heap Removes
noticeable GC pauses
Automatic
Defragmentation
Stores Superset
of Data
Predictable memory
consumption
Fully Transactional
(Write-Ahead Log)
DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY
Server Node Server Node Server Node
Ignite Cluster

8. © 2017 GridGain Systems, Inc.
Ignite Native Persistence
1. Update
RAM
2. Persist
Write-Ahead Log
Partition File 1
2. Ack
4. Checkpointing
Partition File N
Server Node

9. © 2017 GridGain Systems, Inc.
Distributed SQL
JDBC ODBC SQL API
Java .NET C++ BI
SELECT, UPDATE,
INSERT, MERGE,
DELETE, CREATE
and ALTER
DDL, DML Support
Cross-platform
Compatibility
Indexes in
RAM or Disk
Dynamic
Scaling
Server Node Server NodeServer Node
Apache Ignite Cluster
DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY
Tools

10. © 2017 GridGain Systems, Inc.
1. Initial Query
2. Query execution over local data
3. Reduce multiple results in one
1. Initial Query
2. Query execution (local + remote data)
3. Potential data movement
4. Reduce multiple results in one
2
2
1
Collocated Joins Non-Collocated Joins
Server Node
ON-DISK
Server Node
ON-DISK
Client Node
3
2
2
1
Server Node
ON-DISK
Server Node
ON-DISK
Client Node
4
3

11. © 2017 GridGain Systems, Inc.
Compute Grid
DURABLE MEMORY
DURABLE MEMORY
Ignite Cluster
C1
R1
C2
R2
C = C1 + C2
R = R1 + R2
C = Compute
R = Result
in T/2 time
Automatic Failover
Load Balancing
Zero Deployment

12. © 2017 GridGain Systems, Inc.
1. Initial Request
2. Fetch data from remote nodes
3. Process entire data-set
1. Initial Request
2. Co-located processing with data
3. Reduce multiple results in one
3
1
Data 1
2
2 Data 2
2
2
1Client Node
Client-Server Processing Co-located Processing
Server Node
ON-DISK
Server Node
ON-DISK
Server Node
ON-DISK
Server Node
ON-DISK
Client Node
3

13. © 2017 GridGain Systems, Inc.
Genetic Algorithms Grid
DURABLE MEMORY
DURABLE MEMORY
Ignite Cluster
F2, C2, M2
F = F1 + F2
C = C1 + C2
Collocated Computation
Biological Evolution
Simulation
Chromosome and Genes Cluster
M = M1 + M2
F1, C1, M1
F = Fitness Calculation
C = Crossover
M = Mutation

14. © 2017 GridGain Systems, Inc.
Machine Learning Grid
K-Means Regressions Decision Trees
R C++ Python Java
Server Node Server NodeServer Node
Distributed Core Algebra
DURABLE MEMORY DURABLE MEMORY DURABLE MEMORY
Scala REST
Random ForestDistributed Algorithms
Dense and Sparse
Algebra
Large scale
parallelization
Multi-Language
Support
No ETL

15. © 2017 GridGain Systems, Inc.
Thank you for joining us. Follow the conversation.
http://ignite.apache.org
Any Questions?
#apacheignite
#denismagda