Joint talk Microsoft-ActiveEon at Cloud Expo Europe, Big Data Analytics and Cloud management theater. Presenters: Christopher Plieger, Microsoft Azure Product Marketing Manager, and Denis Caromel , CEO - ActiveEon
11. Global Locations
Partnerships
Key information
Management
Denis
Caromel, CEO
François
Tournesac, CSO
Brian Amedro,
CTO
Founded in 2007 by Denis Caromel in Sophia-Antipolis, Spin-off of INRIA
Addressing $80 Billion Hybrid Cloud Market with 27% CAGR
Disruptive Patented Technology w/ Exceptional Business Outcomes
60% of the revenue from international
Sophia-Antipolis (France)
Paris (France)
London (United Kingdom)
San-Jose (United States)
Montreal (Canada)
Fribourg (Switzerland)
Dakar (Senegal)
ProActive Solution
Job Scheduling, Workload Automation
Orchestration & Meta-Scheduling
On-premises and on all clouds
Open Source
12. REST APIsApp-specific Interfaces Integrated Web Portals
ProActive Workflows
Scheduling
Orchestration
Meta-scheduling
Resource Allocation
Big Data, Data Science,
Third Party Software
Local
Scheduler
Resource Manager
Fault Tolerance
Cloud bursting
Resource agnostic
Micro-service
Etc.
Workflow Automation
Cloud Data Lake
LSF
Clusters
13.
14. Legal & General
$1,200,000 Azure Computing per Year
• Utilize ActiveEon’s ProActive to distribute
the load
• Burst calculations into Azure
• “Infinite capacity” reduce Time To Results
15. Legal & General
CHALLENGES
Azure Migration
Resource allocation according to CPU and
memory available
Solvency II analysis on 2.5 million Monte Carlo
scenarios
RESULTS
Optimized resource allocation
Algorithm acceleration
Shorter scenario analysis
Scalable solution
• Migration from private datacenter
• Replacing old Tibco Datasynapse
• Replacing old IBM Algobatch
MAIN DRIVER
REQUIREMENTS
• Cloud capable
• Dynamically define and prioritize workloads
• Minimize time to delivery of results
• Save resources
COMPANY PROFILE
• Industry: Insurance
• Product: Compliance algorithm
16. Legal & General
Time (hours)
105 160
1024Workers
Profile Results
Time (hours)
1024Workers
Profile Results
105 140
I/O intensive tasks
Aggregation &
Reporting
Low Priority
CPU-Intensive tasks
Risk Watch
Simulation
Medium Priority
CPU-Intensive tasks
Risk Watch Simulation
High Priority
CPU-Intensive tasks
Risk Watch Simulation
“More Value, Faster”
Full computation without intermediate result High Priority
Results
Full Report
2H 5H
Batch optimization with ProActive:
• Enforce strong priorities
• Optimal compact execution
• Start tasks as early as possible
• Pipeline and co-allocate
• CPU-intensive with I/O intensive
tasks
18H
17. End-to-end execution
from 18h on-prem to 5h on Azure
Legal & General with Azure
Task monitoring with ProActive
Automated grid start on Azure
Life cycle management of Azure nodes
Fault-Tolerance: this host died, and
ProActive rescheduled the tasks executing
on it, and routed Tasks around the faulty
node afterwards
Execution on 1 024 Azure nodes
18. CHALLENGES
Multiple secured environments behind a firewall
Scale to support Big Data needs
Connect to various databases with specific RBAC
RESULTS
Secured communication between environments
Shared resources for maximum performances
• Allocation of resources across environments
• Respect RBAC during compute time
MAIN DRIVER
REQUIREMENTS
• Support use of Docker
• Support Hadoop, Python, SAS, Spotfire, Greenplum
• High availability
COMPANY PROFILE
• Industry: Government
• Product: Data analysis for criminality reduction
19. Main Benefits
• Central
Orchestration Tool
• Workflow
expressiveness:
Universal &
Comprehensive
• Management of
Security for
highly sensitive
environments
• Management of
Resources for all
appliances (SAS,
GREENPLUM,
TIBCO, …)
Execution prod Analytical prod Staging Dev
Virtualized Infrastructure using Docker
4 000 Physical Cores
21. Data from Mining
Machine Sensors
Data Processing
on Premises &
in the Clouds
Health and performance
of machines
Schedule data analytics
hourly or on events
Data
Real-Time Control & Optimizations
IoT Automation in the Cloud
ActiveEon allowed to migrate
from AWS to Azure
22. Scheduler
Passive
Main Benefits
Deployed On Premise (Capex)
or on a Hosting Service (Opex)
Auto-scaling on infrastructure
to match capacity and demand
Huge costs optimization using
only the VMs needed and
interruptible low cost instances
(e.g. EC2 Spot instances)
Capacity to deliver the system
on any third party Mediametrie
customer
TV Audience
Measurement
Scheduler
Active
EC2 Spot Instances
Low costs
EC2 Instances
Regular costs
IaaS
On-Prem
23. Azure PoC in the Box
Azure Node Source InfrastructureAzure Node Source Infrastructure
Scale automatically
Leverage Azure services
Existing Resources
Local & Network Resources Private Cloud, HPC & Others
Resource
Manager
Workflow
Scheduler
</>
Azure Node Source Infrastructure
LSF
Using Azure Scale Sets
Deploy over
150 k nodes
25. 150 K Nodes Benchmarks on Azure
• Up to 150 000 Nodes with a single Scheduler
instance on Azure
BIG COMPUTE READY
RESPONSIVE & RELIABLE
FAST & SCALABLE
• Average Response time of the scheduler: 6 ms
(Min 4 ms, Max 38 ms)
• Time to deploy Nodes:
• 10K: 5 mn
• Scale up to 40K: 10 mn
• Scale up to 80K: 15 mn
• Scale up to 150K: 25 mn
(To be compared: Average Time to buy a 150-Cores Servers/Cluster: 1 Year!)