Microsoft Innovation Center for Technical
Computing
MICROSOFT AZURE IN HPC SCENARIOS
Lukasz Miroslaw, Ph.D.
lukasz.mirosla...
Challenges
2
57 % of users are dissatisfied with their desktop
computing capacity*
* Source: US Council of Competitiveness...
Challenges
3
$70,000 server => $1M cost over 3 years
High costs of IT infrastructure
Low Cost of running in the cloud
 Cost Model assumes that the hardware makes 7% of Total Costs
4
Fig. Cost of a GFLOP in ...
Motivation
5
Agenda
6
 Use Case #1: Remote physical simulations with external partners.
 Use Case #2: Scale out physical simulations ...
Agenda
7
 Use Case #1: Remote physical simulations with external partners.
Azure IaaS, Remote App, Azure Batch
 Use Case...
What is Computational Fluid Dynamics?
 CFD is the science to simulate fluid flow, heat and mass transfer and
chemical rea...
What is Computational Fluid Dynamics?
 Airflow simulation around sky-diving Santa Claus.
9
* Source: Desktop Engineering
Use Case #1: Collaborative Simulations of Electrical Arcs
10
Use Case #1: Collaborative Simulations of Electrical Arcs
 Goal #1: Develop a Cloud-based algorithm for electrical arc si...
1st Use case: Instant ANSYS
12
 VM:
 D14 with 16 core
CPU, 112 GB RAM,
Windows Server
2012
 MpCCI, ANSYS
preinstalled
...
INSTANT ANSYS
13
No Installation. No configuration. No up-front costs.
 Access to powerful VMs with ANSYS already preinst...
Use Case #1: INSTANT ANSYS
14
IaaS DEMO
2nd Use case: Linux VM
15
 The UberCloud: Making Technical
Computing available in the Cloud
 UberCloud Community:
 +250...
2nd Use case: Linux VM
16
DEMO
The compute environment you ordered is now
ready.
Access your compute environment via remot...
Azure RemoteApp
17
Deliver apps from the cloud, cost-
effectively
Simplify your infrastructure
Run Windows apps anywhere
C...
Azure RemoteApp
18
Windows applications as a service
accessible from anywhere.
Costs
19
 VM with 16 cores and 56 GB RAM costs 2.11 CHF / hour (D14)
 1 TB of Storage costs 30 CHF / month
 RemoteApp s...
Short Summary
20
 + Powerful VMs that can be started/stopped on-demand increase
the productivity in our group.
 + Virtua...
Scalability Tests on Microsoft Azure
HPC Pack IaaS Demo
SimplyHPC: Light-weight Cloud Orchestrator for MS Azure
 What is SimplyHPC?
Framework
23
SimplyHPC:
1) Distributed framew...
SimplyHPC = Simpler Deployment
Performance and Scalability
 Example #1: Solving linear systems with PETSc and HPCG
25
Fig. Performance in GFlops of PETS...
Performance and Scalability
26
 Example #2: ANSYS CFX
Performance and Scalability
27
Fig. Strong scaling of ANSYS CFX of the compressor (11 mln nodes).
 Example #2: ANSYS CFX
Azure Batch
28
Batch is a managed service for batch
processing or batch computing - running a
large volume of similar task...
Short Summary
 SimplyHPC: a framework to simplify cluster deployment and
job submission.
 Set of light-weight PowerShell...
Short Summary
30
 + Scaling properties of Microsoft Azure is comparable to the on-premises
cluster.
 HSR Cluster: 7.3 da...
Microsoft Azure Machine Learning Studio
 Three types of knowlege:
 Know-What (facts)
 Know-How (processes)
 Know-Why (...
AzureML Studio
 Key goals of Machine Learning:
 Prediction
 Classification
 Clustering
 Collaborative Filtering
32
Im...
AzureML: Stellar Classification
 Classification Challenge:
 HYG database* is a compilation of
of stellar data from three...
AzureML Example: Heating Load Prognosis
34
Image credit: SAB Magazine
Input:
- Roof area
- Overall hight
- Glazing area
- ...
AzureML Workflow
35
Machine Learning Workflow
1. Hypothesis
2. Data Preparation
3. Model
4. Test
5. Evaluate
A. Tsanas, A....
AzureML: Cost Model
36
AzureML: Short Summary
37
 Very fast prototyping. Load the system with data, test different
Machine Learning methods.
 P...
Summary
38
 Computing and storage at competitive
price.
 High Availability, data redundancy,
disaster recovery services ...
Nächste SlideShare
Wird geladen in …5
×

Microsoft Azure in HPC scenarios

1.244 Aufrufe

Veröffentlicht am

We present applications of Azure Services such as Azure IaaS/PaaS and Azure RemoteApp in computational fluid dynamics and sparse linear algebra. We also present Microsoft Machine Learning Studio in prediction of the heating load in the buildings.

Veröffentlicht in: Ingenieurwesen
0 Kommentare
0 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Keine Downloads
Aufrufe
Aufrufe insgesamt
1.244
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
363
Aktionen
Geteilt
0
Downloads
13
Kommentare
0
Gefällt mir
0
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

Microsoft Azure in HPC scenarios

  1. 1. Microsoft Innovation Center for Technical Computing MICROSOFT AZURE IN HPC SCENARIOS Lukasz Miroslaw, Ph.D. lukasz.miroslaw@hsr.ch 18.11.2015, MICROSOFT SWITZERLAND
  2. 2. Challenges 2 57 % of users are dissatisfied with their desktop computing capacity* * Source: US Council of Competitiveness: http://www.compete.org, theubercloud.com Computing: too slow Memory: too small Fig. Sometimes solving a problem with IT is hard.
  3. 3. Challenges 3 $70,000 server => $1M cost over 3 years High costs of IT infrastructure
  4. 4. Low Cost of running in the cloud  Cost Model assumes that the hardware makes 7% of Total Costs 4 Fig. Cost of a GFLOP in U.S. Dollars on different Microsoft Azure nodes and a private HSR cluster.
  5. 5. Motivation 5
  6. 6. Agenda 6  Use Case #1: Remote physical simulations with external partners.  Use Case #2: Scale out physical simulations in the cloud.  Use Case #3: Stellar Classification, Prediction of Energy Efficiency in buildings  Conclusions.
  7. 7. Agenda 7  Use Case #1: Remote physical simulations with external partners. Azure IaaS, Remote App, Azure Batch  Use Case #2: Scale out physical simulations in the cloud. SimplyHPC, HPC Pack  Use Case #3: Stellar Classification, Prediction of Energy Efficiency in buildings AzureML  Conclusions.
  8. 8. What is Computational Fluid Dynamics?  CFD is the science to simulate fluid flow, heat and mass transfer and chemical reactions 8
  9. 9. What is Computational Fluid Dynamics?  Airflow simulation around sky-diving Santa Claus. 9 * Source: Desktop Engineering
  10. 10. Use Case #1: Collaborative Simulations of Electrical Arcs 10
  11. 11. Use Case #1: Collaborative Simulations of Electrical Arcs  Goal #1: Develop a Cloud-based algorithm for electrical arc simulation  Microsoft Azure Research Award in 2014  Contact: Kenji Takeda (Microsoft Research)  Goal #2: Provide simulation tool to partners in Brasil and Deutschland  Ongoing collaborations  Streamer International (CTI Project)  Panasonic  Fraunhofer SCAI  WEG 11
  12. 12. 1st Use case: Instant ANSYS 12  VM:  D14 with 16 core CPU, 112 GB RAM, Windows Server 2012  MpCCI, ANSYS preinstalled  Storage: locally redundant, automatically scallable  License Server (LS) on A0 in Germany Customer VM LS
  13. 13. INSTANT ANSYS 13 No Installation. No configuration. No up-front costs.  Access to powerful VMs with ANSYS already preinstalled and preconfigured.  Access to redundant and highly available storage.  Disaster Recovery and 99.5% SLA.  Connection to on-premise infrastructure with IPSec VPN.
  14. 14. Use Case #1: INSTANT ANSYS 14 IaaS DEMO
  15. 15. 2nd Use case: Linux VM 15  The UberCloud: Making Technical Computing available in the Cloud  UberCloud Community:  +2500 companies and individuals:  +60 cloud providers,  +80 software providers,  several hundred consulting firms and individual experts.  OpenFOAM added to Azure Marketplace  Docker containerization www.ubercloud.com
  16. 16. 2nd Use case: Linux VM 16 DEMO The compute environment you ordered is now ready. Access your compute environment via remote desktop connection (Chrome 8+, Firefox 7+, Opera 11+, IE 9+) Launch Your password for remote desktop access is: TN1b39pv4Djw
  17. 17. Azure RemoteApp 17 Deliver apps from the cloud, cost- effectively Simplify your infrastructure Run Windows apps anywhere Centralize your apps, help secure your data
  18. 18. Azure RemoteApp 18 Windows applications as a service accessible from anywhere.
  19. 19. Costs 19  VM with 16 cores and 56 GB RAM costs 2.11 CHF / hour (D14)  1 TB of Storage costs 30 CHF / month  RemoteApp starting price: $10 / user / month (40h included)  Online Calculator Azure in Education Faculty will receive a 12 month, $250/month account Students will receive a 6 month, $100/month account
  20. 20. Short Summary 20  + Powerful VMs that can be started/stopped on-demand increase the productivity in our group.  + Virtual images with OS and different software version to avoid problems with backward compatibility.  + Students and team members can manage their own VMs and reduce the costs of support.  - Storage File Service can be easily mapped to a drive on the VM but not on premises.  - Only a single user can access one VM.
  21. 21. Scalability Tests on Microsoft Azure
  22. 22. HPC Pack IaaS Demo
  23. 23. SimplyHPC: Light-weight Cloud Orchestrator for MS Azure  What is SimplyHPC? Framework 23 SimplyHPC: 1) Distributed framework for Microsoft Azure, 2) Set of PowerShell scripts.
  24. 24. SimplyHPC = Simpler Deployment
  25. 25. Performance and Scalability  Example #1: Solving linear systems with PETSc and HPCG 25 Fig. Performance in GFlops of PETSc solving ruep (right) matrix system and HPCG Benchmark (left) on different Microsoft Azure nodes and a private HSR cluster.
  26. 26. Performance and Scalability 26  Example #2: ANSYS CFX
  27. 27. Performance and Scalability 27 Fig. Strong scaling of ANSYS CFX of the compressor (11 mln nodes).  Example #2: ANSYS CFX
  28. 28. Azure Batch 28 Batch is a managed service for batch processing or batch computing - running a large volume of similar tasks to get some desired result.
  29. 29. Short Summary  SimplyHPC: a framework to simplify cluster deployment and job submission.  Set of light-weight PowerShell scripts to submit, execute and monitor multi-threaded jobs on Windows Azure.  Easy to use. No cloud-related knowledge necessary.  Run the jobs from command line and download the results directly to your Azure Storage.  Up to 9x faster than native MS HPC Pack scripts.  Available at https://github.com/vbaros/SimplyHPC 29 L Miroslaw, V Baros, M Pantic, H Nordborg, Unified Cloud Orchestration Framework for Elastic High Performance Computing on Microsoft Azure, NAFEMS World Congress 2015
  30. 30. Short Summary 30  + Scaling properties of Microsoft Azure is comparable to the on-premises cluster.  HSR Cluster: 7.3 days (176 hours), limited availability.  Microsoft Azure: 4.9 days (118 hours), ca. 50% faster, 100% availability.  + Dynamic scaling (up- / downscaling) and instant access to the newest hardware reduces the costs.  + (Un)limited computing at competitive price. Cluster composed of 32 x A8 nodes (=256 cores) costs 32 x 2.11 CHF/h = ca. 68 CHF/h  - Upscaling > 100 cores should be planned in advance.
  31. 31. Microsoft Azure Machine Learning Studio  Three types of knowlege:  Know-What (facts)  Know-How (processes)  Know-Why (reasons) 31 Image credit: Univ. Hamburg
  32. 32. AzureML Studio  Key goals of Machine Learning:  Prediction  Classification  Clustering  Collaborative Filtering 32 Image credits: OpenCV, Snipview, Stanford
  33. 33. AzureML: Stellar Classification  Classification Challenge:  HYG database* is a compilation of of stellar data from three main catalogues.  Contains ca. 120k stars, 37 spectral characteristics.  2D classification scheme based on temperature (color index) and brightness (absolute magnitude).  Data is incomplete and may contain a few misclassifictions.  Prediction Engine developed in AzureML 33 Credits: Michael Pantic (HSR)* http://www.astronexus.com/hyg
  34. 34. AzureML Example: Heating Load Prognosis 34 Image credit: SAB Magazine Input: - Roof area - Overall hight - Glazing area - Surface area - ... Output: - Heating load prediction
  35. 35. AzureML Workflow 35 Machine Learning Workflow 1. Hypothesis 2. Data Preparation 3. Model 4. Test 5. Evaluate A. Tsanas, A. Xifara: 'Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools', Energy and Buildings, Vol. 49, pp. 560-567, 201 - 8 physical characteristics from 768 buildings - Goal: predict buldings’ heating load and cooling load - Architects need to compare several building designs before selecting the final approach
  36. 36. AzureML: Cost Model 36
  37. 37. AzureML: Short Summary 37  Very fast prototyping. Load the system with data, test different Machine Learning methods.  Platform for Internet of Things: Event Hubs, Stream Analytics.  Share the models & results.  Deploy web services fast.  Develop own methods in Python and R Statistics.
  38. 38. Summary 38  Computing and storage at competitive price.  High Availability, data redundancy, disaster recovery services are included.  Data transfer take some time.  Up- and downscaling resources dynamically. Higher productivity.  „Cloudify” your system’s complexity.

×