Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
Microsoft Innovation Center for Technical
Computing
MICROSOFT AZURE IN HPC SCENARIOS
Lukasz Miroslaw, Ph.D.
lukasz.mirosla...
Challenges
2
57 % of users are dissatisfied with their desktop
computing capacity*
* Source: US Council of Competitiveness...
Challenges
3
$70,000 server => $1M cost over 3 years
High costs of IT infrastructure
Low Cost of running in the cloud
 Cost Model assumes that the hardware makes 7% of Total Costs
4
Fig. Cost of a GFLOP in ...
Motivation
5
Agenda
6
 Use Case #1: Remote physical simulations with external partners.
 Use Case #2: Scale out physical simulations ...
Agenda
7
 Use Case #1: Remote physical simulations with external partners.
Azure IaaS, Remote App, Azure Batch
 Use Case...
What is Computational Fluid Dynamics?
 CFD is the science to simulate fluid flow, heat and mass transfer and
chemical rea...
What is Computational Fluid Dynamics?
 Airflow simulation around sky-diving Santa Claus.
9
* Source: Desktop Engineering
Use Case #1: Collaborative Simulations of Electrical Arcs
10
Use Case #1: Collaborative Simulations of Electrical Arcs
 Goal #1: Develop a Cloud-based algorithm for electrical arc si...
1st Use case: Instant ANSYS
12
 VM:
 D14 with 16 core
CPU, 112 GB RAM,
Windows Server
2012
 MpCCI, ANSYS
preinstalled
...
INSTANT ANSYS
13
No Installation. No configuration. No up-front costs.
 Access to powerful VMs with ANSYS already preinst...
Use Case #1: INSTANT ANSYS
14
IaaS DEMO
2nd Use case: Linux VM
15
 The UberCloud: Making Technical
Computing available in the Cloud
 UberCloud Community:
 +250...
2nd Use case: Linux VM
16
DEMO
The compute environment you ordered is now
ready.
Access your compute environment via remot...
Azure RemoteApp
17
Deliver apps from the cloud, cost-
effectively
Simplify your infrastructure
Run Windows apps anywhere
C...
Azure RemoteApp
18
Windows applications as a service
accessible from anywhere.
Costs
19
 VM with 16 cores and 56 GB RAM costs 2.11 CHF / hour (D14)
 1 TB of Storage costs 30 CHF / month
 RemoteApp s...
Short Summary
20
 + Powerful VMs that can be started/stopped on-demand increase
the productivity in our group.
 + Virtua...
Scalability Tests on Microsoft Azure
HPC Pack IaaS Demo
SimplyHPC: Light-weight Cloud Orchestrator for MS Azure
 What is SimplyHPC?
Framework
23
SimplyHPC:
1) Distributed framew...
SimplyHPC = Simpler Deployment
Performance and Scalability
 Example #1: Solving linear systems with PETSc and HPCG
25
Fig. Performance in GFlops of PETS...
Performance and Scalability
26
 Example #2: ANSYS CFX
Performance and Scalability
27
Fig. Strong scaling of ANSYS CFX of the compressor (11 mln nodes).
 Example #2: ANSYS CFX
Azure Batch
28
Batch is a managed service for batch
processing or batch computing - running a
large volume of similar task...
Short Summary
 SimplyHPC: a framework to simplify cluster deployment and
job submission.
 Set of light-weight PowerShell...
Short Summary
30
 + Scaling properties of Microsoft Azure is comparable to the on-premises
cluster.
 HSR Cluster: 7.3 da...
Microsoft Azure Machine Learning Studio
 Three types of knowlege:
 Know-What (facts)
 Know-How (processes)
 Know-Why (...
AzureML Studio
 Key goals of Machine Learning:
 Prediction
 Classification
 Clustering
 Collaborative Filtering
32
Im...
AzureML: Stellar Classification
 Classification Challenge:
 HYG database* is a compilation of
of stellar data from three...
AzureML Example: Heating Load Prognosis
34
Image credit: SAB Magazine
Input:
- Roof area
- Overall hight
- Glazing area
- ...
AzureML Workflow
35
Machine Learning Workflow
1. Hypothesis
2. Data Preparation
3. Model
4. Test
5. Evaluate
A. Tsanas, A....
AzureML: Cost Model
36
AzureML: Short Summary
37
 Very fast prototyping. Load the system with data, test different
Machine Learning methods.
 P...
Summary
38
 Computing and storage at competitive
price.
 High Availability, data redundancy,
disaster recovery services ...
Nächste SlideShare
Wird geladen in …5
×

Microsoft Azure in HPC scenarios

1.306 Aufrufe

Veröffentlicht am

We present applications of Azure Services such as Azure IaaS/PaaS and Azure RemoteApp in computational fluid dynamics and sparse linear algebra. We also present Microsoft Machine Learning Studio in prediction of the heating load in the buildings.

Veröffentlicht in: Ingenieurwesen
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Microsoft Azure in HPC scenarios

  1. 1. Microsoft Innovation Center for Technical Computing MICROSOFT AZURE IN HPC SCENARIOS Lukasz Miroslaw, Ph.D. lukasz.miroslaw@hsr.ch 18.11.2015, MICROSOFT SWITZERLAND
  2. 2. Challenges 2 57 % of users are dissatisfied with their desktop computing capacity* * Source: US Council of Competitiveness: http://www.compete.org, theubercloud.com Computing: too slow Memory: too small Fig. Sometimes solving a problem with IT is hard.
  3. 3. Challenges 3 $70,000 server => $1M cost over 3 years High costs of IT infrastructure
  4. 4. Low Cost of running in the cloud  Cost Model assumes that the hardware makes 7% of Total Costs 4 Fig. Cost of a GFLOP in U.S. Dollars on different Microsoft Azure nodes and a private HSR cluster.
  5. 5. Motivation 5
  6. 6. Agenda 6  Use Case #1: Remote physical simulations with external partners.  Use Case #2: Scale out physical simulations in the cloud.  Use Case #3: Stellar Classification, Prediction of Energy Efficiency in buildings  Conclusions.
  7. 7. Agenda 7  Use Case #1: Remote physical simulations with external partners. Azure IaaS, Remote App, Azure Batch  Use Case #2: Scale out physical simulations in the cloud. SimplyHPC, HPC Pack  Use Case #3: Stellar Classification, Prediction of Energy Efficiency in buildings AzureML  Conclusions.
  8. 8. What is Computational Fluid Dynamics?  CFD is the science to simulate fluid flow, heat and mass transfer and chemical reactions 8
  9. 9. What is Computational Fluid Dynamics?  Airflow simulation around sky-diving Santa Claus. 9 * Source: Desktop Engineering
  10. 10. Use Case #1: Collaborative Simulations of Electrical Arcs 10
  11. 11. Use Case #1: Collaborative Simulations of Electrical Arcs  Goal #1: Develop a Cloud-based algorithm for electrical arc simulation  Microsoft Azure Research Award in 2014  Contact: Kenji Takeda (Microsoft Research)  Goal #2: Provide simulation tool to partners in Brasil and Deutschland  Ongoing collaborations  Streamer International (CTI Project)  Panasonic  Fraunhofer SCAI  WEG 11
  12. 12. 1st Use case: Instant ANSYS 12  VM:  D14 with 16 core CPU, 112 GB RAM, Windows Server 2012  MpCCI, ANSYS preinstalled  Storage: locally redundant, automatically scallable  License Server (LS) on A0 in Germany Customer VM LS
  13. 13. INSTANT ANSYS 13 No Installation. No configuration. No up-front costs.  Access to powerful VMs with ANSYS already preinstalled and preconfigured.  Access to redundant and highly available storage.  Disaster Recovery and 99.5% SLA.  Connection to on-premise infrastructure with IPSec VPN.
  14. 14. Use Case #1: INSTANT ANSYS 14 IaaS DEMO
  15. 15. 2nd Use case: Linux VM 15  The UberCloud: Making Technical Computing available in the Cloud  UberCloud Community:  +2500 companies and individuals:  +60 cloud providers,  +80 software providers,  several hundred consulting firms and individual experts.  OpenFOAM added to Azure Marketplace  Docker containerization www.ubercloud.com
  16. 16. 2nd Use case: Linux VM 16 DEMO The compute environment you ordered is now ready. Access your compute environment via remote desktop connection (Chrome 8+, Firefox 7+, Opera 11+, IE 9+) Launch Your password for remote desktop access is: TN1b39pv4Djw
  17. 17. Azure RemoteApp 17 Deliver apps from the cloud, cost- effectively Simplify your infrastructure Run Windows apps anywhere Centralize your apps, help secure your data
  18. 18. Azure RemoteApp 18 Windows applications as a service accessible from anywhere.
  19. 19. Costs 19  VM with 16 cores and 56 GB RAM costs 2.11 CHF / hour (D14)  1 TB of Storage costs 30 CHF / month  RemoteApp starting price: $10 / user / month (40h included)  Online Calculator Azure in Education Faculty will receive a 12 month, $250/month account Students will receive a 6 month, $100/month account
  20. 20. Short Summary 20  + Powerful VMs that can be started/stopped on-demand increase the productivity in our group.  + Virtual images with OS and different software version to avoid problems with backward compatibility.  + Students and team members can manage their own VMs and reduce the costs of support.  - Storage File Service can be easily mapped to a drive on the VM but not on premises.  - Only a single user can access one VM.
  21. 21. Scalability Tests on Microsoft Azure
  22. 22. HPC Pack IaaS Demo
  23. 23. SimplyHPC: Light-weight Cloud Orchestrator for MS Azure  What is SimplyHPC? Framework 23 SimplyHPC: 1) Distributed framework for Microsoft Azure, 2) Set of PowerShell scripts.
  24. 24. SimplyHPC = Simpler Deployment
  25. 25. Performance and Scalability  Example #1: Solving linear systems with PETSc and HPCG 25 Fig. Performance in GFlops of PETSc solving ruep (right) matrix system and HPCG Benchmark (left) on different Microsoft Azure nodes and a private HSR cluster.
  26. 26. Performance and Scalability 26  Example #2: ANSYS CFX
  27. 27. Performance and Scalability 27 Fig. Strong scaling of ANSYS CFX of the compressor (11 mln nodes).  Example #2: ANSYS CFX
  28. 28. Azure Batch 28 Batch is a managed service for batch processing or batch computing - running a large volume of similar tasks to get some desired result.
  29. 29. Short Summary  SimplyHPC: a framework to simplify cluster deployment and job submission.  Set of light-weight PowerShell scripts to submit, execute and monitor multi-threaded jobs on Windows Azure.  Easy to use. No cloud-related knowledge necessary.  Run the jobs from command line and download the results directly to your Azure Storage.  Up to 9x faster than native MS HPC Pack scripts.  Available at https://github.com/vbaros/SimplyHPC 29 L Miroslaw, V Baros, M Pantic, H Nordborg, Unified Cloud Orchestration Framework for Elastic High Performance Computing on Microsoft Azure, NAFEMS World Congress 2015
  30. 30. Short Summary 30  + Scaling properties of Microsoft Azure is comparable to the on-premises cluster.  HSR Cluster: 7.3 days (176 hours), limited availability.  Microsoft Azure: 4.9 days (118 hours), ca. 50% faster, 100% availability.  + Dynamic scaling (up- / downscaling) and instant access to the newest hardware reduces the costs.  + (Un)limited computing at competitive price. Cluster composed of 32 x A8 nodes (=256 cores) costs 32 x 2.11 CHF/h = ca. 68 CHF/h  - Upscaling > 100 cores should be planned in advance.
  31. 31. Microsoft Azure Machine Learning Studio  Three types of knowlege:  Know-What (facts)  Know-How (processes)  Know-Why (reasons) 31 Image credit: Univ. Hamburg
  32. 32. AzureML Studio  Key goals of Machine Learning:  Prediction  Classification  Clustering  Collaborative Filtering 32 Image credits: OpenCV, Snipview, Stanford
  33. 33. AzureML: Stellar Classification  Classification Challenge:  HYG database* is a compilation of of stellar data from three main catalogues.  Contains ca. 120k stars, 37 spectral characteristics.  2D classification scheme based on temperature (color index) and brightness (absolute magnitude).  Data is incomplete and may contain a few misclassifictions.  Prediction Engine developed in AzureML 33 Credits: Michael Pantic (HSR)* http://www.astronexus.com/hyg
  34. 34. AzureML Example: Heating Load Prognosis 34 Image credit: SAB Magazine Input: - Roof area - Overall hight - Glazing area - Surface area - ... Output: - Heating load prediction
  35. 35. AzureML Workflow 35 Machine Learning Workflow 1. Hypothesis 2. Data Preparation 3. Model 4. Test 5. Evaluate A. Tsanas, A. Xifara: 'Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools', Energy and Buildings, Vol. 49, pp. 560-567, 201 - 8 physical characteristics from 768 buildings - Goal: predict buldings’ heating load and cooling load - Architects need to compare several building designs before selecting the final approach
  36. 36. AzureML: Cost Model 36
  37. 37. AzureML: Short Summary 37  Very fast prototyping. Load the system with data, test different Machine Learning methods.  Platform for Internet of Things: Event Hubs, Stream Analytics.  Share the models & results.  Deploy web services fast.  Develop own methods in Python and R Statistics.
  38. 38. Summary 38  Computing and storage at competitive price.  High Availability, data redundancy, disaster recovery services are included.  Data transfer take some time.  Up- and downscaling resources dynamically. Higher productivity.  „Cloudify” your system’s complexity.

×