Microsoft Azure in HPC scenarios

1.191 Aufrufe

Veröffentlicht am

We present applications of Azure Services such as Azure IaaS/PaaS and Azure RemoteApp in computational fluid dynamics and sparse linear algebra. We also present Microsoft Machine Learning Studio in prediction of the heating load in the buildings.

Veröffentlicht in: Ingenieurwesen
0 Kommentare
0 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Keine Downloads
Aufrufe
Aufrufe insgesamt
1.191
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
348
Aktionen
Geteilt
0
Downloads
12
Kommentare
0
Gefällt mir
0
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

Microsoft Azure in HPC scenarios

  1. 1. Microsoft Innovation Center for Technical Computing MICROSOFT AZURE IN HPC SCENARIOS Lukasz Miroslaw, Ph.D. lukasz.miroslaw@hsr.ch 18.11.2015, MICROSOFT SWITZERLAND
  2. 2. Challenges 2 57 % of users are dissatisfied with their desktop computing capacity* * Source: US Council of Competitiveness: http://www.compete.org, theubercloud.com Computing: too slow Memory: too small Fig. Sometimes solving a problem with IT is hard.
  3. 3. Challenges 3 $70,000 server => $1M cost over 3 years High costs of IT infrastructure
  4. 4. Low Cost of running in the cloud  Cost Model assumes that the hardware makes 7% of Total Costs 4 Fig. Cost of a GFLOP in U.S. Dollars on different Microsoft Azure nodes and a private HSR cluster.
  5. 5. Motivation 5
  6. 6. Agenda 6  Use Case #1: Remote physical simulations with external partners.  Use Case #2: Scale out physical simulations in the cloud.  Use Case #3: Stellar Classification, Prediction of Energy Efficiency in buildings  Conclusions.
  7. 7. Agenda 7  Use Case #1: Remote physical simulations with external partners. Azure IaaS, Remote App, Azure Batch  Use Case #2: Scale out physical simulations in the cloud. SimplyHPC, HPC Pack  Use Case #3: Stellar Classification, Prediction of Energy Efficiency in buildings AzureML  Conclusions.
  8. 8. What is Computational Fluid Dynamics?  CFD is the science to simulate fluid flow, heat and mass transfer and chemical reactions 8
  9. 9. What is Computational Fluid Dynamics?  Airflow simulation around sky-diving Santa Claus. 9 * Source: Desktop Engineering
  10. 10. Use Case #1: Collaborative Simulations of Electrical Arcs 10
  11. 11. Use Case #1: Collaborative Simulations of Electrical Arcs  Goal #1: Develop a Cloud-based algorithm for electrical arc simulation  Microsoft Azure Research Award in 2014  Contact: Kenji Takeda (Microsoft Research)  Goal #2: Provide simulation tool to partners in Brasil and Deutschland  Ongoing collaborations  Streamer International (CTI Project)  Panasonic  Fraunhofer SCAI  WEG 11
  12. 12. 1st Use case: Instant ANSYS 12  VM:  D14 with 16 core CPU, 112 GB RAM, Windows Server 2012  MpCCI, ANSYS preinstalled  Storage: locally redundant, automatically scallable  License Server (LS) on A0 in Germany Customer VM LS
  13. 13. INSTANT ANSYS 13 No Installation. No configuration. No up-front costs.  Access to powerful VMs with ANSYS already preinstalled and preconfigured.  Access to redundant and highly available storage.  Disaster Recovery and 99.5% SLA.  Connection to on-premise infrastructure with IPSec VPN.
  14. 14. Use Case #1: INSTANT ANSYS 14 IaaS DEMO
  15. 15. 2nd Use case: Linux VM 15  The UberCloud: Making Technical Computing available in the Cloud  UberCloud Community:  +2500 companies and individuals:  +60 cloud providers,  +80 software providers,  several hundred consulting firms and individual experts.  OpenFOAM added to Azure Marketplace  Docker containerization www.ubercloud.com
  16. 16. 2nd Use case: Linux VM 16 DEMO The compute environment you ordered is now ready. Access your compute environment via remote desktop connection (Chrome 8+, Firefox 7+, Opera 11+, IE 9+) Launch Your password for remote desktop access is: TN1b39pv4Djw
  17. 17. Azure RemoteApp 17 Deliver apps from the cloud, cost- effectively Simplify your infrastructure Run Windows apps anywhere Centralize your apps, help secure your data
  18. 18. Azure RemoteApp 18 Windows applications as a service accessible from anywhere.
  19. 19. Costs 19  VM with 16 cores and 56 GB RAM costs 2.11 CHF / hour (D14)  1 TB of Storage costs 30 CHF / month  RemoteApp starting price: $10 / user / month (40h included)  Online Calculator Azure in Education Faculty will receive a 12 month, $250/month account Students will receive a 6 month, $100/month account
  20. 20. Short Summary 20  + Powerful VMs that can be started/stopped on-demand increase the productivity in our group.  + Virtual images with OS and different software version to avoid problems with backward compatibility.  + Students and team members can manage their own VMs and reduce the costs of support.  - Storage File Service can be easily mapped to a drive on the VM but not on premises.  - Only a single user can access one VM.
  21. 21. Scalability Tests on Microsoft Azure
  22. 22. HPC Pack IaaS Demo
  23. 23. SimplyHPC: Light-weight Cloud Orchestrator for MS Azure  What is SimplyHPC? Framework 23 SimplyHPC: 1) Distributed framework for Microsoft Azure, 2) Set of PowerShell scripts.
  24. 24. SimplyHPC = Simpler Deployment
  25. 25. Performance and Scalability  Example #1: Solving linear systems with PETSc and HPCG 25 Fig. Performance in GFlops of PETSc solving ruep (right) matrix system and HPCG Benchmark (left) on different Microsoft Azure nodes and a private HSR cluster.
  26. 26. Performance and Scalability 26  Example #2: ANSYS CFX
  27. 27. Performance and Scalability 27 Fig. Strong scaling of ANSYS CFX of the compressor (11 mln nodes).  Example #2: ANSYS CFX
  28. 28. Azure Batch 28 Batch is a managed service for batch processing or batch computing - running a large volume of similar tasks to get some desired result.
  29. 29. Short Summary  SimplyHPC: a framework to simplify cluster deployment and job submission.  Set of light-weight PowerShell scripts to submit, execute and monitor multi-threaded jobs on Windows Azure.  Easy to use. No cloud-related knowledge necessary.  Run the jobs from command line and download the results directly to your Azure Storage.  Up to 9x faster than native MS HPC Pack scripts.  Available at https://github.com/vbaros/SimplyHPC 29 L Miroslaw, V Baros, M Pantic, H Nordborg, Unified Cloud Orchestration Framework for Elastic High Performance Computing on Microsoft Azure, NAFEMS World Congress 2015
  30. 30. Short Summary 30  + Scaling properties of Microsoft Azure is comparable to the on-premises cluster.  HSR Cluster: 7.3 days (176 hours), limited availability.  Microsoft Azure: 4.9 days (118 hours), ca. 50% faster, 100% availability.  + Dynamic scaling (up- / downscaling) and instant access to the newest hardware reduces the costs.  + (Un)limited computing at competitive price. Cluster composed of 32 x A8 nodes (=256 cores) costs 32 x 2.11 CHF/h = ca. 68 CHF/h  - Upscaling > 100 cores should be planned in advance.
  31. 31. Microsoft Azure Machine Learning Studio  Three types of knowlege:  Know-What (facts)  Know-How (processes)  Know-Why (reasons) 31 Image credit: Univ. Hamburg
  32. 32. AzureML Studio  Key goals of Machine Learning:  Prediction  Classification  Clustering  Collaborative Filtering 32 Image credits: OpenCV, Snipview, Stanford
  33. 33. AzureML: Stellar Classification  Classification Challenge:  HYG database* is a compilation of of stellar data from three main catalogues.  Contains ca. 120k stars, 37 spectral characteristics.  2D classification scheme based on temperature (color index) and brightness (absolute magnitude).  Data is incomplete and may contain a few misclassifictions.  Prediction Engine developed in AzureML 33 Credits: Michael Pantic (HSR)* http://www.astronexus.com/hyg
  34. 34. AzureML Example: Heating Load Prognosis 34 Image credit: SAB Magazine Input: - Roof area - Overall hight - Glazing area - Surface area - ... Output: - Heating load prediction
  35. 35. AzureML Workflow 35 Machine Learning Workflow 1. Hypothesis 2. Data Preparation 3. Model 4. Test 5. Evaluate A. Tsanas, A. Xifara: 'Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools', Energy and Buildings, Vol. 49, pp. 560-567, 201 - 8 physical characteristics from 768 buildings - Goal: predict buldings’ heating load and cooling load - Architects need to compare several building designs before selecting the final approach
  36. 36. AzureML: Cost Model 36
  37. 37. AzureML: Short Summary 37  Very fast prototyping. Load the system with data, test different Machine Learning methods.  Platform for Internet of Things: Event Hubs, Stream Analytics.  Share the models & results.  Deploy web services fast.  Develop own methods in Python and R Statistics.
  38. 38. Summary 38  Computing and storage at competitive price.  High Availability, data redundancy, disaster recovery services are included.  Data transfer take some time.  Up- and downscaling resources dynamically. Higher productivity.  „Cloudify” your system’s complexity.

×