SlideShare ist ein Scribd-Unternehmen logo
1 von 38
Downloaden Sie, um offline zu lesen
WSV206
Taking High Performance Computing to the Cloud: Windows HPC and
Taking High Performance Computing to the Cloud: Windows HPC and
massively parallel
Becoming critical part of R&D for businesses
Traditionally required specialized staffing
X64 Server
1991 1998 2008
$40,000,000 $1,000,000 $1,000
Taking High Performance Computing to the Cloud: Windows HPC and
• Advances in technical computing and research are
powered by access to computing resources
Barrier to entry for technical workloads
How Do we take existing technical workloads onto azure?
My CRAY CX1 Windows Azure Only
Initial cost: Planning, Hardware, Space, Power,
Cooling, $10000s
Get an account.
Cost of ownership: Maintenance, Staff, Power, Backups,
Internet, Storage, Down time, Disk failures,
Paying for idle computing!
Pay as you go, limited only by your
budget
Suitable Applications: MPI problems require high speed network.
HPC Batch Scheduler System
Applications without much traffic.
No HPC batch submission engine
Node interconnects: 20 Gbps Infini-band! 1 Gbps Ethernet
As an officemate: No! Silent and Invisible
CPU & Memory Hardware: Dedicated 8 nodes with 8 cores &16gb N+1 nodes 8 cores &16gb
Access to My data: LAN Connection Internet Connection first upload
LAN Connection if already in blob
On-premise
Desktop User
HPC Head Node
Broker Node(s)
HPC Cluster
Desktop Compute Cloud via
Idle Win 7 Workstation Cores
Azure Compute Instances
Azure Compute Proxies
Transparently
Taking High Performance Computing to the Cloud: Windows HPC and
…
Fabric
Compute Storage
Application
Fabric
A closer look
VM
Web Role Worker Role
Agent Agent
main()
{ … }
Load
Balancer
HTTP
IIS
ASP.NET, WCF,
etc.
demo
Taking High Performance Computing to the Cloud: Windows HPC and
Infiniband running at 20Gbps!
Taking High Performance Computing to the Cloud: Windows HPC and
Taking High Performance Computing to the Cloud: Windows HPC and
Windows HPC on Azure
Mixed Mode Deployment On-premise andAzure
On-premise
• “Burst” into cloud on-demand while
keeping control over data and
corporate policies
• Pay only for what you use
• A stepping stone to hybrid and
public clouds.
• Dynamically adjust how much runs
on-premise and in the cloud
• Azure computer instances include
both worker role and VM role
Azure
Compute Nodes
DesktopsHPC Head Node
Broker Node
Compute Nodes On-Premise and in Azure Simultaneously
Azure
Compute Instances
Compute Proxies
Embarrassingly parallel, no data
Embarrassingly parallel + data
Shape Optimization with Sculptor
Cut design times
Cut design costs
Provide better products
Geometric optimization
New trend in design
compute what you can afford
Latin hyperCube sampling
A Statistical method
Know your cost up front
Know you time to solution
More design insight
Best statistical accuracy
Change shape of intake
Improve flows
Better gas mileage
Better performance
demo
MPI on-premises + data shared in Azure
Taking High Performance Computing to the Cloud: Windows HPC and
Taking High Performance Computing to the Cloud: Windows HPC and
Transparently
Taking High Performance Computing to the Cloud: Windows HPC and
DeveloperTools
Compilers:
• PGI
• Intel
• GCC
Build:
• CMake
• Visual Studio
Unix Env:
• Cygwin
• SUA
• MinGW
Libraries
Solvers:
• PETSc*
• Trilinos
• FFTW
IO:
• HDF5MPI
• NetCDF
PopularHPCApps
Chemistry
Bio Science*
CFD
Structure
Climate*
Physics
Data Mining
Computer
Science
AnalysisTools
Parallel Visualization
• VisIT*
• ParaView*
Python
• Ipython*
• NumPY
• SciPY
HPC Server 2008 R2 Roadmap
Microsoft HPC Server 2008 R2 SP1 &
LINQtoHPC CTP
 Integration with Windows Azure:
 “Burst” SOA and Parametric sweep jobs to
Azure as worker roles.
 All compute nodes in Azure with head-node
only on-premise.
 User activity scheduling for workstation nodes
 C# cluster SOA debugger for VS 2010.
Microsoft HPC Server 2008 R2
 Excel on HPC clusters
 Schedule jobs to workstations
 Business critical SOA
 Scale to 1000+ nodes out-of-the-box
 Dual boot clusters
 Diskless compute nodes
 Extensible diagnostics and troubleshooting
framework
 Flexible, customizable color-coded Heat map
Microsoft HPC Server 2008 R2 SP2*
 LINQ to HPC for large scale data analytics
 MPI runtime in Windows Azure
 Azure VM role
 New job scheduler policies
 Lend/Borrow resource pools
 Over/under subscribe nodes
 Common data for SOA Apps
 Multi-domain support for workstation nodes
 Web Portal for job submission and job status
 Smart Card authentication
July 2010 Dec 2010 June 2011
try
http://blogs.msdn.com/hpctrekker
Application Models and Data Considerations
http://connect.microsoft.com
free Trial of Azure for 30 days
try out the Azure Burst Samples
Related Content
WSV205 | Dryad: Running “Big Data” Applications on a Windows HPC Server
Cluster
WSV207 | Distributed Scale-Out Applications On-Premises and in the Cloud:
An Introduction to Windows HPC Server
WSV472-INT | Building Scale Out Clusters with Windows HPC Server 2008 R2:
Cluster Deployment, Troubleshooting and Basic Job Submission Techniques
Product Demo Stations (Windows HPC Server 2008 R2 SP2)
Find Me Later At…Server HPC Booth Thursday 10:30am – 1:00pm
My Blog: http://blogs.msdn.com/hpctrekker
The Azure Burst Samples and White paper
Windows HPC ResourceKit
Windows HPC Server TechCenter
Blue Section
http://www.microsoft.com/windowsazure/
http://www.microsoft.com/systemcenter/
http://www.microsoft.com/forefront/
http://www.microsoft.com/windowsserver/
http://www.microsoft.com/cloud/
http://www.microsoft.com/privatecloud/
www.microsoft.com/teched www.microsoft.com/learning
http://microsoft.com/technet http://microsoft.com/msdn
http://northamerica.msteched.com
Taking High Performance Computing to the Cloud: Windows HPC and
Taking High Performance Computing to the Cloud: Windows HPC and
Taking High Performance Computing to the Cloud: Windows HPC and
Taking High Performance Computing to the Cloud: Windows HPC and

Weitere ähnliche Inhalte

Was ist angesagt?

How to Run TensorFlow Cheaper in the Cloud Using Elastic GPUs
How to Run TensorFlow Cheaper in the Cloud Using Elastic GPUsHow to Run TensorFlow Cheaper in the Cloud Using Elastic GPUs
How to Run TensorFlow Cheaper in the Cloud Using Elastic GPUsAltoros
 
High Performance Computing: an Introduction for the Society of Actuaries
High Performance Computing: an Introduction for the Society of ActuariesHigh Performance Computing: an Introduction for the Society of Actuaries
High Performance Computing: an Introduction for the Society of ActuariesAdam DeConinck
 
High Performance Computing
High Performance ComputingHigh Performance Computing
High Performance ComputingDell World
 
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic SystemTimely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic SystemAccumulo Summit
 
Experiments with Complex Scientific Applications on Hybrid Cloud Infrastructures
Experiments with Complex Scientific Applications on Hybrid Cloud InfrastructuresExperiments with Complex Scientific Applications on Hybrid Cloud Infrastructures
Experiments with Complex Scientific Applications on Hybrid Cloud InfrastructuresRafael Ferreira da Silva
 
Scientific Computing With Amazon Web Services
Scientific Computing With Amazon Web ServicesScientific Computing With Amazon Web Services
Scientific Computing With Amazon Web ServicesJamie Kinney
 
Accumulo and the Convergence of Machine Learning, Big Data, and Supercomputing
Accumulo and the Convergence of Machine Learning, Big Data, and SupercomputingAccumulo and the Convergence of Machine Learning, Big Data, and Supercomputing
Accumulo and the Convergence of Machine Learning, Big Data, and SupercomputingAccumulo Summit
 
SCasia 2018 MSFT hands on session for Azure Batch AI
SCasia 2018 MSFT hands on session for Azure Batch AISCasia 2018 MSFT hands on session for Azure Batch AI
SCasia 2018 MSFT hands on session for Azure Batch AIHiroshi Tanaka
 
Introduction to High Performance Computing
Introduction to High Performance ComputingIntroduction to High Performance Computing
Introduction to High Performance ComputingUmarudin Zaenuri
 
Iris: Inter-cloud Resource Integration System for Elastic Cloud Data Center
Iris: Inter-cloud Resource Integration System for Elastic Cloud Data CenterIris: Inter-cloud Resource Integration System for Elastic Cloud Data Center
Iris: Inter-cloud Resource Integration System for Elastic Cloud Data CenterRyousei Takano
 
High performance computing for research
High performance computing for researchHigh performance computing for research
High performance computing for researchEsteban Hernandez
 
AIST Super Green Cloud: lessons learned from the operation and the performanc...
AIST Super Green Cloud: lessons learned from the operation and the performanc...AIST Super Green Cloud: lessons learned from the operation and the performanc...
AIST Super Green Cloud: lessons learned from the operation and the performanc...Ryousei Takano
 
Job sequence scheduling for cloud computing
Job sequence scheduling for cloud computingJob sequence scheduling for cloud computing
Job sequence scheduling for cloud computingSamruddhi Gaikwad
 
Serverless Data Architecture at scale on Google Cloud Platform
Serverless Data Architecture at scale on Google Cloud PlatformServerless Data Architecture at scale on Google Cloud Platform
Serverless Data Architecture at scale on Google Cloud PlatformMeetupDataScienceRoma
 
HPC in the Cloud
HPC in the CloudHPC in the Cloud
HPC in the CloudGuy Tel-Zur
 
What's New in H2O Driverless AI? - Arno Candel - H2O AI World London 2018
What's New in H2O Driverless AI? - Arno Candel - H2O AI World London 2018What's New in H2O Driverless AI? - Arno Candel - H2O AI World London 2018
What's New in H2O Driverless AI? - Arno Candel - H2O AI World London 2018Sri Ambati
 
(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014
(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014
(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014Amazon Web Services
 
High Performance Computing in the Cloud?
High Performance Computing in the Cloud?High Performance Computing in the Cloud?
High Performance Computing in the Cloud?Ian Lumb
 
Introduce_non-volatile_generic_object_programming_model_for_In-Memory_Computing
Introduce_non-volatile_generic_object_programming_model_for_In-Memory_ComputingIntroduce_non-volatile_generic_object_programming_model_for_In-Memory_Computing
Introduce_non-volatile_generic_object_programming_model_for_In-Memory_ComputingYanpingWang
 

Was ist angesagt? (20)

How to Run TensorFlow Cheaper in the Cloud Using Elastic GPUs
How to Run TensorFlow Cheaper in the Cloud Using Elastic GPUsHow to Run TensorFlow Cheaper in the Cloud Using Elastic GPUs
How to Run TensorFlow Cheaper in the Cloud Using Elastic GPUs
 
High Performance Computing: an Introduction for the Society of Actuaries
High Performance Computing: an Introduction for the Society of ActuariesHigh Performance Computing: an Introduction for the Society of Actuaries
High Performance Computing: an Introduction for the Society of Actuaries
 
High Performance Computing
High Performance ComputingHigh Performance Computing
High Performance Computing
 
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic SystemTimely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
Timely Year Two: Lessons Learned Building a Scalable Metrics Analytic System
 
Experiments with Complex Scientific Applications on Hybrid Cloud Infrastructures
Experiments with Complex Scientific Applications on Hybrid Cloud InfrastructuresExperiments with Complex Scientific Applications on Hybrid Cloud Infrastructures
Experiments with Complex Scientific Applications on Hybrid Cloud Infrastructures
 
Scientific Computing With Amazon Web Services
Scientific Computing With Amazon Web ServicesScientific Computing With Amazon Web Services
Scientific Computing With Amazon Web Services
 
Accumulo and the Convergence of Machine Learning, Big Data, and Supercomputing
Accumulo and the Convergence of Machine Learning, Big Data, and SupercomputingAccumulo and the Convergence of Machine Learning, Big Data, and Supercomputing
Accumulo and the Convergence of Machine Learning, Big Data, and Supercomputing
 
SCasia 2018 MSFT hands on session for Azure Batch AI
SCasia 2018 MSFT hands on session for Azure Batch AISCasia 2018 MSFT hands on session for Azure Batch AI
SCasia 2018 MSFT hands on session for Azure Batch AI
 
Introduction to High Performance Computing
Introduction to High Performance ComputingIntroduction to High Performance Computing
Introduction to High Performance Computing
 
Iris: Inter-cloud Resource Integration System for Elastic Cloud Data Center
Iris: Inter-cloud Resource Integration System for Elastic Cloud Data CenterIris: Inter-cloud Resource Integration System for Elastic Cloud Data Center
Iris: Inter-cloud Resource Integration System for Elastic Cloud Data Center
 
High performance computing for research
High performance computing for researchHigh performance computing for research
High performance computing for research
 
AIST Super Green Cloud: lessons learned from the operation and the performanc...
AIST Super Green Cloud: lessons learned from the operation and the performanc...AIST Super Green Cloud: lessons learned from the operation and the performanc...
AIST Super Green Cloud: lessons learned from the operation and the performanc...
 
Job sequence scheduling for cloud computing
Job sequence scheduling for cloud computingJob sequence scheduling for cloud computing
Job sequence scheduling for cloud computing
 
Serverless Data Architecture at scale on Google Cloud Platform
Serverless Data Architecture at scale on Google Cloud PlatformServerless Data Architecture at scale on Google Cloud Platform
Serverless Data Architecture at scale on Google Cloud Platform
 
HPC in the Cloud
HPC in the CloudHPC in the Cloud
HPC in the Cloud
 
What's New in H2O Driverless AI? - Arno Candel - H2O AI World London 2018
What's New in H2O Driverless AI? - Arno Candel - H2O AI World London 2018What's New in H2O Driverless AI? - Arno Candel - H2O AI World London 2018
What's New in H2O Driverless AI? - Arno Candel - H2O AI World London 2018
 
(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014
(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014
(BDT202) HPC Now Means 'High Personal Computing' | AWS re:Invent 2014
 
Openstack
OpenstackOpenstack
Openstack
 
High Performance Computing in the Cloud?
High Performance Computing in the Cloud?High Performance Computing in the Cloud?
High Performance Computing in the Cloud?
 
Introduce_non-volatile_generic_object_programming_model_for_In-Memory_Computing
Introduce_non-volatile_generic_object_programming_model_for_In-Memory_ComputingIntroduce_non-volatile_generic_object_programming_model_for_In-Memory_Computing
Introduce_non-volatile_generic_object_programming_model_for_In-Memory_Computing
 

Andere mochten auch

Innholders Presentation 27.01.15
Innholders Presentation 27.01.15Innholders Presentation 27.01.15
Innholders Presentation 27.01.15Elena Leva
 
презентация ра алего Www.alego.ru
презентация ра алего Www.alego.ruпрезентация ра алего Www.alego.ru
презентация ра алего Www.alego.rum3alego
 
Digital Health & Wellness Summit @ Mobile World Congress 2016
Digital Health & Wellness Summit @ Mobile World Congress 2016Digital Health & Wellness Summit @ Mobile World Congress 2016
Digital Health & Wellness Summit @ Mobile World Congress 20163GDR
 
Social Media Analysis Pharma Pd Fvs2
Social Media Analysis Pharma Pd Fvs2Social Media Analysis Pharma Pd Fvs2
Social Media Analysis Pharma Pd Fvs2CraigSullivan72
 
Charting the Course for the Future: Kantar Retail's Big Idea
Charting the Course for the Future: Kantar Retail's Big IdeaCharting the Course for the Future: Kantar Retail's Big Idea
Charting the Course for the Future: Kantar Retail's Big IdeaJeff Cushing
 
Do You Have Big Data? (Most Likely!)
Do You Have Big Data? (Most Likely!)Do You Have Big Data? (Most Likely!)
Do You Have Big Data? (Most Likely!)Saptak Sen
 
Suman Shome Presentation
Suman Shome PresentationSuman Shome Presentation
Suman Shome Presentationsuman Shome
 
Statistics report on Socio-economic background of Investors
Statistics report on Socio-economic background of InvestorsStatistics report on Socio-economic background of Investors
Statistics report on Socio-economic background of InvestorsPranab Ghosh
 
Dmovat resume June 16- 6
Dmovat resume June 16- 6Dmovat resume June 16- 6
Dmovat resume June 16- 6Douglas Movat
 

Andere mochten auch (12)

Shane CV 2015
Shane CV 2015Shane CV 2015
Shane CV 2015
 
Innholders Presentation 27.01.15
Innholders Presentation 27.01.15Innholders Presentation 27.01.15
Innholders Presentation 27.01.15
 
презентация ра алего Www.alego.ru
презентация ра алего Www.alego.ruпрезентация ра алего Www.alego.ru
презентация ра алего Www.alego.ru
 
Digital Health & Wellness Summit @ Mobile World Congress 2016
Digital Health & Wellness Summit @ Mobile World Congress 2016Digital Health & Wellness Summit @ Mobile World Congress 2016
Digital Health & Wellness Summit @ Mobile World Congress 2016
 
Els vertebrats
Els vertebratsEls vertebrats
Els vertebrats
 
Social Media Analysis Pharma Pd Fvs2
Social Media Analysis Pharma Pd Fvs2Social Media Analysis Pharma Pd Fvs2
Social Media Analysis Pharma Pd Fvs2
 
Charting the Course for the Future: Kantar Retail's Big Idea
Charting the Course for the Future: Kantar Retail's Big IdeaCharting the Course for the Future: Kantar Retail's Big Idea
Charting the Course for the Future: Kantar Retail's Big Idea
 
Do You Have Big Data? (Most Likely!)
Do You Have Big Data? (Most Likely!)Do You Have Big Data? (Most Likely!)
Do You Have Big Data? (Most Likely!)
 
Suman Shome Presentation
Suman Shome PresentationSuman Shome Presentation
Suman Shome Presentation
 
Statistics report on Socio-economic background of Investors
Statistics report on Socio-economic background of InvestorsStatistics report on Socio-economic background of Investors
Statistics report on Socio-economic background of Investors
 
Dmovat resume June 16- 6
Dmovat resume June 16- 6Dmovat resume June 16- 6
Dmovat resume June 16- 6
 
Animals invertebrats
Animals invertebratsAnimals invertebrats
Animals invertebrats
 

Ähnlich wie Taking High Performance Computing to the Cloud: Windows HPC and

High Performance Computing Pitch Deck
High Performance Computing Pitch DeckHigh Performance Computing Pitch Deck
High Performance Computing Pitch DeckNicholas Vossburg
 
Solving enterprise challenges through scale out storage & big compute final
Solving enterprise challenges through scale out storage & big compute finalSolving enterprise challenges through scale out storage & big compute final
Solving enterprise challenges through scale out storage & big compute finalAvere Systems
 
Managing and Deploying High Performance Computing Clusters using Windows HPC ...
Managing and Deploying High Performance Computing Clusters using Windows HPC ...Managing and Deploying High Performance Computing Clusters using Windows HPC ...
Managing and Deploying High Performance Computing Clusters using Windows HPC ...Saptak Sen
 
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...Qubole
 
Building FoundationDB
Building FoundationDBBuilding FoundationDB
Building FoundationDBFoundationDB
 
Azure Overview Csco
Azure Overview CscoAzure Overview Csco
Azure Overview Cscorajramab
 
2014.10.22 Building Azure Solutions with Office 365
2014.10.22 Building Azure Solutions with Office 3652014.10.22 Building Azure Solutions with Office 365
2014.10.22 Building Azure Solutions with Office 365Marco Parenzan
 
FInal Project - USMx CC605x Cloud Computing for Enterprises - Hugo Aquino
FInal Project - USMx CC605x Cloud Computing for Enterprises - Hugo AquinoFInal Project - USMx CC605x Cloud Computing for Enterprises - Hugo Aquino
FInal Project - USMx CC605x Cloud Computing for Enterprises - Hugo AquinoHugo Aquino
 
Mainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft AzureMainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft AzurePrecisely
 
High Performance Computing on AWS: Accelerating Innovation with virtually unl...
High Performance Computing on AWS: Accelerating Innovation with virtually unl...High Performance Computing on AWS: Accelerating Innovation with virtually unl...
High Performance Computing on AWS: Accelerating Innovation with virtually unl...Amazon Web Services
 
Cloud computing shim
Cloud computing shimCloud computing shim
Cloud computing shimtistrue
 
Citi Tech Talk: Hybrid Cloud
Citi Tech Talk: Hybrid CloudCiti Tech Talk: Hybrid Cloud
Citi Tech Talk: Hybrid Cloudconfluent
 
The Hidden Value of Hadoop Migration
The Hidden Value of Hadoop MigrationThe Hidden Value of Hadoop Migration
The Hidden Value of Hadoop MigrationDatabricks
 
Amf304 optimizing-design-and-e-660cc73d-5c4c-4331-8f59-48cccdc1b7f4-135588426...
Amf304 optimizing-design-and-e-660cc73d-5c4c-4331-8f59-48cccdc1b7f4-135588426...Amf304 optimizing-design-and-e-660cc73d-5c4c-4331-8f59-48cccdc1b7f4-135588426...
Amf304 optimizing-design-and-e-660cc73d-5c4c-4331-8f59-48cccdc1b7f4-135588426...Ramprasad Nagaraja
 
Windowsazureplatform Overviewlatest
Windowsazureplatform OverviewlatestWindowsazureplatform Overviewlatest
Windowsazureplatform Overviewlatestrajramab
 
Understanding The Azure Platform Jan
Understanding The Azure Platform   JanUnderstanding The Azure Platform   Jan
Understanding The Azure Platform JanDavidGristwood
 
Cloud computing - dien toan dam may
Cloud computing - dien toan dam mayCloud computing - dien toan dam may
Cloud computing - dien toan dam mayNguyen Duong
 
How to Choose a Host for a Big Data Project
How to Choose a Host for a Big Data ProjectHow to Choose a Host for a Big Data Project
How to Choose a Host for a Big Data ProjectPeak Hosting
 

Ähnlich wie Taking High Performance Computing to the Cloud: Windows HPC and (20)

High Performance Computing Pitch Deck
High Performance Computing Pitch DeckHigh Performance Computing Pitch Deck
High Performance Computing Pitch Deck
 
Solving enterprise challenges through scale out storage & big compute final
Solving enterprise challenges through scale out storage & big compute finalSolving enterprise challenges through scale out storage & big compute final
Solving enterprise challenges through scale out storage & big compute final
 
Managing and Deploying High Performance Computing Clusters using Windows HPC ...
Managing and Deploying High Performance Computing Clusters using Windows HPC ...Managing and Deploying High Performance Computing Clusters using Windows HPC ...
Managing and Deploying High Performance Computing Clusters using Windows HPC ...
 
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...
Data Warehouse Modernization - Big Data in the Cloud Success with Qubole on O...
 
Building FoundationDB
Building FoundationDBBuilding FoundationDB
Building FoundationDB
 
Azure Overview Csco
Azure Overview CscoAzure Overview Csco
Azure Overview Csco
 
2014.10.22 Building Azure Solutions with Office 365
2014.10.22 Building Azure Solutions with Office 3652014.10.22 Building Azure Solutions with Office 365
2014.10.22 Building Azure Solutions with Office 365
 
FInal Project - USMx CC605x Cloud Computing for Enterprises - Hugo Aquino
FInal Project - USMx CC605x Cloud Computing for Enterprises - Hugo AquinoFInal Project - USMx CC605x Cloud Computing for Enterprises - Hugo Aquino
FInal Project - USMx CC605x Cloud Computing for Enterprises - Hugo Aquino
 
Mainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft AzureMainframe Modernization with Precisely and Microsoft Azure
Mainframe Modernization with Precisely and Microsoft Azure
 
High Performance Computing on AWS: Accelerating Innovation with virtually unl...
High Performance Computing on AWS: Accelerating Innovation with virtually unl...High Performance Computing on AWS: Accelerating Innovation with virtually unl...
High Performance Computing on AWS: Accelerating Innovation with virtually unl...
 
Cloud computing shim
Cloud computing shimCloud computing shim
Cloud computing shim
 
Citi Tech Talk: Hybrid Cloud
Citi Tech Talk: Hybrid CloudCiti Tech Talk: Hybrid Cloud
Citi Tech Talk: Hybrid Cloud
 
The Hidden Value of Hadoop Migration
The Hidden Value of Hadoop MigrationThe Hidden Value of Hadoop Migration
The Hidden Value of Hadoop Migration
 
Amf304 optimizing-design-and-e-660cc73d-5c4c-4331-8f59-48cccdc1b7f4-135588426...
Amf304 optimizing-design-and-e-660cc73d-5c4c-4331-8f59-48cccdc1b7f4-135588426...Amf304 optimizing-design-and-e-660cc73d-5c4c-4331-8f59-48cccdc1b7f4-135588426...
Amf304 optimizing-design-and-e-660cc73d-5c4c-4331-8f59-48cccdc1b7f4-135588426...
 
Self-Service Supercomputing
Self-Service SupercomputingSelf-Service Supercomputing
Self-Service Supercomputing
 
Windowsazureplatform Overviewlatest
Windowsazureplatform OverviewlatestWindowsazureplatform Overviewlatest
Windowsazureplatform Overviewlatest
 
Understanding The Azure Platform Jan
Understanding The Azure Platform   JanUnderstanding The Azure Platform   Jan
Understanding The Azure Platform Jan
 
Cloud computing - dien toan dam may
Cloud computing - dien toan dam mayCloud computing - dien toan dam may
Cloud computing - dien toan dam may
 
How to Choose a Host for a Big Data Project
How to Choose a Host for a Big Data ProjectHow to Choose a Host for a Big Data Project
How to Choose a Host for a Big Data Project
 
AWS for HPC in Drug Discovery
AWS for HPC in Drug DiscoveryAWS for HPC in Drug Discovery
AWS for HPC in Drug Discovery
 

Mehr von Saptak Sen

Apache Spark with Hortonworks Data Platform - Seattle Meetup
Apache Spark with Hortonworks Data Platform - Seattle MeetupApache Spark with Hortonworks Data Platform - Seattle Meetup
Apache Spark with Hortonworks Data Platform - Seattle MeetupSaptak Sen
 
Introduction to Apache NiFi - Seattle Scalability Meetup
Introduction to Apache NiFi - Seattle Scalability MeetupIntroduction to Apache NiFi - Seattle Scalability Meetup
Introduction to Apache NiFi - Seattle Scalability MeetupSaptak Sen
 
Data Management in Microsoft HDInsight: How to Move and Store Your Data
Data Management in Microsoft HDInsight: How to Move and Store Your DataData Management in Microsoft HDInsight: How to Move and Store Your Data
Data Management in Microsoft HDInsight: How to Move and Store Your DataSaptak Sen
 
LINQ to HPC: Developing Big Data Applications on Windows HPC Server
LINQ to HPC: Developing Big Data Applications on Windows HPC ServerLINQ to HPC: Developing Big Data Applications on Windows HPC Server
LINQ to HPC: Developing Big Data Applications on Windows HPC ServerSaptak Sen
 
Predictive Analytics with Microsoft Big Data
Predictive Analytics with Microsoft Big DataPredictive Analytics with Microsoft Big Data
Predictive Analytics with Microsoft Big DataSaptak Sen
 
Data Management in Microsoft HDInsight: How to Move and Store Your Data
Data Management in Microsoft HDInsight: How to Move and Store Your DataData Management in Microsoft HDInsight: How to Move and Store Your Data
Data Management in Microsoft HDInsight: How to Move and Store Your DataSaptak Sen
 
Apache Spark Workshop at Hadoop Summit
Apache Spark Workshop at Hadoop SummitApache Spark Workshop at Hadoop Summit
Apache Spark Workshop at Hadoop SummitSaptak Sen
 

Mehr von Saptak Sen (7)

Apache Spark with Hortonworks Data Platform - Seattle Meetup
Apache Spark with Hortonworks Data Platform - Seattle MeetupApache Spark with Hortonworks Data Platform - Seattle Meetup
Apache Spark with Hortonworks Data Platform - Seattle Meetup
 
Introduction to Apache NiFi - Seattle Scalability Meetup
Introduction to Apache NiFi - Seattle Scalability MeetupIntroduction to Apache NiFi - Seattle Scalability Meetup
Introduction to Apache NiFi - Seattle Scalability Meetup
 
Data Management in Microsoft HDInsight: How to Move and Store Your Data
Data Management in Microsoft HDInsight: How to Move and Store Your DataData Management in Microsoft HDInsight: How to Move and Store Your Data
Data Management in Microsoft HDInsight: How to Move and Store Your Data
 
LINQ to HPC: Developing Big Data Applications on Windows HPC Server
LINQ to HPC: Developing Big Data Applications on Windows HPC ServerLINQ to HPC: Developing Big Data Applications on Windows HPC Server
LINQ to HPC: Developing Big Data Applications on Windows HPC Server
 
Predictive Analytics with Microsoft Big Data
Predictive Analytics with Microsoft Big DataPredictive Analytics with Microsoft Big Data
Predictive Analytics with Microsoft Big Data
 
Data Management in Microsoft HDInsight: How to Move and Store Your Data
Data Management in Microsoft HDInsight: How to Move and Store Your DataData Management in Microsoft HDInsight: How to Move and Store Your Data
Data Management in Microsoft HDInsight: How to Move and Store Your Data
 
Apache Spark Workshop at Hadoop Summit
Apache Spark Workshop at Hadoop SummitApache Spark Workshop at Hadoop Summit
Apache Spark Workshop at Hadoop Summit
 

Kürzlich hochgeladen

Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsJaydeep Chhasatia
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native BuildpacksVish Abrams
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.Sharon Liu
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIIvo Andreev
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionsNirav Modi
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdfMeon Technology
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?AmeliaSmith90
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorShane Coughlan
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxAutus Cyber Tech
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageDista
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampVICTOR MAESTRE RAMIREZ
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadIvo Andreev
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxJoão Esperancinha
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntelliSource Technologies
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...OnePlan Solutions
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmonyelliciumsolutionspun
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Jaydeep Chhasatia
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfTobias Schneck
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyRaymond Okyere-Forson
 

Kürzlich hochgeladen (20)

Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software TeamsYour Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
Your Vision, Our Expertise: TECUNIQUE's Tailored Software Teams
 
Streamlining Your Application Builds with Cloud Native Buildpacks
Streamlining Your Application Builds  with Cloud Native BuildpacksStreamlining Your Application Builds  with Cloud Native Buildpacks
Streamlining Your Application Builds with Cloud Native Buildpacks
 
Kawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in TrivandrumKawika Technologies pvt ltd Software Development Company in Trivandrum
Kawika Technologies pvt ltd Software Development Company in Trivandrum
 
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
20240319 Car Simulator Plan.pptx . Plan for a JavaScript Car Driving Simulator.
 
JS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AIJS-Experts - Cybersecurity for Generative AI
JS-Experts - Cybersecurity for Generative AI
 
eAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspectionseAuditor Audits & Inspections - conduct field inspections
eAuditor Audits & Inspections - conduct field inspections
 
online pdf editor software solutions.pdf
online pdf editor software solutions.pdfonline pdf editor software solutions.pdf
online pdf editor software solutions.pdf
 
How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?How Does the Epitome of Spyware Differ from Other Malicious Software?
How Does the Epitome of Spyware Differ from Other Malicious Software?
 
OpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS CalculatorOpenChain Webinar: Universal CVSS Calculator
OpenChain Webinar: Universal CVSS Calculator
 
ERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptxERP For Electrical and Electronics manufecturing.pptx
ERP For Electrical and Electronics manufecturing.pptx
 
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales CoverageSales Territory Management: A Definitive Guide to Expand Sales Coverage
Sales Territory Management: A Definitive Guide to Expand Sales Coverage
 
Deep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - DatacampDeep Learning for Images with PyTorch - Datacamp
Deep Learning for Images with PyTorch - Datacamp
 
Cybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and BadCybersecurity Challenges with Generative AI - for Good and Bad
Cybersecurity Challenges with Generative AI - for Good and Bad
 
Fields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptxFields in Java and Kotlin and what to expect.pptx
Fields in Java and Kotlin and what to expect.pptx
 
Introduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptxIntroduction-to-Software-Development-Outsourcing.pptx
Introduction-to-Software-Development-Outsourcing.pptx
 
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
Transforming PMO Success with AI - Discover OnePlan Strategic Portfolio Work ...
 
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine HarmonyLeveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
Leveraging DxSherpa's Generative AI Services to Unlock Human-Machine Harmony
 
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
Optimizing Business Potential: A Guide to Outsourcing Engineering Services in...
 
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdfARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
ARM Talk @ Rejekts - Will ARM be the new Mainstream in our Data Centers_.pdf
 
AI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human BeautyAI Embracing Every Shade of Human Beauty
AI Embracing Every Shade of Human Beauty
 

Taking High Performance Computing to the Cloud: Windows HPC and

  • 4. massively parallel Becoming critical part of R&D for businesses Traditionally required specialized staffing X64 Server 1991 1998 2008 $40,000,000 $1,000,000 $1,000
  • 6. • Advances in technical computing and research are powered by access to computing resources Barrier to entry for technical workloads How Do we take existing technical workloads onto azure?
  • 7. My CRAY CX1 Windows Azure Only Initial cost: Planning, Hardware, Space, Power, Cooling, $10000s Get an account. Cost of ownership: Maintenance, Staff, Power, Backups, Internet, Storage, Down time, Disk failures, Paying for idle computing! Pay as you go, limited only by your budget Suitable Applications: MPI problems require high speed network. HPC Batch Scheduler System Applications without much traffic. No HPC batch submission engine Node interconnects: 20 Gbps Infini-band! 1 Gbps Ethernet As an officemate: No! Silent and Invisible CPU & Memory Hardware: Dedicated 8 nodes with 8 cores &16gb N+1 nodes 8 cores &16gb Access to My data: LAN Connection Internet Connection first upload LAN Connection if already in blob
  • 8. On-premise Desktop User HPC Head Node Broker Node(s) HPC Cluster Desktop Compute Cloud via Idle Win 7 Workstation Cores Azure Compute Instances Azure Compute Proxies
  • 11. … Fabric Compute Storage Application Fabric A closer look VM Web Role Worker Role Agent Agent main() { … } Load Balancer HTTP IIS ASP.NET, WCF, etc.
  • 12. demo
  • 17. Windows HPC on Azure Mixed Mode Deployment On-premise andAzure On-premise • “Burst” into cloud on-demand while keeping control over data and corporate policies • Pay only for what you use • A stepping stone to hybrid and public clouds. • Dynamically adjust how much runs on-premise and in the cloud • Azure computer instances include both worker role and VM role Azure Compute Nodes DesktopsHPC Head Node Broker Node Compute Nodes On-Premise and in Azure Simultaneously Azure Compute Instances Compute Proxies
  • 20. Shape Optimization with Sculptor Cut design times Cut design costs Provide better products Geometric optimization New trend in design
  • 21. compute what you can afford Latin hyperCube sampling A Statistical method Know your cost up front Know you time to solution More design insight Best statistical accuracy
  • 22. Change shape of intake Improve flows Better gas mileage Better performance
  • 23. demo MPI on-premises + data shared in Azure
  • 28. DeveloperTools Compilers: • PGI • Intel • GCC Build: • CMake • Visual Studio Unix Env: • Cygwin • SUA • MinGW Libraries Solvers: • PETSc* • Trilinos • FFTW IO: • HDF5MPI • NetCDF PopularHPCApps Chemistry Bio Science* CFD Structure Climate* Physics Data Mining Computer Science AnalysisTools Parallel Visualization • VisIT* • ParaView* Python • Ipython* • NumPY • SciPY
  • 29. HPC Server 2008 R2 Roadmap Microsoft HPC Server 2008 R2 SP1 & LINQtoHPC CTP  Integration with Windows Azure:  “Burst” SOA and Parametric sweep jobs to Azure as worker roles.  All compute nodes in Azure with head-node only on-premise.  User activity scheduling for workstation nodes  C# cluster SOA debugger for VS 2010. Microsoft HPC Server 2008 R2  Excel on HPC clusters  Schedule jobs to workstations  Business critical SOA  Scale to 1000+ nodes out-of-the-box  Dual boot clusters  Diskless compute nodes  Extensible diagnostics and troubleshooting framework  Flexible, customizable color-coded Heat map Microsoft HPC Server 2008 R2 SP2*  LINQ to HPC for large scale data analytics  MPI runtime in Windows Azure  Azure VM role  New job scheduler policies  Lend/Borrow resource pools  Over/under subscribe nodes  Common data for SOA Apps  Multi-domain support for workstation nodes  Web Portal for job submission and job status  Smart Card authentication July 2010 Dec 2010 June 2011
  • 30. try http://blogs.msdn.com/hpctrekker Application Models and Data Considerations http://connect.microsoft.com free Trial of Azure for 30 days try out the Azure Burst Samples
  • 31. Related Content WSV205 | Dryad: Running “Big Data” Applications on a Windows HPC Server Cluster WSV207 | Distributed Scale-Out Applications On-Premises and in the Cloud: An Introduction to Windows HPC Server WSV472-INT | Building Scale Out Clusters with Windows HPC Server 2008 R2: Cluster Deployment, Troubleshooting and Basic Job Submission Techniques Product Demo Stations (Windows HPC Server 2008 R2 SP2) Find Me Later At…Server HPC Booth Thursday 10:30am – 1:00pm
  • 32. My Blog: http://blogs.msdn.com/hpctrekker The Azure Burst Samples and White paper Windows HPC ResourceKit Windows HPC Server TechCenter