Intermachine Parallelism

Folding@home distributed computing on PS3 now contributes 2/3rds of total performance (1050/1452 TFLOPS) but only 1/6 (~0.5M/~3M) CPUs in project. GPUs have even more impressive performance. 200W avg for PS3. inst.eecs.berkeley.edu/~cs61c UCB CS61C : Machine Structures Lecture 42 – Inter-machine Parallelism 2008-05-09 Lecturer SOE Dan Garcia http://fah-web.stanford.edu/cgi-bin/main.py?qtype=osstats

Review ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],en.wikipedia.org/wiki/Image:AmdahlsLaw.svg

Today’s Outline ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Big Problems ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Example Applications ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Performance Requirements ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],www.epm.ornl.gov/chammp/chammp.html Reference:http://www.hpcwire.com/hpcwire/hpcwireWWW/04/0827/108259.html

High Resolution Climate Modeling on NERSC-3 P. Duffy, et al., LLNL

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],What Can We Do? Use Many CPUs!

Distributed Computing Themes ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Distributed Computing Challenges ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Programming Models: What is MPI? ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],http://www.mpi-forum.org/ http://forum.stanford.edu/events/2007/plenary/slides/Olukotun.ppt http://www.tbray.org/ongoing/When/200x/2006/05/24/On-Grids

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Challenges with MPI

A New Hope: Google’s MapReduce ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],MapReduce Programming Model code.google.com/edu/parallel/mapreduce-tutorial.html

MapReduce WordCount Example ,[object Object],[object Object],[object Object],[object Object],[object Object]

MapReduce WordCount Diagram ah ah er ah if or or uh or ah if ah:1,1,1,1 ah:1 if:1 or:1 or:1 uh:1 or:1 ah:1 if:1 er:1 if:1,1 or:1,1,1 uh:1 ah:1 ah:1 er:1 4 1 2 3 1 file 1 file 2 file 3 file 4 file 5 file 6 file 7 (ah) (er) (if) (or) (uh) map (String input_key, String input_value): // input_key : doc name // input_value: doc contents for each word w in input_value: EmitIntermediate(w, "1"); reduce (String output_key, Iterator intermediate_values): // output_key : a word // output_values: a list of counts int result = 0; for each v in intermediate_values: result += ParseInt(v); Emit(AsString(result));

MapReduce in CS61A (and CS3?!) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Our Scheme MapReduce interface

MapReduce Advantages/Disadvantages ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Peer Instruction ABC 0: FFF 1: FF T 2: F T F 3: F TT 4: T FF 5: T F T 6: TT F 7: TTT ,[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],Peer Instruction Answer ,[object Object],[object Object],[object Object],ABC 0: FFF 1: FF T 2: F T F 3: F TT 4: T FF 5: T F T 6: TT F 7: TTT

Summary ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

Bonus slides ,[object Object],[object Object],Bonus

To Learn More… ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],Basic MPI Functions (1)

[object Object],[object Object],[object Object],[object Object],[object Object],Basic MPI Functions (2)

Basic MPI Functions (3) ,[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object],[object Object]

[object Object],[object Object],[object Object],[object Object],[object Object],MPI Program Template

Intermachine Parallelism

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Intermachine Parallelism

Ähnlich wie Intermachine Parallelism (20)

Mehr von Sri Prasanna

Mehr von Sri Prasanna (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Intermachine Parallelism