1. Patterns of Parallel
Programming
Prepared by Yan Drugalya
ydrugalya@gmail.com
@ydrugalya
2. Agenda
• Why parallel?
• Terms and measures
• Building Blocks
• Patterns overview
– Pipeline and data flow
– Producer-Consumer
– Map-Reduce
– Other
3. Why Moore's law is not working
anymore
• Power consumption
• Wire delays
• DRAM access latency
• Diminishing returns of more instruction-level
parallelism
4. Power consumption
Sun’s Surface
10,000
1,000 Rocket Nozzle
Power Density (W/cm2)
100 Nuclear Reactor
10 Pentium® processors
Hot Plate
1
8080
‘70 ‘80 ’90 ’00 ‘10
10. Definitions
• Concurrent
- Several things happenings at the same time
• Multithreaded
– Multiple execution contexts
• Parallel
– Multiple simultaneous computations
• Asynchronous
– Not having to wait
14. Fork-Join
• Additional work may be started only when specific subsets of
the original elements have completed processing
• All elements should be given the chance to run even if one
invocation fails (Ping)
Parallel.Invoke(
() => ComputeMean(),
Fork () => ComputeMedian(),
() => ComputeMode());
Compute Compute Compute static void MyParallelInvoke(params Action[] actions)
Median Mean Mode {
var tasks = new Task[actions.Length];
for (int i = 0; i < actions.Length; i++)
tasks[i] = Task.Factory.StartNew(actions[i]);
Join Task.WaitAll(tasks);
}
19. References
• Patterns for Parallel Programming: Understanding and Applying
Parallel Patterns with the .NET Framework 4
• Pluralsight:
– Introduction to Async and Parallel Programming in .NET 4
– Async and Parallel Programming: Application Design
• The Free Lunch Is Over: A Fundamental Turn Toward
Concurrency in Software
• Chapter 27 Multithreaded Algorithms from Introduction to
algorithms 3rd edition