"Multithreading" lecture @ ELEKS Summer School 2012 (Jul 2012)

1. Summer School 2012
Multithreading
Yuriy Guts
R&D Engineer
yuriy.guts@eleks.com

2. Summer School 2012
“ Although threads seem to be a small step from sequential computation,
in fact, they represent a huge step.
They discard the most essential and appealing properties of sequential
computation: understandability, predictability, and determinism.
Threads, as a model of computation, are wildly nondeterministic, and
the job of the programmer becomes one of pruning that nondeterminism.
”
Edward A. Lee, UC Berkeley, 2006

3. Summer School 2012
History: so why did we need threads?
• CPU virtualization
• Increased robustness
• Quazi-multitasking

4. Summer School 2012
Process as a resource isolation unit

5. Summer School 2012
Concurrent execution
does not exist
on a single logical CPU!

6. Summer School 2012
Context Switching

7. Summer School 2012
Scheduling
• Processes are not scheduled. Threads are.
• 32 different thread priority levels!
Thread Process
1. Idle 1. Idle
2. Lowest 2. Below Normal
3. Below Normal 3. Normal
4. Normal 4. Above Normal
5. Above Normal 5. High
6. Highest 6. Realtime
7. Time-Critical

8. Summer School 2012
Thread Overhead
• Thread Kernel Object
• Thread Environment Block (TEB)
• User-Mode Stack [1 MB]
• Kernel-Mode Stack [12 / 24 KB]
• DLL: Attach/Detach Notifications

9. Summer School 2012
Moore’s Law

10. Summer School 2012
Threads for parallelization?

11. Summer School 2012
Multiple Logical CPUs
• Multi-socket motherboards
• Single-core CPUs with Hyper Threading (HT)
• Multi-core CPUs
• Multi-core CPUs with per-core HT

12. Summer School 2012
Thread Operations
Compute-bound I/O-bound

13. Summer School 2012
Compute-bound Operations
static void Main(string[] args)
{
Thread workerThread = new Thread(Factorial);
workerThread.Start(10);
workerThread.Join();
Console.ReadKey(true);
}
static void Factorial(object arg)
{
int n = (int)arg;
int result = 1;
for (int i = 2; i <= n; i++) result *= i;
Console.WriteLine(result);
}

14. Summer School 2012
Thread Lifecycle

15. Summer School 2012
Too many threads = too bad
• Kernel object overhead
• Memory overhead
• Context switch overhead
• GC overhead
Use threads wisely!

16. Summer School 2012
Thread Pool
• Designed for background processing
• Self-balancing
• Avoids much overhead

17. Summer School 2012
Thread Pool: Compute-bound
static void Main(string[] args)
{
ThreadPool.QueueUserWorkItem(Factorial, 10);
Console.ReadKey(true);
}
static void Factorial(object n)
{
int x = (int)n;
int result = 1;
for (int i = 2; i <= x; i++) result *= i;
Console.WriteLine(result);
}

18. Summer School 2012
Self-Balancing
Too few threads: Too many threads:
Allocate more Remove excessive ones

19. Summer School 2012
Task-Based Threading
“Work Stealing” Principle!

20. Summer School 2012
Tasks: Compute-bound
static void Main(string[] args)
{
Task<Int32> task = new Task<Int32>(Factorial, 10);
task.Start();
task.Wait();
Console.WriteLine("The Sum is: " + task.Result);
Console.ReadKey(true);
}
static int Factorial(object n)
{
int x = (int)n;
int result = 1;
for (int i = 2; i <= x; i++) result *= i;
return result;
}

21. Summer School 2012
Tasks 101
• Each task can have child tasks…
• …and, therefore, throw multiple exceptions
• “Task” does not mean “separate thread”!
• Task templating via TaskFactory
• Custom scheduling via TaskScheduler
• Cooperative cancellation via CancellationToken
• Continuations, WaitAny, WaitAll, …

22. Summer School 2012
Tasks: Continuations
static void Main(string[] args)
{
Task<Int32> task = new Task<Int32>(Factorial, 10);
task.ContinueWith(t => Console.WriteLine("Completed"),
TaskContinuationOptions.OnlyOnRanToCompletion);
task.ContinueWith(t => Console.WriteLine("Canceled"),
TaskContinuationOptions.OnlyOnCanceled);
task.ContinueWith(t => Console.WriteLine("Error"),
TaskContinuationOptions.OnlyOnFaulted);
task.Start();
}

23. Summer School 2012
Parallel LINQ (PLINQ)
enumerable.AsParallel()

24. Summer School 2012
Parallel Computations via PLINQ
static void Main(string[] args)
{
Parallel.For(1, 10, n => Console.WriteLine("{0}: {1}", n, Factorial(n)));
Enumerable.Range(1, 9).AsParallel().ForAll(n => /* Console... */);
new int [] { 1, 2, 3 }.AsParallel().ForAll(n => /* Console... */);
}

25. Summer School 2012
.AsParallel()
does not mean faster execution
• Possible GC pressure ( i => new { ... } )
• Item acquisition and delegate execution are implemented in virtual methods
• You have to benchmark every particular situation

26. Summer School 2012
Synchronous I/O

27. Summer School 2012
Async I/O

28. Summer School 2012
Async I/O Benefits
• Less threads
• Less GC
• Faster debugging
• Faster I/O if made parallel
• Responsive UI (a must for WinPhone & Silverlight)

29. Summer School 2012
Asynchronous Programming Model (APM)
Implemented by:
• Streams
• Sockets
• Serial Ports
• SQL Commands
• DNS Requests
• Web Services
…etc.

30. Summer School 2012
The Async Model Soup
Which one to follow? Stop the madness!

31. Summer School 2012
New APM in .NET 4.5 / C# vNext
private byte[] GetURLContents(string url)
{
var content = new MemoryStream();
var webReq = (HttpWebRequest)WebRequest.Create(url);
using (var response = webReq.GetResponse())
using (Stream responseStream = response.GetResponseStream())
responseStream.CopyTo(content);
return content.ToArray();
}
private async Task<byte[]> GetURLContentsAsync(string url)
{
var content = new MemoryStream();
var webReq = (HttpWebRequest)WebRequest.Create(url);
using (WebResponse response = await webReq.GetResponseAsync())
using (Stream responseStream = response.GetResponseStream())
await responseStream.CopyToAsync(content);
return content.ToArray();
}

32. Summer School 2012
Threads and Shared State

33. Summer School 2012
Threads and Race Conditions

34. Summer School 2012
Synchronization Constructs
User-Mode
• Volatile Read/Write
• Interlocked
Kernel-Mode
• Events
• Semaphores
• …and everything derived from them
Hybrid
• Double-checked locking
• Many others…

35. Summer School 2012
To make things even worse
• Language Compiler optimizations
• JIT Compiler optimizations
• CPU optimizations
– Out-of-order execution
– Branch prediction
– Memory barriers

36. Summer School 2012
Atomic (Interlocked) Operations
• Atomic Swap
• Test-and-Set
• Compare-and-Swap
• Fetch-and-Add
• Load-Link / Store-Conditional

37. Summer School 2012
Kernel-mode Constructs
• WaitHandle (base class)
• AutoResetEvent
• ManualResetEvent
• CountdownEvent
• Semaphore
• Mutex

38. Summer School 2012
Hybrid constructs: double-checked locking
internal sealed class Singleton {
private static readonly Object s_lock = new Object();
private static Singleton s_value = null;
private Singleton() { }
public static Singleton GetSingleton() {
if (s_value != null) return s_value;
Monitor.Enter(s_lock);
if (s_value == null) {
Singleton temp = new Singleton();
Interlocked.Exchange(ref s_value, temp);
}
Monitor.Exit(s_lock);
return s_value;
}
}

39. Summer School 2012
Concurrent Collections
• BlockingCollection<T>
• ConcurrentBag<T>
• ConcurrentDictionary<K, V>
• ConcurrentQueue<T>
• ConcurrentStack<T>

40. Summer School 2012
Building Scalable Applications
• Avoid thread blocking
• Avoid shared state
• Avoid statics
• Avoid mutable structures

41. Summer School 2012
?? ?
Q&A
yuriy.guts@eleks.com

42. Summer School 2012
Thank you!