Multicoreexpo2011 05-05fridiusingtemplate-110518193827-phpapp02

aicas Technology

Multicore for Real-Time
and Safety-Critical Software:
Avoid the Pitfalls

Dr. Fridtjof Siebert, CTO, aicas
Dr. James J. Hunt, CEO aicas
MultiCoreExpo 2011, 5th May 2011

Multi-Core for Real-Time and Safety-Critical Software: Avoid the Pitfalls
Dr. Fridtjof Siebert, CTO, aicas GmbH
Dr. James J. Hunt, CEO, aicas GmbH
1

Agenda
Race conditions Reaching peak
Synchronization performance
CPU affinities
Atomic operations
Multicore scheduling
Memory model
Lock free algorithms
Values out of thin air Compare and Swap

2

Typical Problems on Multicore

typical code sequence (C/C++ or Java)
int counter;

void increment()
{
counter++;
}

3


int counter;

void increment()
{
counter++;
}  r1 = counter;
r2 = r1 + 1;
counter = r2;

4


int counter;

void increment() Thread 1 Thread 2
{
counter++;
}  r1 = counter;
r2 = r1 + 1;
counter = r2;
r1 = counter;
r2 = r1 + 1;
counter = r2;

5


int counter;

void increment() Thread 1 Thread 2
{
counter++;
}  r1 = counter;
r2 = r1 + 1;
counter = r2;
r1 = counter;
r2 = r1 + 1;
counter = r2;

An increment can get lost!

6




int counter;

void increment()
{
counter++;
}

7




int counter;

void increment()
{
counter++;
}
code lacks synchronization
but on a single core, it practically always works!
on a multicore, chances for failure explode!

8

Synchronization

solution: synchronize
int counter;

synchronized void increment()
{
counter++;
}
Easy, problem solved.
Right? See later.

9

Atomic Operations

What is the result of
int a, b; /* 32 bit, initially 0 */

Thread 1 Thread 2
b = a; a = 1;

?

10

Atomic Operations

int a, b; /* 32 bit, initially 0 */

Thread 1 Thread 2
b = a; a = 1;

?
b == 0
b == 1

11

Atomic Operations

long a, b; /* 64 bit, initially 0 */

Thread 1 Thread 2
b = a; a = 1;

?

12

Atomic Operations

long a, b; /* 64 bit, initially 0 */

Thread 1 Thread 2
b = a; a = 1;

?
b == 0
b == 1
b == 4294967296
b == 4294967295

13

Cache Structure

CPUs may have local caches for performance

Main Memory

L2 Cache L2 Cache

L1 Cache L1 Cache L1CPU0
Cache L1CPU1
Cache

CPU0 CPU1 CPU2
CPU0 CPU1
CPU3

14

Cache Structure
Main Memory

L2 Cache L2 Cache

L1 Cache L1 Cache L1 Cache L1 Cache

CPU0 CPU1 CPU2 CPU3

Modifications do not become visible immediately
Modifications may be re-ordered
Reads may refer to outdated (cached) data
Reads may be re-ordered

15


polling update
long counter;
[..]
do
  {
    doSomething();
  }
while (counter < MAX);
counter is incremented by parallel thread
on a Multicore, changes to counter may not
become visible!

16


polling update
long counter;


[..]
do
  {
    doSomething();
  }
counter is incremented by parallel thread
on a Multicore, changes to counter may not
become visible!

17

Solution: volatile?

polling update
volatile long counter;
[..]
do
  {
    doSomething();
  }
works for Java
does not work for C!

18

Solution: volatile?

polling update
volatile long counter;
[..]
do
  {
    doSomething();
  }


works for Java
does not work for C!

19

Understanding the Memory Model

Memory model specifies what optimizations are
permitted by the compiler or underlying
hardware
C/C++ programs have undefined semantics in
case of race conditions
Java defines a strict memory model

20

Java's Memory Model

ordering operations are
synchronized block
accessing a volatile variable
The presence of an ordering operation
determines the visible state in shared memory

21

Java's Memory Model: Defined Order

all reads are completed before
entering synchronized block, or
reading a volatile variable

 read fence
all writes are completed before
exiting a synchronized block, or
writing a volatile variable

 write fence
22

Java's Memory Model: Data Races

data races are not forbidden in Java
you can use shared memory variables
your code has to tolerate optimizations
examples
collecting debugging / profiling information
useful if occasional errors due to data races are
tolerable

23

Memory Model Example

Shared memory communication
Ptr     p;
boolean p_valid;

Thread 1

p = new Ptr();
p_valid = true;


24


Ptr     p;
boolean p_valid;

Thread 1 Thread 2
p = new Ptr();
p_valid = true;



25


Ptr     p;
boolean p_valid;

Thread 1 Thread 2
p = new Ptr();          if (p_valid)
p_valid = true;           p.call();


26



Thread 1 Thread 2


27



Thread 1 Thread 2
What may happen:

28



Thread 1 Thread 2
What may happen:
t1 = new Ptr();
t2 = true;
p_valid = t2;
p = t1;

29



Thread 1 Thread 2
What may happen:
t1 = new Ptr();
t2 = true;
p_valid = t2;
p = t1;
Writes reordered!

30




Thread 1 Thread 2
What may happen:
t1 = new Ptr();
t2 = true;
p_valid = t2;
p = t1;
Writes reordered!

31




Thread 1 Thread 2
What may happen:
t1 = new Ptr();         t3 = p;
t2 = true;              if (p_valid)
p_valid = t2;             t3.call();
p = t1;
Writes reordered! Reads reordered!

32


 

Thread 1 Thread 2
What may happen:
t1 = new Ptr();         t3 = p;
t2 = true;              if (p_valid)
p_valid = t2;             t3.call();
p = t1;
Writes reordered! Reads reordered!

33

Example Use of Java's Memory Model


 volatile Ptr     p;
volatile boolean p_valid;

Thread 1 Thread 2

in Java

34


volatile Ptr     p;

Thread 1 Thread 2



in Java

35


volatile Ptr     p;

Thread 1 Thread 2

 

in Java

36

Example Use of C's Memory Model

volatile Obj    *p;

Thread 1 Thread 2
p = malloc(..);         if (p_valid)
p_valid = TRUE;           p>f = ..;();

in C?

37


volatile Obj    *p;

Thread 1 Thread 2

in C?
CPU may reorder memory accesses!

38


volatile Obj    *p;


Thread 1 Thread 2

in C?

39


volatile Obj    *p;

 
Thread 1 Thread 2

in C?

40


volatile Obj    *p;

Thread 1 Thread 2
p_valid = TRUE;           p>f = ..;

How to fix it?

41


volatile Obj    *p;

Thread 1 Thread 2
asm volatile(             p>f = ..;
  "sfence":::"memory");
p_valid = TRUE;

How to fix it? Add memory fences!

42


volatile Obj    *p;

Thread 1 Thread 2
asm volatile(             {
  "sfence":::"memory");     asm volatile(
p_valid = TRUE;             "lfence":::"memory");
                            p>f = ..;
                          }

43


volatile Obj    *p;

Thread 1 Thread 2
asm volatile(             {


  "sfence":::"memory");     asm volatile(


p_valid = TRUE;             "lfence":::"memory");
                            p>f = ..;
                          }

44

Out of Thin Air

imagine this code
int x = 0, n = 0;

Thread 1 Thread 2
for (i=0; i<n; i++) x = 42;
x += f(i); print(x);

45

Out of Thin Air

imagine this code
int x = 0, n = 0;

Thread 1 Thread 2
for (i=0; i<n; i++) x = 42;
x += f(i); print(x);
can only print 42 in Java

46

Out of Thin Air: Introduction of Writes

loop optimization in C/C++
int x = 0, n = 0;

Thread 1 Thread 2
tmp = x;
for (i=0; i<n; i++)     x = 42;
  tmp += f(i);
x = tmp;
                        print(x);

47

Out of Thin Air: Introduction of Writes

loop optimization in C/C++
int x = 0, n = 0;

Thread 1 Thread 2
tmp = x;
for (i=0; i<n; i++)     x = 42;
  tmp += f(i);
x = tmp;
                        print(x);
can print 0 in C/C++

48

Out of Thin Air

imagine this code
int x = 0, y = 0;

Thread 1 Thread 2
r1 = x; r2 = y;
y = r1; x = r2;

49

Out of Thin Air

imagine this code
int x = 0, y = 0;

Thread 1 Thread 2
r1 = x;                 r2 = y;
y = r1;                 x = r2;

Expected result
  x == 0; y == 0;

Only possible result in Java

50

Out of Thin Air: Optimization in C/C++

imagine this code
int x = 0, y = 0;

Thread 1 Thread 2
y = 42;                 r2 = y;
r1 = x;                 x = r2;
if (r1 != 42)
  y = r1;

Possible results in upcoming C++ MM
  x == 42; y == 42;

51

Performance on Multicore: Example

Single core application, 3 threads
All threads synchronize frequently on same lock

52


All threads synchronize frequently on same lock
while (true)
  {
    synchronized (lock)
      {
        counter++;
      }
    doSomething();
  }

53



54



On a multicore

55

Performance on a Multicore

Frequent synchronization can kill the performance
Typical non-RTOS will use heuristics to improve
average performance
spin-lock for a short time
blocking for longer periods

56

Performance on a Multicore

These heuristics introduce priority-inversion and
generally destroy predictability
A typical semaphore implementation
does not take thread priority into account
does not limit worst-case-execution-time

57

CPU Affinities

OSes provide APIs to lock threads certain CPUs
This limits the decision space of the RTOS
a global scheduler for n CPUs would always run the n
highest priority threads
with affinities, this may not be possible, e.g., if the two
highest priority threads are locked to the same CPU
CPU affinities can introduce priority inversion!
So what are CPU affinities good for?

58

CPU Affinities: No More interrupts

Locking interrupts to a dedicated CPU
protects all other CPUs from interrupts, and
invalidated cashes
WCETA is simplified considerably

59

CPU Affinities: No More interrupts

Locking interrupts to a dedicated CPU
protects all other CPUs from interrupts, and
invalidated cashes
WCETA is simplified considerably Interrupt

CPU0 non-RT Task

CPU1 A A A A A A A A A A A RT Task

60

CPU Affinities: Separating Threads

Locking thread A to one CPU and threads B, C, ...
to other CPUs may increase A's performance.
A will not be preempted by B, C, ..
A will not see its caches invalidated by B, C, ...

61

CPU Affinities: Separating Threads

Locking thread A to one CPU and threads B, C, ...
to other CPUs may increase A's performance.
A will not be preempted by B, C, ..
A will not see its caches invalidated by B, C, ...

CPU0 B B B B B non-RT Task

CPU1 A A A A A A A A A A A RT Task

CPU2 B C B C B C

62

CPU Affinities: Grouping Threads

Locking threads A and B to the same CPU
will increase shared memory communication between
A and B
will avoid performance degradation on locking
will enable simple scheduling analysis (RMA)

63

CPU Affinities: Grouping Threads

Locking threads A and B to the same CPU
will increase shared memory communication between
A and B
will avoid performance degradation on locking
will enable simple schedule feasibility analysis (RMA)

CPU0 C D C DC D E G D RT Task A

CPU1 A B A BA B A RT Task B

CPU2 E E G E E C E Other Task

64

CPU Affinities: Difficult Decisions

what if A and B both computation intensive and
both access the same shared memory?
How can we use idle time?
...

65

Multicore Scheduling for Realtime

A pure priority based scheduler is not sufficient:
Imagine three tasks A, B, C on 2 CPUS

A

B

C

Release Deadline
66


Priorities will cause deadline miss

CPU0 A pri=10 C pri=9

CPU1 B pri=10

Release Deadline
67


Priorities will cause deadline miss

CPU0

CPU1
A pri=10

B pri=10 
C pri=9

Release Deadline
68


CPU affinities do not help

CPU0 A pri=10, {CPU0} C pri=9

CPU1 B pri=10, {CPU1}

Release Deadline
69


Round robin could help

CPU0 A C B A C B

CPU1 B A C B A C

Release Deadline
70


Round robin could help

CPU0 A C B A C B


CPU1 B A C B A C

Release Deadline
71

Performance on Multicore

Synchronization is expensive
Can synchronization be avoided?
Can lock free algorithms be used?
Use compare and swap (CAS) instructions instead

72

Lock Free Algorithms

Typical code sequence
do
  {
    x = counter;
    result = CAS(counter,x,x+1);
  }
while (result != x);

73

Compare-and-Swap Issues

do
  {
    x = counter;
  }
What is the WCET? ∞?

74

Compare-and-Swap Issues
On dual core:
do
  {
    x = counter;
  }

frequency
What is the WCET? ∞?

# iterations

75

Lock Free Library Code

Using libraries helps
AtomicInteger counter = new AtomicInteger();
void increment()
{
(void)counter.incrementAndGet();
}
Code is easier and safer
Hand made lock free algorithms are not for
normal application development

76

Compare-and-Swap Solutions

One way state changes, without retry
Bounded number of retries

77

One Way State Changes Using CAS

Example code
for (i=0; i<N; i++)
  {
    new Thread()
      {
        public void run()
          {
            CAS(state,INIT,STARTING);
            [..]
          }
      }.start();
  }

78

Bounding Retries for CAS

introduce long enough code sections in between
2 compare-and-swap loops
then, if a retry is required, one other CPU was
successful
after n-1 conflicts, one can be sure that all other
CPUs are outside the CAS loop

79

Bounding Retries for CAS: Example
     AtomicInteger counter = new AtomicInteger();
     static final int GRANULARITY = 64;
     [..]
     new Thread(){
       int local_counter;
       public void incCounter() {
         local_counter++;
         if (local_counter >= GRANULARITY) {
           local_counter = 0;
           counter.addAndSet(GRANULARITY);
         }
       }
     }.start();

80

Measurements
Number of CAS tries is bounded:
1E+11
1E+10
1E+9
1E+8
1
1E+7 2
3
1E+6
4
1E+5 5
6
1E+4
7
1E+3 8
1E+2
9

1E+1
1E+0
1E-1
alloc free sweep available Memory

On 8-CPU x86 system (Linux)

81

CAS Lists

List modifaction using CAS, single linked list
A B C
head next next next

data data data

82

CAS Lists

Add
A B C
head next next next

data data data

X
next

data

83

CAS Lists

Add
A B C
head next next next

data data data

X
next

data

84

CAS Lists

Add: CAS(head,A,X)
A B C
head next next next

data data data

X
next

data

85

CAS Lists

Add: CAS(head,A,X)
A B C
head  next next next

data data data

X
next

data

86

CAS Lists

Add
A B C
head next next next

data data data

X
next

data

87

CAS Lists

Remove
A B C
head next next next

data data data

88

CAS Lists

Remove: CAS(head,A,B)
A B C
head next next next

data data data

89

CAS Lists

Remove: CAS(head,A,B)
A B C
head  next next next

data data data

90

CAS Lists

Remove
B C
head next next

data data

91

CAS Lists, ABA Problem

Now, consider concurrent modifications:
Thread 1 head
A
next
B
next
C
next
Thread 2
data data data

92

Thread 1 head
A
next
B
next
C
next
Thread 2
remove: data data data
CAS(head,A,B)

93

Thread 1 head
A
next
B
next
C
next
Thread 2
pre
e
mp
ted

CAS(head,A,B)

94

Thread 1 head
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A,B)
pre
e
mp
ted

CAS(head,A,B)

95

Thread 1 head
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A,B)
B C
pre

head next next

data data
e
mp
ted

CAS(head,A,B)

96

Thread 1 head
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A,B)
B C
pre

head next next

data data
remove:
e

CAS(head,B,C)
mp
ted

CAS(head,A,B)

97

Thread 1 head
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A,B)
B C
pre

head next next

data data
remove:
e

CAS(head,B,C)
mp

C
head next
t

data
ed

CAS(head,A,B)

98

Thread 1 head
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A,B)
B C
pre

head next next

data data
remove:
e

CAS(head,B,C)
mp

C
head next
t

data
ed

add A:
CAS(head,C,A)

CAS(head,A,B)

99

Thread 1 head
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A,B)
B C
pre

head next next

data data
remove:
e

CAS(head,A,B)
mp

C
head next
t

data
ed

A C add A:
head next next CAS(head,C,A)
data data

CAS(head,A,B)

100

Thread 1 head
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A,B)
B C
pre

head next next

data data
remove:
e

CAS(head,A,B)
mp

C
head next
t

data
ed

A C add A:
data data

CAS(head,A,B) B A C
head next next next
data data data

101

Thread 1 head
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A,B)
B C
pre

head next next

data data
remove:
e

CAS(head,A,B)
mp

C
head next


t

data
ed

A C add A:
data data

CAS(head,A,B) B A C
head next next next
data data data

B was re-introduced!
102

ABA Problem: Solutions

Non solution: In C: free() after remove
Reason: newly allocated block may be same
Solution: In Java: only add new references
Reason: GC ensures old values no longer visible
Solution: Sync all threads before reuse
Example: Phase 1 moves elements from List1 to
List2, Phase 2 moves elements back
Solution: Use 64-/128-bit CAS and mod. counter

103

ABA Solved via Modification Counter
Thread 1 head: 42
A
next
B
next
C
next
Thread 2
pre
e
mp
ted

104

Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A:42,
B:43)
pre
e
mp
ted

105

Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A:42,
B C
B:43)
head: 43 next next

data data

106

Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A:42,
B C
B:43)
head: 43 next next

data data
remove:
C CAS(head,B:43,
head: 44 next C:44)
data

107

Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A:42,
B C
B:43)
head: 43 next next

data data
remove:
C CAS(head,B:43,
head: 44 next C:44)
data

A C add A:
head: 45 next next CAS(head,C:44,
data data A:45)

108

Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A:42,
B C
B:43)
head: 43 next next

data data
remove:
C CAS(head,B:43,
head: 44 next C:44)
data

A C add A:
CAS(head,A:42, data data A:45)
B:43)
retry!

109

Thread 1 head: 42
A
next
B
next
C
next
Thread 2
remove:
CAS(head,A:42,
B C
B:43)
head: 43 next next

data data
remove:
C CAS(head,B:43,
head: 44 next C:44)
data

A C add A:
CAS(head,A:42, data data A:45)
B:43) C
retry! head: 46 next

CAS(head,A:45, data

C:46)

110

Conclusion
Code that runs well on single CPU may fail on
multicore
Clear semantics of concurrent code is required
for functional correctness
Cost of locking may be prohibitive
CPU affinities may help, but it is difficult to make
the application more efficient
Lock free code is very hard to get right
A reliable memory model and good concurrent
libraries are basis to avoid pitfalls.

111

Thank you.
Please come see us at booth 2306

112

Multicoreexpo2011 05-05fridiusingtemplate-110518193827-phpapp02

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (17)

Similar to Multicoreexpo2011 05-05fridiusingtemplate-110518193827-phpapp02

Similar to Multicoreexpo2011 05-05fridiusingtemplate-110518193827-phpapp02 (20)

Recently uploaded

Recently uploaded (20)

Multicoreexpo2011 05-05fridiusingtemplate-110518193827-phpapp02