1. Introduction
Lock-Free Explained
Lock-Free Performance
Summary
Lock-Free Data Exchange for Real-Time
Applications
Peter Soetens
Flander’s Mechatronics Technology Centre
Leuven
25 Feb 2006
Free and Open Source Developers Meeting
Peter Soetens Lock-Free Data Exchange

2. Introduction
Lock-Free Explained
Lock-Free Performance
Summary
Outline
1 Introduction
About You and Me
Application Domain
2 Lock-Free Explained
The Good ’Ol Days
A New Kind of Computer Science
3 Lock-Free Performance
Time determinism
Algorithm Overhead
Peter Soetens Lock-Free Data Exchange

3. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
Outline
1 Introduction
About You and Me
Application Domain
2 Lock-Free Explained
The Good ’Ol Days
A New Kind of Computer Science
3 Lock-Free Performance
Time determinism
Algorithm Overhead
Peter Soetens Lock-Free Data Exchange

4. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
What You Will Learn.
What lock-free data exchange offers.
When to use it.
Why it is better.
Peter Soetens Lock-Free Data Exchange

5. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
What I Will Offer.
Give an introduction to lock-free algorithms for non-experts.
Show an experimental comparison between traditional
locks and lock-free algorithms.
Provide good leads for lock-free application and kernel
development.
Peter Soetens Lock-Free Data Exchange

6. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
What I Assume About You.
You. . .
Peter Soetens Lock-Free Data Exchange

7. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
What I Assume About You.
Peter Soetens Lock-Free Data Exchange

8. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
What I Assume About You.
You. . .
know what multi-threaded applications are.
care about real-time or embedded application design.
Peter Soetens Lock-Free Data Exchange

9. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
Outline
1 Introduction
About You and Me
Application Domain
2 Lock-Free Explained
The Good ’Ol Days
A New Kind of Computer Science
3 Lock-Free Performance
Time determinism
Algorithm Overhead
Peter Soetens Lock-Free Data Exchange

10. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
Definitions
An algorithm is lock-free if. . .
it does not ’block’ threads
every step taken achieves global progress.
it allows individual threads to starve (loop forever) but
denies livelock.
Redefinition (2003): “Obstruction Free”
Real-Time
A term to denote execution time determinism of an action or
sequence of actions in response to an event. This means that
the action always completes (and/or starts) within a bounded
time interval.
Peter Soetens Lock-Free Data Exchange

11. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
When to Use Lock-Free Algorithms
Lock-Free is especially useful for
multi-threaded or -process applications
blob data- or pointer-exchange
OS kernels and applications
Real-Time (time determinism)
Embedded (simple scheduler)
Peter Soetens Lock-Free Data Exchange

12. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
When to Use Lock-Free Algorithms
Lock-Free is especially useful for
multi-threaded or -process applications
blob data- or pointer-exchange
OS kernels and applications
Real-Time (time determinism)
Embedded (simple scheduler)
Peter Soetens Lock-Free Data Exchange

13. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
What Lock-Free Requires
Lock-Free requires
Processor support
A Compare-And-Swap (CAS, CMPXCH) or
Load-Linked/Store-Conditional (LL/SC) processor
instruction or
Atomic increment/decrement and test.
Architecture support
A piece of physically Shared Memory.
An algorithm
Fortunately, they all look the same.
Peter Soetens Lock-Free Data Exchange

14. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
What Lock-Free Requires
Lock-Free requires
Processor support
A Compare-And-Swap (CAS, CMPXCH) or
Load-Linked/Store-Conditional (LL/SC) processor
instruction or
Atomic increment/decrement and test.
Architecture support
A piece of physically Shared Memory.
An algorithm
Fortunately, they all look the same.
Peter Soetens Lock-Free Data Exchange

15. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
What Lock-Free Requires
Lock-Free requires
Processor support
A Compare-And-Swap (CAS, CMPXCH) or
Load-Linked/Store-Conditional (LL/SC) processor
instruction or
Atomic increment/decrement and test.
Architecture support
A piece of physically Shared Memory.
An algorithm
Fortunately, they all look the same.
Peter Soetens Lock-Free Data Exchange

16. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
What Lock-Free Does Not Require
Lock-Free does not require
OS support
Scheduler support
Peter Soetens Lock-Free Data Exchange

17. Introduction
Lock-Free Explained About You and Me
Lock-Free Performance Application Domain
Summary
Example Uses
Common uses are:
From Orocos.org:
FIFO/LIFO data buffers
(Single) linked lists
Pointer queues
Shared data
From Boost.org (0.33.0)
Smart pointers
. . . in many readers/many writers environments
Peter Soetens Lock-Free Data Exchange

18. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Outline
1 Introduction
About You and Me
Application Domain
2 Lock-Free Explained
The Good ’Ol Days
A New Kind of Computer Science
3 Lock-Free Performance
Time determinism
Algorithm Overhead
Peter Soetens Lock-Free Data Exchange

19. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
The Example Application
D
Shared Data
Writer Reader
Thread Thread
’Thread’ : May be an interrupt, thread, process,...
’Data’ : Modifyable in a single (composite) transaction.
Peter Soetens Lock-Free Data Exchange

20. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Unprotected Access
D
Peter Soetens Lock-Free Data Exchange

21. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Unprotected Access
Unrestricted
Access
D
Corrupted Data
Peter Soetens Lock-Free Data Exchange

22. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
The Lock
D
Peter Soetens Lock-Free Data Exchange

23. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
D
Peter Soetens Lock-Free Data Exchange

24. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
D
Peter Soetens Lock-Free Data Exchange

25. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
D
Peter Soetens Lock-Free Data Exchange

26. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
D
Peter Soetens Lock-Free Data Exchange

27. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
ZZzzz...
D
A worst case scenario. . .
Peter Soetens Lock-Free Data Exchange

28. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
ZZzzz...
D
A worst case scenario. . .
Peter Soetens Lock-Free Data Exchange

29. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
ZZzzz...
D
A worst case scenario. . .
Peter Soetens Lock-Free Data Exchange

30. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
ZZzzz...
D
A worst case scenario. . .
Peter Soetens Lock-Free Data Exchange

31. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
D
Is this scalable ?
Peter Soetens Lock-Free Data Exchange

32. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
D
Which thread has the lowest priority ?
Peter Soetens Lock-Free Data Exchange

33. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
Lowest
D Priority !
Which thread has the lowest priority ?
Peter Soetens Lock-Free Data Exchange

34. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
Pre−empting
threads
always have
higher Lowest
priority ! D Priority !
Which thread has the lowest priority ?
Peter Soetens Lock-Free Data Exchange

35. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
Lowest
Priority ! D
Which thread has the lowest priority ?
Peter Soetens Lock-Free Data Exchange

36. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Using Locks
D
Lowest
Priority !
Which thread has the lowest priority ?
Peter Soetens Lock-Free Data Exchange

37. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Based Reader-Writer Summary
Effect on priorities ?
A higher priority means a worse delay
Best case access time ?
Grab the lock, modify, release lock (Taccess )
Worst case access time ?
= 2 ∗ Taccess using priority inheritance or
∞ without priority inheritance (priority inversions)
Memory requirements ?
= sizeof (D) + sizeof (Lock ) + sizeof (SchedAlgorithms)
Peter Soetens Lock-Free Data Exchange

38. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Based Reader-Writer Summary
- Effect on priorities ?
A higher priority means a worse delay
Best case access time ?
Grab the lock, modify, release lock (Taccess )
Worst case access time ?
= 2 ∗ Taccess using priority inheritance or
∞ without priority inheritance (priority inversions)
Memory requirements ?
= sizeof (D) + sizeof (Lock ) + sizeof (SchedAlgorithms)
Peter Soetens Lock-Free Data Exchange

39. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Based Reader-Writer Summary
- Effect on priorities ?
A higher priority means a worse delay
- Best case access time ?
Grab the lock, modify, release lock (Taccess )
Worst case access time ?
= 2 ∗ Taccess using priority inheritance or
∞ without priority inheritance (priority inversions)
Memory requirements ?
= sizeof (D) + sizeof (Lock ) + sizeof (SchedAlgorithms)
Peter Soetens Lock-Free Data Exchange

40. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Based Reader-Writer Summary
- Effect on priorities ?
A higher priority means a worse delay
- Best case access time ?
Grab the lock, modify, release lock (Taccess )
- Worst case access time ?
= 2 ∗ Taccess using priority inheritance or
∞ without priority inheritance (priority inversions)
Memory requirements ?
= sizeof (D) + sizeof (Lock ) + sizeof (SchedAlgorithms)
Peter Soetens Lock-Free Data Exchange

41. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Based Reader-Writer Summary
- Effect on priorities ?
A higher priority means a worse delay
- Best case access time ?
Grab the lock, modify, release lock (Taccess )
- Worst case access time ?
= 2 ∗ Taccess using priority inheritance or
∞ without priority inheritance (priority inversions)
+ Memory requirements ?
= sizeof (D) + sizeof (Lock ) + sizeof (SchedAlgorithms)
Peter Soetens Lock-Free Data Exchange

42. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Based Reader-Writer Summary
- Effect on priorities ?
A higher priority means a worse delay
- Best case access time ?
Grab the lock, modify, release lock (Taccess )
- Worst case access time ?
= 2 ∗ Taccess using priority inheritance or
∞ without priority inheritance (priority inversions)
+ Memory requirements ?
= sizeof (D) + sizeof (Lock ) + sizeof (SchedAlgorithms)
Peter Soetens Lock-Free Data Exchange

43. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Let’s get rid of locks!
Peter Soetens Lock-Free Data Exchange

44. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Outline
1 Introduction
About You and Me
Application Domain
2 Lock-Free Explained
The Good ’Ol Days
A New Kind of Computer Science
3 Lock-Free Performance
Time determinism
Algorithm Overhead
Peter Soetens Lock-Free Data Exchange

45. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 1 Writer
D 2 Reader
Threads
Shared
1 Writer Data
Thread
Peter Soetens Lock-Free Data Exchange

46. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 1 Writer
Mark most recent
D D D
Read most recent
Pre-allocate data blocks to avoid run-time allocation (not
real-time). No heap required.
Peter Soetens Lock-Free Data Exchange

47. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 1 Writer
Write...
modify
D D D
Yep, that’s RCU (Read-Copy-Update) for you experts.
Peter Soetens Lock-Free Data Exchange

48. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 1 Writer
Done!
D D D
Peter Soetens Lock-Free Data Exchange

49. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 1 Writer
D D D
Peter Soetens Lock-Free Data Exchange

50. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 1 Writer
D D D
Peter Soetens Lock-Free Data Exchange

51. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 1 Writer
D D D
Peter Soetens Lock-Free Data Exchange

52. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 1 Writer
D D D
Peter Soetens Lock-Free Data Exchange

53. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 1 Writer
Mark Writer
modify
D D D D
Reader Reader
Worst case number of required data blocks = 2 + R
For a single writer - many readers setup.
Peter Soetens Lock-Free Data Exchange

54. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Single Writer Summary
Effect on priorities ?
None
Best case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
Worst case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
Memory requirements ?
= sizeof (D) ∗ (2 + R)
Did I use CAS ?
No
Peter Soetens Lock-Free Data Exchange

55. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Single Writer Summary
+ Effect on priorities ?
None
Best case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
Worst case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
Memory requirements ?
= sizeof (D) ∗ (2 + R)
Did I use CAS ?
No
Peter Soetens Lock-Free Data Exchange

56. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Single Writer Summary
+ Effect on priorities ?
None
+ Best case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
Worst case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
Memory requirements ?
= sizeof (D) ∗ (2 + R)
Did I use CAS ?
No
Peter Soetens Lock-Free Data Exchange

57. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Single Writer Summary
+ Effect on priorities ?
None
+ Best case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
+ Worst case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
Memory requirements ?
= sizeof (D) ∗ (2 + R)
Did I use CAS ?
No
Peter Soetens Lock-Free Data Exchange

58. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Single Writer Summary
+ Effect on priorities ?
None
+ Best case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
+ Worst case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
- Memory requirements ?
= sizeof (D) ∗ (2 + R)
Did I use CAS ?
No
Peter Soetens Lock-Free Data Exchange

59. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Single Writer Summary
+ Effect on priorities ?
None
+ Best case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
+ Worst case access time ?
Writer: TFindpointer + TModify + TPointercopy
Reader: TRead
- Memory requirements ?
= sizeof (D) ∗ (2 + R)
Did I use CAS ?
No
Peter Soetens Lock-Free Data Exchange

60. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 2 Writers
2 Writer D 2 Reader
Threads Threads
Shared
Data
Peter Soetens Lock-Free Data Exchange

61. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 2 Writers
Which one is most recent ?
D D D
Peter Soetens Lock-Free Data Exchange

62. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 2 Writers
Writer 1
modify
D D D
Peter Soetens Lock-Free Data Exchange

63. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 2 Writers
Writer 1 Writer 2
Higher
modify priority
D D D
Peter Soetens Lock-Free Data Exchange

64. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 2 Writers
Compare Writer 1 Writer 2
−and−
Swap: modify
D D D
Peter Soetens Lock-Free Data Exchange

65. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 2 Writers
Writer 1
Done!
modify
D D D
Peter Soetens Lock-Free Data Exchange

66. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 2 Writers
Writer 1
CAS
Fails ! modify
D D D
Peter Soetens Lock-Free Data Exchange

67. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 2 Writers
Writer 1 modify
again!
D D D
Peter Soetens Lock-Free Data Exchange

68. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 2 Writers
Done!
D D D
Peter Soetens Lock-Free Data Exchange

69. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Away with Locks: 2 Writers
D D D
Peter Soetens Lock-Free Data Exchange

70. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Multiple Writer Summary
Effect on priorities ?
Highest priority wins!
Best case access time ?
= TFindpointer + TModify + TPointercopy
Always the case for the highest priority thread.
Worst case access time ?
= WHigherPriority ∗ TBestcase
Depends on the number of higher priority writers.
Memory requirements ?
= sizeof (D) ∗ (2 ∗ W + R)
Peter Soetens Lock-Free Data Exchange

71. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Multiple Writer Summary
+ Effect on priorities ?
Highest priority wins!
Best case access time ?
= TFindpointer + TModify + TPointercopy
Always the case for the highest priority thread.
Worst case access time ?
= WHigherPriority ∗ TBestcase
Depends on the number of higher priority writers.
Memory requirements ?
= sizeof (D) ∗ (2 ∗ W + R)
Peter Soetens Lock-Free Data Exchange

72. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Multiple Writer Summary
+ Effect on priorities ?
Highest priority wins!
+ Best case access time ?
= TFindpointer + TModify + TPointercopy
Always the case for the highest priority thread.
Worst case access time ?
= WHigherPriority ∗ TBestcase
Depends on the number of higher priority writers.
Memory requirements ?
= sizeof (D) ∗ (2 ∗ W + R)
Peter Soetens Lock-Free Data Exchange

73. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Multiple Writer Summary
+ Effect on priorities ?
Highest priority wins!
+ Best case access time ?
= TFindpointer + TModify + TPointercopy
Always the case for the highest priority thread.
+ Worst case access time ?
= WHigherPriority ∗ TBestcase
Depends on the number of higher priority writers.
Memory requirements ?
= sizeof (D) ∗ (2 ∗ W + R)
Peter Soetens Lock-Free Data Exchange

74. Introduction
Lock-Free Explained The Good ’Ol Days
Lock-Free Performance A New Kind of Computer Science
Summary
Lock-Free Multiple Writer Summary
+ Effect on priorities ?
Highest priority wins!
+ Best case access time ?
= TFindpointer + TModify + TPointercopy
Always the case for the highest priority thread.
+ Worst case access time ?
= WHigherPriority ∗ TBestcase
Depends on the number of higher priority writers.
- Memory requirements ?
= sizeof (D) ∗ (2 ∗ W + R)
Peter Soetens Lock-Free Data Exchange

75. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Outline
1 Introduction
About You and Me
Application Domain
2 Lock-Free Explained
The Good ’Ol Days
A New Kind of Computer Science
3 Lock-Free Performance
Time determinism
Algorithm Overhead
Peter Soetens Lock-Free Data Exchange

76. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Recap: Definition
Real-Time
A term to denote execution time determinism of an action or
sequence of actions in response to an event. This means that
the action always completes (and/or starts) within a bounded
time interval.
Peter Soetens Lock-Free Data Exchange

77. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Going Real-Time
Given one of these Real-Time schedulers:
Rate Monotonic Scheduler (RMS)
Deadline Monotonic Scheduler (DMS)
Earliest Deadline First Scheduler (EDFS)
The following properties are always true for any lock-free
algorithm:
The highest priority writer thread has best case access
time.
The other writer threads have bounded access time.
Any reader has always best case access time.
Peter Soetens Lock-Free Data Exchange

78. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Going Real-Time
Given one of these Real-Time schedulers:
Rate Monotonic Scheduler (RMS)
Deadline Monotonic Scheduler (DMS)
Earliest Deadline First Scheduler (EDFS)
The following properties are always true for any lock-free
algorithm:
The highest priority writer thread has best case access
time.
The other writer threads have bounded access time.
Any reader has always best case access time.
Peter Soetens Lock-Free Data Exchange

79. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Going Real-Time
Given one of these Real-Time schedulers:
Rate Monotonic Scheduler (RMS)
Deadline Monotonic Scheduler (DMS)
Earliest Deadline First Scheduler (EDFS)
The following properties are always true for any lock-free
algorithm:
The highest priority writer thread has best case access
time.
The other writer threads have bounded access time.
Any reader has always best case access time.
Peter Soetens Lock-Free Data Exchange

80. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Going Real-Time
Given one of these Real-Time schedulers:
Rate Monotonic Scheduler (RMS)
Deadline Monotonic Scheduler (DMS)
Earliest Deadline First Scheduler (EDFS)
The following properties are always true for any lock-free
algorithm:
The highest priority writer thread has best case access
time.
The other writer threads have bounded access time.
Any reader has always best case access time.
Peter Soetens Lock-Free Data Exchange

81. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Real-Time Validation Experiment
Real-time Machine Controller
Pentium III 750MHz, 128MB RAM
(this is vastly oversized for our purpose, but allowed
on-target data capturing)
Software
Linux 2.4.18 with RTAI/LXRT 3.0 Patch
Orocos configured for LXRT
Many readers / many writers test applications
Both FIFO buffers and shared data exchange
Peter Soetens Lock-Free Data Exchange

82. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Real-Time Validation: Data Flow
Sma ll
NRT RT RT
500Hz 1KHz
RT RT
500Hz 500Hz
Concu rr ent
NRT RT NRT RT
2KHz 2KHz
RT RT
1Hz 1Hz
Peter Soetens Lock-Free Data Exchange

83. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Real-Time Validation: No Communication
1e+06
1ms/0.5ms
100000 5ms/1ms
10000
Occurences
1000
100
Execution
10
1 latencies
0.1
1e-05 1e-04 0.001
Latency time ( s ). Bucket size: 5 us
0.01
For a small (2 RT
1e+06
0.5ms/0.1ms threads) and
100000 1ms/0.2ms
10000
2ms/0.3ms concurrent (3 RT
Occurences
1000
100
threads)
10 application.
1
0.1
1e-05 1e-04 0.001 0.01
Latency time ( s ). Bucket size: 5 us
Peter Soetens Lock-Free Data Exchange

84. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Real-Time Validation: Data Exchange
1e+06
1ms/0.5ms
100000
10000
Occurences
1000
100
10
1
0.1
1e-06 1e-05 1e-04 0.001 0.01 0.1
Communication
1e+06
Latency time ( s ). Bucket size: 5 us latencies
1ms/0.5ms
100000
Lock based (top)
10000
Occurences
1000 and lock free
100
10
(bottom).
1
0.1
1e-06 1e-05 1e-04 0.001 0.01 0.1
Latency time ( s ). Bucket size: 5 us
Small application, high priority thread.
Peter Soetens Lock-Free Data Exchange

85. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Real-Time Validation: Data Exchange
100000
5ms/0.5ms
10000
Occurences
1000
100
10
1
0.1
1e-06 1e-05 1e-04 0.001 0.01 0.1
Communication
100000
Latency time ( s ). Bucket size: 5 us latencies
5ms/0.5ms
10000 Lock based (top)
Occurences
1000
100
and lock-free
10 (bottom).
1
0.1
1e-06 1e-05 1e-04 0.001 0.01 0.1
Latency time ( s ). Bucket size: 5 us
Small application, low priority thread.
Peter Soetens Lock-Free Data Exchange

86. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Real-Time Validation: Data Exchange
100000
0.5ms/0.1ms
10000
Occurences
1000
100
10
1
0.1
1e-06 1e-05 1e-04 0.001 0.01 0.1
Communication
100000
Latency time ( s ). Bucket size: 5 us latencies
0.5ms/0.1ms
10000 Lock based (top)
Occurences
1000
100
and lock free
10 (bottom).
1
0.1
1e-06 1e-05 1e-04 0.001 0.01 0.1
Latency time ( s ). Bucket size: 5 us
Concurrent application, high priority thread.
Peter Soetens Lock-Free Data Exchange

87. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Real-Time Validation: Data Exchange
100000
1ms/0.2ms
10000
Occurences
1000
100
10
1
0.1
1e-06 1e-05 1e-04 0.001 0.01 0.1
Communication
1e+06
Latency time ( s ). Bucket size: 5 us latencies
1ms/0.2ms
100000
Lock based (top)
10000
Occurences
1000 and lock free
100
10
(bottom).
1
0.1
1e-06 1e-05 1e-04 0.001 0.01 0.1
Latency time ( s ). Bucket size: 5 us
Concurrent application, medium priority thread.
Peter Soetens Lock-Free Data Exchange

88. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Real-Time Validation: Data Exchange
100000
2ms/0.3ms
10000
Occurences
1000
100
10
1
0.1
1e-06 1e-05 1e-04 0.001 0.01 0.1
Communication
100000
Latency time ( s ). Bucket size: 5 us latencies
2ms/0.3ms
10000 Lock based (top)
Occurences
1000
100
and lock free
10 (bottom).
1
0.1
1e-06 1e-05 1e-04 0.001 0.01 0.1
Latency time ( s ). Bucket size: 5 us
Concurrent application, medium priority thread.
Peter Soetens Lock-Free Data Exchange

89. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Real-Time Validation: Conclusions
Small Applications
Lock-free performs on average better
Lock-free performs worst case better
Concurrent Applications
Lock-free performs on average better
Lock-free performs worst case better
Lock-free prevents dead-line failures
Peter Soetens Lock-Free Data Exchange

90. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Outline
1 Introduction
About You and Me
Application Domain
2 Lock-Free Explained
The Good ’Ol Days
A New Kind of Computer Science
3 Lock-Free Performance
Time determinism
Algorithm Overhead
Peter Soetens Lock-Free Data Exchange

91. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Memory Overhead
Memory consumption increases linearly
OK for most real-time and
embedded applications, number of
threads is well known.
Worse, if not catastrophic, for OS Reference counted
kernels, number of threads is
unknown. D D D
Reference counted memory
Both readers and writers need to
reference count data blocks.
Requires ’atomic’ processor
instructions.
Peter Soetens Lock-Free Data Exchange

92. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Overhead for Readers
Increase reference count
Since a data block may not be freed
before all readers are done reading
it, a reference counting
Reference counted
implementation is required.
Analogous to RCU D D D
Detect moved ’Most Recent’ pointer.
If a reader ’locks’ the data block but
detects that the refcount is one, it
must retry, since the block may be in
re-use already.
Peter Soetens Lock-Free Data Exchange

93. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Overhead for Writers
Find an empty data block
Possibly race against other writers
Increase reference count
Copy and update the data Reference counted
Large data blocks will reduce D D D
performance
Retry if necessary (W > 1)
In case source data block changed,
startover with the copy-update from
the new data block.
Peter Soetens Lock-Free Data Exchange

94. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Exception: Lock-free Pointer Queues
Best case access time ?
= TFindpointer + TPointercopy
Always the case for the highest priority thread.
Worst case access time ?
= WHigherPriority ∗ TBestcase
Depends on the number of higher priority writers.
Memory requirements ?
= sizeof (D) ∗ Nqueue
⇒ Best of both worlds ! Independent of number of threads.
Peter Soetens Lock-Free Data Exchange

95. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Exception: Lock-free Pointer Queues
+ Best case access time ?
= TFindpointer + TPointercopy
Always the case for the highest priority thread.
Worst case access time ?
= WHigherPriority ∗ TBestcase
Depends on the number of higher priority writers.
Memory requirements ?
= sizeof (D) ∗ Nqueue
⇒ Best of both worlds ! Independent of number of threads.
Peter Soetens Lock-Free Data Exchange

96. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Exception: Lock-free Pointer Queues
+ Best case access time ?
= TFindpointer + TPointercopy
Always the case for the highest priority thread.
+ Worst case access time ?
= WHigherPriority ∗ TBestcase
Depends on the number of higher priority writers.
Memory requirements ?
= sizeof (D) ∗ Nqueue
⇒ Best of both worlds ! Independent of number of threads.
Peter Soetens Lock-Free Data Exchange

97. Introduction
Lock-Free Explained Time determinism
Lock-Free Performance Algorithm Overhead
Summary
Exception: Lock-free Pointer Queues
+ Best case access time ?
= TFindpointer + TPointercopy
Always the case for the highest priority thread.
+ Worst case access time ?
= WHigherPriority ∗ TBestcase
Depends on the number of higher priority writers.
+ Memory requirements ?
= sizeof (D) ∗ Nqueue
⇒ Best of both worlds ! Independent of number of threads.
Peter Soetens Lock-Free Data Exchange

98. Introduction
Lock-Free Explained
Lock-Free Performance
Summary
Summary
Lock-Free algorithms are a drastic improvement for
real-time applications
Lock-Free algorithms don’t require any scheduler
intervention.
But be aware of memory requirements.
Peter Soetens Lock-Free Data Exchange

99. Introduction
Lock-Free Explained
Lock-Free Performance
Summary
Thank you for your attention !
Peter Soetens Lock-Free Data Exchange

100. Introduction
Lock-Free Explained
Lock-Free Performance
Summary
References
http://www.orocos.org
Anderson, J., S. Ramamurthy, and K. Jeffay (1995).
Real-time com- puting with lock-free shared objects.
Proceedings of the 16th IEEE Real-Time Systems
Symposium.
Herlihy, M. (1991). Wait-free synchronization. ACM Trans.
Program. Lang. Syst. 13 (1), 124-149.
Herlihy, M., V. Luchangco, and M. Moir (2003).
Obstruction-free synchronization: Double-ended queues
as an example. In 03: Proceedings of the 23rd
International Conference on Dis- tributed Computing
Systems, Washington, DC, USA, pp. 522. IEEE Computer
Society.
Peter Soetens Lock-Free Data Exchange