1. Linked Lists: Locking, LockFree, and Beyond …
Dr. C.V. Suresh Babu

2. Today: Concurrent Objects
• Adding threads should not lower
throughput
– Contention effects
– Mostly fixed by Queue locks
• Should increase throughput
– Not possible if inherently sequential
– Surprising things are parallelizable
Linked Lists
2

3. Coarse-Grained
Synchronization
• Each method locks the object
– Avoid contention using queue locks
Linked Lists
3

4. Coarse-Grained
Synchronization
• Each method locks the object
– Avoid contention using queue locks
– Easy to reason about
• In simple cases
Linked Lists
4

5. Coarse-Grained
Synchronization
• Each method locks the object
– Avoid contention using queue locks
– Easy to reason about
• In simple cases
• So, are we done?
Linked Lists
5

6. Coarse-Grained
Synchronization
• Sequential bottleneck
– Threads “stand in line”
Linked Lists
6

7. Coarse-Grained
Synchronization
• Sequential bottleneck
– Threads “stand in line”
• Adding more threads
– Does not improve throughput
– Struggle to keep it from getting worse
Linked Lists
7

8. Coarse-Grained
Synchronization
• Sequential bottleneck
– Threads “stand in line”
• Adding more threads
– Does not improve throughput
– Struggle to keep it from getting worse
• So why even use a multiprocessor?
– Well, some apps inherently parallel …
Linked Lists
8

9. This Lecture
• Introduce four “patterns”
– Bag of tricks …
– Methods that work more than once …
Linked Lists
9

10. This Lecture
• Introduce four “patterns”
– Bag of tricks …
– Methods that work more than once …
• For highly-concurrent objects
– Concurrent access
– More threads, more throughput
Linked Lists
10

11. First:
Fine-Grained Synchronization
• Instead of using a single lock …
• Split object into
– Independently-synchronized components
• Methods conflict when they access
– The same component …
– At the same time
Linked Lists
11

12. Second:
Optimistic Synchronization
• Search without locking …
• If you find it, lock and check …
– OK: we are done
– Oops: start over
• Evaluation
– Usually cheaper than locking, but
– Mistakes are expensive
Linked Lists
12

13. Third:
Lazy Synchronization
• Postpone hard work
• Removing components is tricky
– Logical removal
• Mark component to be deleted
– Physical removal
• Do what needs to be done
Linked Lists
13

14. Fourth:
Lock-Free Synchronization
• Don’t use locks at all
– Use compareAndSet() & relatives …
• Advantages
– No Scheduler Assumptions/Support
• Disadvantages
– Complex
– Sometimes high overhead
Linked Lists
14

15. Linked List
• Illustrate these patterns …
• Using a list-based Set
– Common application
– Building block for other apps
Linked Lists
15

16. Set Interface
• Unordered collection of items
• No duplicates
• Methods
– add(x) put x in set
– remove(x) take x out of set
– contains(x) tests if x in set
Linked Lists
16

17. List-Based Sets
public interface Set<T> {
public boolean add(T x);
public boolean remove(T x);
public boolean contains(T x);
}
Linked Lists
17

18. List-Based Sets
public interface Set<T> {
public boolean add(T x);
public boolean remove(T x);
public boolean contains(T x);
}
Add item to set
Linked Lists
18

19. List-Based Sets
public interface Set<T> {
public boolean add(T x);
public boolean remove(T x);
public boolean contains(Tt x);
}
Remove item from set
Linked Lists
19

20. List-Based Sets
public interface Set<T> {
public boolean add(T x);
public boolean remove(T x);
public boolean contains(T x);
}
Is item in set?
Linked Lists
20

21. List Node
public class Node {
public T item;
public int key;
public Node next;
}
Linked Lists
21

22. List Node
public class Node {
public T item;
public int key;
public Node next;
}
item of interest
Linked Lists
22

23. List Node
public class Node {
public T item;
public int key;
public Node next;
}
Usually hash code
Linked Lists
23

24. List Node
public class Node {
public T item;
public int key;
public Node next;
}
Reference to next node
Linked Lists
24

25. The List-Based Set
-∞
a
b
c
+∞
Sorted with Sentinel nodes
(min & max possible keys)
Linked Lists
25

26. Invariants
• Sentinel nodes
– tail reachable from head
• Sorted
• No duplicates
Linked Lists
26

27. Sequential List Based Set
Add()
a
c
d
a
b
c
Remove()
Linked Lists
27

28. Sequential List Based Set
Add()
a
c
d
b
c
b
Remove()
a
Linked Lists
28

29. Coarse Grained Locking
a
b
Linked Lists
d
29

30. Coarse Grained Locking
a
d
b
c
Linked Lists
30

31. Coarse Grained Locking
a
d
b
honk!
honk!
c
Simple but hotspot + bottleneck
Linked Lists
31

32. Coarse-Grained Locking
• Easy, same as synchronized methods
– “One lock to rule them all …”
Linked Lists
32

33. Coarse-Grained Locking
• Easy, same as synchronized methods
– “One lock to rule them all …”
• Simple, clearly correct
– Deserves respect!
• Works poorly with contention
– Queue locks help
– But bottleneck still an issue
Linked Lists
33

34. Fine-grained Locking
• Requires careful thought
• Split object into pieces
– Each piece has own lock
– Methods that work on disjoint pieces
need not exclude each other
Linked Lists
34

35. Hand-over-Hand locking
a
b
Linked Lists
c
35

36. Hand-over-Hand locking
a
b
Linked Lists
c
36

37. Hand-over-Hand locking
a
b
Linked Lists
c
37

38. Hand-over-Hand locking
a
b
Linked Lists
c
38

39. Hand-over-Hand locking
a
b
Linked Lists
c
39

40. Removing a Node
a
b
c
d
remove(b)
Linked Lists
40

41. Removing a Node
a
b
c
d
remove(b)
Linked Lists
41

42. Removing a Node
a
b
c
d
remove(b)
Linked Lists
42

43. Removing a Node
a
b
c
d
remove(b)
Linked Lists
43

44. Removing a Node
a
b
c
d
remove(b)
Linked Lists
44

45. Removing a Node
a
remove(b)
c
d
Why hold 2 locks?
Linked Lists
45

46. Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Linked Lists
46

47. Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Linked Lists
47

48. Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Linked Lists
48

49. Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Linked Lists
49

50. Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Linked Lists
50

51. Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Linked Lists
51

52. Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Linked Lists
52

53. Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Linked Lists
53

54. Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Linked Lists
54

55. Concurrent Removes
a
b
c
d
remove(c)
remove(b)
Linked Lists
55

56. Uh, Oh
a
c
d
remove(c)
remove(b)
Linked Lists
56

57. Uh, Oh
Bad news, c not removed
a
c
d
remove(c)
remove(b)
Linked Lists
57

58. Problem
• To delete node c
– Swing node b’s next field to d
a
b
c
• Problem is,
– Someone deleting b concurrently could
direct a pointer
a
b
c
to c
Linked Lists
58

59. Insight
• If a node is locked
– No one can delete node’s successor
• If a thread locks
– Node to be deleted
– And its predecessor
– Then it works
Linked Lists
59

60. Hand-Over-Hand Again
a
b
c
d
remove(b)
Linked Lists
60

61. Hand-Over-Hand Again
a
b
c
d
remove(b)
Linked Lists
61

62. Hand-Over-Hand Again
a
b
c
d
remove(b)
Linked Lists
62

63. Hand-Over-Hand Again
a
b
c
d
Found
it!
remove(b)
Linked Lists
63

64. Hand-Over-Hand Again
a
b
c
d
Found
it!
remove(b)
Linked Lists
64

65. Hand-Over-Hand Again
a
c
d
remove(b)
Linked Lists
65

66. Removing a Node
a
b
c
d
remove(c)
remove(b)
Linked Lists
66

67. Removing a Node
a
b
c
d
remove(c)
remove(b)
Linked Lists
67

68. Removing a Node
a
b
c
d
remove(c)
remove(b)
Linked Lists
68

69. Removing a Node
a
b
c
d
remove(c)
remove(b)
Linked Lists
69

70. Removing a Node
a
b
c
d
remove(c)
remove(b)
Linked Lists
70

71. Removing a Node
a
b
c
d
remove(c)
remove(b)
Linked Lists
71

72. Removing a Node
a
b
c
d
remove(c)
remove(b)
Linked Lists
72

73. Removing a Node
a
b
c
d
remove(c)
remove(b)
Linked Lists
73

74. Removing a Node
a
b
c
d
remove(c)
Must
acquire
Lock of b
Linked Lists
74

75. Removing a Node
a
b
c
d
remove(c)
Cannot
acquire
lock of b
Linked Lists
75

76. Removing a Node
a
b
c
d
remove(c)
Wait!
Linked Lists
76

77. Removing a Node
a
b
d
Proceed
to
remove(b)
Linked Lists
77

78. Removing a Node
a
b
d
remove(b)
Linked Lists
78

79. Removing a Node
a
b
d
remove(b)
Linked Lists
79

80. Removing a Node
a
d
remove(b)
Linked Lists
80

81. Removing a Node
a
d
Linked Lists
81

82. Remove method
public boolean remove(Item item) {
int key = item.hashCode();
Node pred, curr;
try {
…
} finally {
curr.unlock();
pred.unlock();
}}
Linked Lists
82

83. Remove method
public boolean remove(Item item) {
int key = item.hashCode();
Node pred, curr;
try {
…
} finally {
curr.unlock();
pred.unlock();
}}
Key used to order node
Linked Lists
83

84. Remove method
public boolean remove(Item item) {
int key = item.hashCode();
Node pred, curr;
try {
…
} finally {
currNode.unlock();
predNode.unlock();
}}
Predecessor and current nodes
Linked Lists
84

85. Remove method
public boolean remove(Item item) {
int key = item.hashCode();
Node pred, curr;
try {
Make sure
…
locks released
} finally {
curr.unlock();
pred.unlock();
}}
Linked Lists
85

86. Remove method
public boolean remove(Item item) {
int key = item.hashCode();
Node pred, curr;
try {
…
} finally {
curr.unlock();
Everything else
pred.unlock();
}}
Linked Lists
86

87. Remove method
try {
pred = this.head;
pred.lock();
curr = pred.next;
curr.lock();
…
} finally { … }
Linked Lists
87

88. Remove method
lock pred == head
try {
pred = this.head;
pred.lock();
curr = pred.next;
curr.lock();
…
} finally { … }
Linked Lists
88

89. Remove method
try {
pred = this.head;
pred.lock();
curr = pred.next;
curr.lock();
…
} finally { … }
Linked Lists
Lock current
89

90. Remove method
try {
pred = this.head;
pred.lock();
curr = pred.next;
curr.lock();
…
} finally { … }
Linked Lists
Traversing list
90

91. Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Linked Lists
91

92. Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
Search key range
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Linked Lists
92

93. Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock(); At start of each loop:
pred = curr;
curr and pred locked
curr = curr.next;
curr.lock();
}
return false;
Linked Lists
93

94. Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return found,
If item false; remove node
Linked Lists
94

95. Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false; remove it
If node found,
Linked Lists
95

96. Remove: searching
Unlock predecessor
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Linked Lists
96

97. Remove: searching
Only one node locked!
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Linked Lists
97

98. Remove: searching
while (curr.key <= key) {
demote current
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Linked Lists
98

99. Remove: searching
while (curr.key <= key) {
Find and lock new current
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = currNode;
curr = curr.next;
curr.lock();
}
return false;
Linked Lists
99

100. Remove: searching
while (curr.key <= key) {
Lock invariant restored {
if (item == curr.item)
pred.next = curr.next;
return true;
}
pred.unlock();
pred = currNode;
curr = curr.next;
curr.lock();
}
return false;
Linked Lists
100

101. Remove: searching
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
Otherwise, not present
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Linked Lists
101

102. Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
•pred reachable from head
curr.lock();
•curr is pred.next
}
•So curr.item is in the set
return false;
Linked Lists
103

103. Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
Linearization point if
return false;
item is present
Linked Lists
104

104. Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
Node locked, so no other
return false;
thread can remove it ….
Linked Lists
105

105. Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;
Item not present
curr.lock();
}
return false;
Linked Lists
106

106. Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
pred.unlock();
pred = curr;
curr = curr.next;•pred reachable from head
•curr is pred.next
curr.lock();
•pred.key < key
}
•key < curr.key
return false;
Linked Lists
107

107. Why remove() is linearizable
while (curr.key <= key) {
if (item == curr.item) {
pred.next = curr.next;
return true;
}
Linearization point
pred.unlock();
pred = curr;
curr = curr.next;
curr.lock();
}
return false;
Linked Lists
108

108. Adding Nodes
• To add node e
– Must lock predecessor
– Must lock successor
• Neither can be deleted
– (Is successor lock actually required?)
Linked Lists
109

109. Drawbacks
• Better than coarse-grained lock
– Threads can traverse in parallel
• Still not ideal
– Long chain of acquire/release
– Inefficient
Linked Lists
110

110. Optimistic Synchronization
• Find nodes without locking
• Lock nodes
• Check that everything is OK
Linked Lists
111

111. Optimistic: Traverse without
Locking
a
b
d
e
Aha!
add(c)
Linked Lists
112

112. Optimistic: Lock and Load
a
b
d
e
add(c)
Linked Lists
113

113. Optimistic: Lock and Load
c
a
b
d
e
add(c)
Linked Lists
114

114. What could go wrong?
a
b
d
e
Aha!
add(c)
Linked Lists
115

115. What could go wrong?
a
b
d
e
add(c)
Linked Lists
116

116. What could go wrong?
a
b
d
e
remove(b
)
Linked Lists
117

117. What could go wrong?
a
b
d
e
remove(b
)
Linked Lists
118

118. What could go wrong?
a
b
d
e
add(c)
Linked Lists
119

119. What could go wrong?
c
a
b
d
e
add(c)
Linked Lists
120

120. What could go wrong?
a
add(c)
d
e
Uh-oh
Linked Lists
121

121. Validate – Part 1
a
b
d
e
Yes, b
still
reachable
from head
add(c)
Linked Lists
122

122. What Else Could Go Wrong?
a
b
d
e
Aha!
add(c)
Linked Lists
123

123. What Else Coould Go Wrong?
a
b
d
e
add(b’)
add(c)
Linked Lists
124

124. What Else Coould Go Wrong?
a
b
d
b’
e
add(b’)
add(c)
Linked Lists
125

125. What Else Could Go Wrong?
a
b
d
e
b’
add(c)
Linked Lists
126

126. What Else Could Go Wrong?
c
a
b
d
e
add(c)
Linked Lists
127

127. Validate Part 2
(while holding locks)
a
b
d
e
Yes, b
still points
to d
add(c)
Linked Lists
128

128. Optimistic: Linearization Point
a
b
d
e
c
add(c)
Linked Lists
129

129. Correctness
• If
– Nodes b and c both locked
– Node b still accessible
– Node c still successor to b
• Then
– Neither will be deleted
– OK to delete and return true
Linked Lists
130

130. Unsuccessful Remove
a
b
d
e
Aha!
remove(c)
Linked Lists
131

131. Validate (1)
a
b
d
e
Yes, b still
reachable
from head
remove(c)
Linked Lists
132

132. Validate (2)
a
b
d
e
Yes, b still
points to d
remove(c)
Linked Lists
133

133. OK Computer
a
b
remove(c)
d
e
return false
Linked Lists
134

134. Correctness
• If
– Nodes b and d both locked
– Node b still accessible
– Node d still successor to b
• Then
– Neither will be deleted
– No thread can add c after b
– OK to return false
Linked Lists
135

135. Validation
private boolean
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
}
Linked Lists
136

136. Validation
private boolean
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
} Predecessor &
return false;
current nodes
}
Linked Lists
137

137. Validation
private boolean
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
Begin at the
return false;
}
beginning
Linked Lists
138

138. Validation
private boolean
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
Search range of keys
return false;
}
Linked Lists
139

139. Validation
private boolean
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
Predecessor reachable
}
Linked Lists
140

140. Validation
private boolean
validate(Node pred,
Node curry) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
Is current node next?
}
Linked Lists
141

141. Validation
private boolean
Otherwise move on
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
}
Linked Lists
142

142. Validation
private boolean Predecessor not reachable
validate(Node pred,
Node curr) {
Node node = head;
while (node.key <= pred.key) {
if (node == pred)
return pred.next == curr;
node = node.next;
}
return false;
}
Linked Lists
143

143. Remove: searching
public boolean remove(Item item) {
int key = item.hashCode();
retry: while (true) {
Node pred = this.head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
} …
Linked Lists
144

144. Remove: searching
public boolean remove(Item item) {
int key = item.hashCode();
retry: while (true) {
Node pred = this.head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
} …
Search key
Linked Lists
145

145. Remove: searching
public boolean remove(Item item) {
int key = item.hashCode();
retry: while (true) {
Node pred = this.head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
} … Retry on synchronization conflict
Linked Lists
146

146. Remove: searching
public boolean remove(Item item) {
int key = item.hashCode();
retry: while (true) {
Node pred = this.head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
Examine predecessor and current nodes
} …
Linked Lists
147

147. Remove: searching
public boolean remove(Item item) {
int key = item.hashCode();
retry: while (true) {
Node pred = this.head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
Search by key
} …
Linked Lists
148

148. Remove: searching
public boolean remove(Item item) {
int key = item.hashCode();
retry: while (true) {
Node pred = this.head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
Stop if we find item
} …
Linked Lists
149

149. Remove: searching
public boolean remove(Item item) {
Move along
int key = item.hashCode();
retry: while (true) {
Node pred = this.head;
Node curr = pred.next;
while (curr.key <= key) {
if (item == curr.item)
break;
pred = curr;
curr = curr.next;
} …
Linked Lists
150

150. On Exit from Loop
• If item is present
– curr holds item
– pred just before curr
• If item is absent
– curr has first higher key
– pred just before curr
• Assuming no synchronization problems
Linked Lists
151

151. Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
} else {
return false;
}}} finally {
pred.unlock();
curr.unlock();
}}}
Linked Lists
152

152. Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
} else {
return false;
Always unlock
}}} finally {
pred.unlock();
curr.unlock();
}}}
Linked Lists
153

153. Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
} else {
return false;
Lock both nodes
}}} finally {
pred.unlock();
curr.unlock();
}}}
Linked Lists
154

154. Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
Check for synchronization
} else {
return false;
conflicts
}}} finally {
pred.unlock();
curr.unlock();
}}}
Linked Lists
155

155. Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
} else {
return false;
target found,
}}} finally {
remove node
pred.unlock();
curr.unlock();
}}}
Linked Lists
156

156. Remove Method
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.item == item) {
pred.next = curr.next;
return true;
target not found
} else {
return false;
}}} finally {
pred.unlock();
curr.unlock();
}}}
Linked Lists
157

157. Optimistic List
• Limited hot-spots
– Targets of add(), remove(), contains()
– No contention on traversals
• Moreover
– Traversals are wait-free
– Food for thought …
Linked Lists
158

158. So Far, So Good
• Much less lock acquisition/release
– Performance
– Concurrency
• Problems
– Need to traverse list twice
– contains() method acquires locks
Linked Lists
159

159. Evaluation
• Optimistic is effective if
– cost of scanning twice without locks
is less than
– cost of scanning once with locks
• Drawback
– contains() acquires locks
– 90% of calls in many apps
Linked Lists
160

160. Lazy List
• Like optimistic, except
– Scan once
– contains(x) never locks …
• Key insight
– Removing nodes causes trouble
– Do it “lazily”
Linked Lists
161

161. Lazy List
• remove()
– Scans list (as before)
– Locks predecessor & current (as before)
• Logical delete
– Marks current node as removed (new!)
• Physical delete
– Redirects predecessor’s next (as before)
Linked Lists
162

162. Lazy Removal
a
b
Linked Lists
c
d
163

163. Lazy Removal
a
b
c
d
Present in list
Linked Lists
164

164. Lazy Removal
a
b
c
d
Logically deleted
Linked Lists
165

165. Lazy Removal
a
b
c
d
Physically deleted
Linked Lists
166

166. Lazy Removal
a
b
d
Physically deleted
Linked Lists
167

167. Lazy List
• All Methods
– Scan through locked and marked nodes
– Removing a node doesn’t slow down other
method calls …
• Must still lock pred and curr nodes.
Linked Lists
168

168. Validation
•
•
•
•
No need to rescan list!
Check that pred is not marked
Check that curr is not marked
Check that pred points to curr
Linked Lists
169

169. Business as Usual
a
b
Linked Lists
c
170

170. Business as Usual
a
b
Linked Lists
c
171

171. Business as Usual
a
b
Linked Lists
c
172

172. Business as Usual
a
b
c
remove(b)
Linked Lists
173

173. Business as Usual
a
b
c
a not
marked
Linked Lists
174

174. Business as Usual
a
b
c
a still
points
to b
Linked Lists
175

175. Business as Usual
a
b
c
Logical
delete
Linked Lists
176

176. Business as Usual
a
b
c
physical
delete
Linked Lists
177

177. Business as Usual
a
b
Linked Lists
c
178

178. Validation
private boolean
validate(Node pred, Node curr) {
return
!pred.marked &&
!curr.marked &&
pred.next == curr);
}
Linked Lists
179

179. List Validate Method
private boolean
validate(Node pred, Node curr) {
return
!pred.marked &&
!curr.marked &&
pred.next == curr);
}
Predecessor not
Logically removed
Linked Lists
180

180. List Validate Method
private boolean
validate(Node pred, Node curr) {
return
!pred.marked &&
!curr.marked &&
pred.next == curr);
}
Current not
Logically removed
Linked Lists
181

181. List Validate Method
private boolean
validate(Node pred, Node curr) {
return
!pred.marked &&
!curr.marked &&
pred.next == curr);
}
Predecessor still
Points to current
Linked Lists
182

182. Remove
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.key == key) {
curr.marked = true;
pred.next = curr.next;
return true;
} else {
return false;
}}} finally {
pred.unlock();
curr.unlock();
}}}
Linked Lists
183

183. Remove
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.key == key) {
curr.marked = true;
pred.next = curr.next;
return true;
Validate as before
} else {
return false;
}}} finally {
pred.unlock();
curr.unlock();
}}}
Linked Lists
184

184. Remove
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.key == key) {
curr.marked = true;
pred.next = curr.next;
return true;
} else {
return false;
Key found
}}} finally {
pred.unlock();
curr.unlock();
}}}
Linked Lists
185

185. Remove
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.key == key) {
curr.marked = true;
pred.next = curr.next;
return true;
} else {
return false;
}}} finally {
pred.unlock(); Logical remove
curr.unlock();
}}}
Linked Lists
186

186. Remove
try {
pred.lock(); curr.lock();
if (validate(pred,curr) {
if (curr.key == key) {
curr.marked = true;
pred.next = curr.next;
return true;
} else {
return false;
}}} finally {
pred.unlock(); physical remove
curr.unlock();
}}}
Linked Lists
187

187. Contains
public boolean contains(Item item) {
int key = item.hashCode();
Node curr = this.head;
while (curr.key < key) {
curr = curr.next;
}
return curr.key == key && !curr.marked;
}
Linked Lists
188

188. Contains
public boolean contains(Item item) {
int key = item.hashCode();
Node curr = this.head;
while (curr.key < key) {
curr = curr.next;
}
return curr.key == key && !curr.marked;
}
Start at the head
Linked Lists
189

189. Contains
public boolean contains(Item item) {
int key = item.hashCode();
Node curr = this.head;
while (curr.key < key) {
curr = curr.next;
}
return curr.key == key && !curr.marked;
}
Search key range
Linked Lists
190

190. Contains
public boolean contains(Item item) {
int key = item.hashCode();
Node curr = this.head;
while (curr.key < key) {
curr = curr.next;
}
return curr.key == key && !curr.marked;
}
Traverse without locking
(nodes may have been removed)
Linked Lists
191

191. Contains
public boolean contains(Item item) {
int key = item.hashCode();
Node curr = this.head;
while (curr.key < key) {
curr = curr.next;
}
return curr.key == key && !curr.marked;
}
Present and undeleted?
Linked Lists
192

192. Summary: Wait-free Contains
a 0
b 0
d 1
c0
e 0
Use Mark bit + list ordering
1. Not marked  in the set
2. Marked or missing  not in the set
Linked Lists
193

193. Lazy List
a 0
b 0
d 1
c0
e 0
Lazy add() and remove() + Wait-free contains()
Linked Lists
194

194. Evaluation
• Good:
–
–
–
–
contains() doesn’t lock
In fact, its wait-free!
Good because typically high % contains()
Uncontended calls don’t re-traverse
• Bad
– Contended add() and remove() calls do
re-traverse
– Traffic jam if one thread delays
Linked Lists
195

195. Traffic Jam
• Any concurrent data structure based
on mutual exclusion has a weakness
• If one thread
– Enters critical section
– And “eats the big muffin”
• Cache miss, page fault, descheduled …
– Everyone else using that lock is stuck!
– Need to trust the scheduler….
Linked Lists
196

196. Reminder: Lock-Free Data
Structures
• No matter what …
– Guarantees minimal progress in any
execution
– i.e. Some thread will always complete a
method call
– Even if others halt at malicious times
– Implies that implementation can’t use
locks
Linked Lists
197

197. Lock-free Lists
• Next logical step
– Wait-free contains()
– lock-free add() and remove()
• Use only compareAndSet()
– What could go wrong?
Linked Lists
198

198. Remove Using CAS
Logical Removal =
Set Mark Bit
a 0
b 0
Use CAS to verify pointer
is correct
Not enough!
Linked Lists
c 1
c0
e 0
Physical
Removal
CAS pointer
199

199. Problem…
Logical Removal =
Set Mark Bit
a 0
b 0
c 1
c0
Problem:
Physical
d not added to list…
Removal
Must Prevent
CAS
manipulation of
removed node’s pointer
Linked Lists
e 0
d 0
Node added
Before
Physical
Removal CAS
200

200. The Solution: Combine Bit and
Pointer
Logical Removal =
Set Mark Bit
a 0
b 0
e 0
c 1
c0
d 0
Physical
Removal Fail CAS: Node not
added after logical
CAS
Removal
Mark-Bit and Pointer
are CASed together
(AtomicMarkableReference)
Linked Lists
201

201. Solution
• Use AtomicMarkableReference
• Atomically
– Swing reference and
– Update flag
• Remove in two steps
– Set mark bit in next field
– Redirect predecessor’s pointer
Linked Lists
202

202. Marking a Node
• AtomicMarkableReference class
– Java.util.concurrent.atomic package
Reference
address
F
mark bit
Linked Lists
203

203. Extracting Reference & Mark
Public Object get(boolean[] marked);
Linked Lists
204

204. Extracting Reference &
Mark
Public Object get(boolean[] marked);
Returns
reference
Returns mark at
array index 0!
Linked Lists
205

205. Extracting Mark Only
public boolean isMarked();
Value of
mark
Linked Lists
206

206. Changing State
Public boolean compareAndSet(
Object expectedRef,
Object updateRef,
boolean expectedMark,
boolean updateMark);
Linked Lists
207

207. Changing State
If this is the current
reference …
Public boolean compareAndSet(
Object expectedRef,
Object updateRef,
boolean expectedMark,
boolean updateMark);
And this is the
current mark …
Linked Lists
208

208. Changing State
…then change to this
new reference …
Public boolean compareAndSet(
Object expectedRef,
Object updateRef,
boolean expectedMark,
boolean updateMark);
… and this new
mark
Linked Lists
209

209. Changing State
public boolean attemptMark(
Object expectedRef,
boolean updateMark);
Linked Lists
210

210. Changing State
public boolean attemptMark(
Object expectedRef,
boolean updateMark);
If this is the current
reference …
Linked Lists
211

211. Changing State
public boolean attemptMark(
Object expectedRef,
boolean updateMark);
.. then change to
this new mark.
Linked Lists
212

212. Removing a Node
a
CAS
b
c
d
remov
e c
Linked Lists
213

213. Removing a Node
failed
CAS
a CAS b
c
d
remov
e c
remov
e b
Linked Lists
214

214. Removing a Node
a
b
c
d
remov
e c
remov
e b
Linked Lists
215

215. Removing a Node
a
d
remov
e c
remov
e b
Linked Lists
216

216. Traversing the List
• Q: what do you do when you find a
“logically” deleted node in your path?
• A: finish the job.
– CAS the predecessor’s next field
– Proceed (repeat as needed)
Linked Lists
217

217. Lock-Free Traversal
(only Add and Remove)
pred
curr
pred
a
CAS
curr
b
c
d
Uh-oh
Linked Lists
218

218. The Window Class
class Window
public Node
public Node
Window(Node
this.pred
}
}
{
pred;
curr;
pred, Node curr) {
= pred; this.curr = curr;
Linked Lists
219

219. The Window Class
class Window
public Node
public Node
Window(Node
this.pred
}
}
{
pred;
curr;
pred, Node curr) {
= pred; this.curr = curr;
A container for pred
and current values
Linked Lists
220

220. Using the Find Method
Window window = find(head, key);
Node pred = window.pred;
curr = window.curr;
Linked Lists
221

221. Using the Find Method
Window window = find(head, key);
Node pred = window.pred;
curr = window.curr;
Find returns window
Linked Lists
222

222. Using the Find Method
Window window = find(head, key);
Node pred = window.pred;
curr = window.curr;
Extract pred and curr
Linked Lists
223

223. The Find Method
Window window = find(item);
item
At some instant,
pred
curr
or …
succ
Linked Lists© Herlihy-Shavit 2007
224

224. The Find Method
Window window = find(item);
At some instant,
pred
curr= null
item
not in list
succ
Linked Lists© Herlihy-Shavit 2007
225

225. Remove
public boolean remove(T item) {
Boolean snip;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.attemptMark(succ, true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
Linked Lists
226

226. Remove
public boolean remove(T item) {
Boolean snip;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.attemptMark(succ, true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
Linked Lists
Keep trying
227

227. Remove
public boolean remove(T item) {
Boolean snip;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.attemptMark(succ, true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
Find neighbors
Linked Lists
228

228. Remove
public boolean remove(T item) {
Boolean snip;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.attemptMark(succ, true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
She’s not there …
Linked Lists
229

229. Remove
public boolean remove(T item) {
Boolean Try to mark node as deleted
snip;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.attemptMark(succ, true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
Linked Lists
230

230. Remove
public boolean remove(T item) {
Boolean snip;
If it doesn’t
while just {
work,(true) retry,
Window window = find(head, key);
if it does, job
Node pred = window.pred, curr = window.curr;
if (curr.key !=
essentially donekey) {
return false;
} else {
Node succ = curr.next.getReference();
snip = curr.next.attemptMark(succ, true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
Linked Lists
231

231. Remove
public boolean remove(T item) {
Boolean snip;
while (true) {
a
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key != key) {
return false;
Try to advance reference
} else {
(if succ = curr.next.getReference();
Node we don’t succeed, someone else did or will).
snip = curr.next.attemptMark(succ, true);
if (!snip) continue;
pred.next.compareAndSet(curr, succ, false, false);
return true;
}}}
Linked Lists
232

232. Add
public boolean add(T item) {
boolean splice;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key == key) {
return false;
} else {
Node node = new Node(item);
node.next = new AtomicMarkableRef(curr, false);
if (pred.next.compareAndSet(curr, node, false,
false)) {return true;}
}}}
Linked Lists
233

233. Add
public boolean add(T item) {
boolean splice;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key == key) {
return false;
} else {
Node node = new Node(item);
node.next = new AtomicMarkableRef(curr, false);
if (pred.next.compareAndSet(curr, node, false,
false)) {return true;}
}}}
Item already there.
Linked Lists
234

234. Add
public boolean add(T item) {
boolean splice;
while (true) {
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key == key) {
return false;
} else {
Node node = new Node(item);
node.next = new AtomicMarkableRef(curr, false);
if (pred.next.compareAndSet(curr, node, false,
false)) {return true;}
}}}
create new node
Linked Lists
235

235. Add
public boolean add(T item) {
boolean splice;
Install new node,
while (true) {
else retry loop
Window window = find(head, key);
Node pred = window.pred, curr = window.curr;
if (curr.key == key) {
return false;
} else {
Node node = new Node(item);
node.next = new AtomicMarkableRef(curr, false);
if (pred.next.compareAndSet(curr, node, false,
false)) {return true;}
}}}
Linked Lists
236

236. Wait-free Contains
public boolean contains(Tt item) {
boolean marked;
int key = item.hashCode();
Node curr = this.head;
while (curr.key < key)
curr = curr.next;
Node succ = curr.next.get(marked);
return (curr.key == key && !marked[0])
}
Linked Lists
237

237. Wait-free Contains
public boolean contains(T item) {
Only diff is that we
boolean marked;
get and check
int key = item.hashCode();
Node curr = this.head; marked
while (curr.key < key)
curr = curr.next;
Node succ = curr.next.get(marked);
return (curr.key == key && !marked[0])
}
Linked Lists
238

238. Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
while (marked[0]) {
…
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Linked Lists
239

239. Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
If list changes
while (true) {
while
succ = curr.next.get(marked);
while (marked[0]) {
traversed,
…
start over
}
Lock-Free
if (curr.key >= key)
because we
return new Window(pred, curr);
pred = curr;
start over only
curr = succ;
if someone else
}
}}
Linked Lists
makes progress
240

240. Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr looking succ = head
Start = null, from null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
while (marked[0]) {
…
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Linked Lists
241

241. Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) { Move down the list
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
while (marked[0]) {
…
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Linked Lists
242

242. Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
while (marked[0]) {
…
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = Get ref to successor
succ;
}
}}
and
current deleted bit
Linked Lists
243

243. Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
while (marked[0]) {
…
}
Try
}}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
to remove deleted nodes in
}
path…code details soon
Linked Lists
244

244. Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
Ifwhile (true)that is greater or
curr key {
succ
curr.next.get(marked);
equal,= return pred and curr
while (marked[0]) {
…
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Linked Lists
245

245. Lock-free Find
public Window find(Node head, int key) {
Node pred = null, curr = null, succ = null;
boolean[] marked = {false}; boolean snip;
retry: while (true) {
pred = head;
curr = pred.next.getReference();
while (true) {
succ = curr.next.get(marked);
Otherwise advance window and
while (marked[0]) {
…
loop again
}
if (curr.key >= key)
return new Window(pred, curr);
pred = curr;
curr = succ;
}
}}
Linked Lists
246

246. Lock-free Find
retry: while (true) {
…
while (marked[0]) {
snip = pred.next.compareAndSet(curr,
succ, false, false);
if (!snip) continue retry;
curr = succ;
succ = curr.next.get(marked);
}
…
Linked Lists
247

247. Lock-free Find
Try to snip out node
retry: while (true) {
…
while (marked[0]) {
snip = pred.next.compareAndSet(curr,
succ, false, false);
if (!snip) continue retry;
curr = succ;
succ = curr.next.get(marked);
}
…
Linked Lists
248

248. Lock-free Find
if predecessor’s next field
changed must retry whole
traversal
{
retry: while (true)
…
while (marked[0]) {
snip = pred.next.compareAndSet(curr,
succ, false, false);
if (!snip) continue retry;
curr = succ;
succ = curr.next.get(marked);
}
…
Linked Lists
249

249. Lock-free Find
Otherwise move on to
check if next node deleted
retry: while (true) {
…
while (marked[0]) {
snip = pred.next.compareAndSet(curr,
succ, false, false);
if (!snip) continue retry;
curr = succ;
succ = curr.next.get(marked);
}
…
Linked Lists
250

250. Performance
On 16 node shared memory machine
Benchmark throughput of Java List-based Set
algs. Vary % of Contains() method Calls.
Linked Lists
251

251. High Contains Ratio
Lock-free
Lazy list
Course Grained
Fine Lock-coupling
Linked Lists
252

252. Low Contains Ratio
Lock-free
Lazy list
Course Grained
Fine Lock-coupling
Linked Lists
253

253. As Contains Ratio Increases
Lock-free
Lazy list
Course Grained
Fine Lock-coupling
% Contains()
Linked Lists
254

254. Summary
•
•
•
•
Coarse-grained locking
Fine-grained locking
Optimistic synchronization
Lock-free synchronization
Linked Lists
255