Mutexes and locks are used in Go and C to synchronize access to shared resources between threads. The document discusses various lock types including spin locks, mutexes, and reader-writer locks. It provides code examples from Go and glibc to illustrate how locks are implemented and compares the approaches. Specifically, it covers how Go's sync.Mutex implements fairness and compares it to pthread mutexes. The document also discusses data race detection in Go.

1. An Introduction to Locks in Go
Cherie Hsieh
Golang Taiwan, Gopher Taipei

2. Source Code Versions
Go version 1.12
glibc version 2.29

3. Mutex in Go
Before discuss this topic, we briefly review the basic concept of some
fundamental locks.

4. Spin Lock
• Appropriate for multi-core processors
In uniprocessor, a waiting thread spend all its time
slice on spinning
• The most common lock in linux kernel
Spin lock is used in interrupt handlers which cannot
be rescheduled
• Hold a spin lock for a short time
To prevent heavy CPU consumption

5. Spin Lock
Question: Why is glibc spin lock unfair ?
int pthread_spin_lock (pthread_spinlock_t *lock)
{
int val = 0;
if (__glibc_likely(atomic_compare_exchange_weak_acquire (lock, &val, 1)))
return 0;
do
{
do
{
atomic_spin_nop(); // Important!! CPU pause
val = atomic_load_relaxed(lock);
}
while (val != 0);
}
while (!atomic_compare_exchange_weak_acquire (lock, &val, 1));
return 0;
}
Difference from ticket spin lock, the lock is not fair as it doesn’t guarantee FIFO
ordering amongst the threads competing for the lock. [source code]

6. Semaphore
• Sleeping locks
• Obtained only in process context which is schedulable.
• Allow for an arbitrary number of simultaneous lock holder.
// Example: glibc semaphore struct
// partial code from internaltypes.h
struct new_sem
{
unsigned int value; // Max to 2^31-1
unsigned int nwaiter; // 決定是否要執⾏行行 futex wake system call
}

7. Mutex
• Sleeping lock that implements mutual exclusion
• For Linux kernel, whoever locked a mutex must unlock it
• Difference between semaphore and mutex - mutex introduces thread
ownership
// partial code from thread-shared-types.h
struct struct __pthread_mutex_s
{
int __lock;
unsigned int __count;
int __owner; // ppid
}

8. Mutex
The __owner structure member helps us to debug mutex.
$2 = {__data = {__lock = 1, __count = 0, __owner = 4060, __nusers = 1, __kind = 0, __spins = 0, __elision = 0,
__list = {__prev = 0x0, __next = 0x0}},
__size = "001000000000000000000000334017000000001", '000' <repeats 26 times>, __align = 1}
(gdb) info thread
Id Target Id Frame
* 2 Thread 0x7ffff77f1700 (LWP 4061) "mutex.o" unlock_thread (args=0x7fffffffe330) at mutex.c:8
1 Thread 0x7ffff7fee740 (LWP 4060) "mutex.o" 0x00007ffff7bc7f47 in pthread_join () from /lib64/
libpthread.so.0

9. Mutex
• glibc pthread_mutex_lock
The mutex of gnu C library does not define a behavior when it is unlocked by
other threads. The _owner member variable will be reset to zero without any
owner validation as mutex is unlocking.
Mutex Type Relock Unlock When Not Owner
DEFAULT deadlock undefined behavior

10. Mutex in Go sync.Mutex
• Hybrid Mutex - Combination of spin lock and mutex
• Two modes: starvation, normal that make it fair

11. Mutex in Go sync.Mutex
Normal Mode
Can spin?
1. Only if running on a multicore machine
2. Spin only few times( < 4 )
3. At least one other running P and local runq
is empty

12. Mutex in Go sync.Mutex
Normal Mode
Problems
A woken up waiter does not own the mutex
and competes with new arriving goroutines
over the ownership. New arriving goroutines
have a greater chance to get the lock.

13. Mutex in Go sync.Mutex
Starvation Mode
New arriving goroutine will be pushed to the waiting queue to guarantee FIFO
ordering.

14. Mutex in Go sync.Mutex
Switch from starvation to normal
• The duration of waiting for locks is less than 1ms
• No other goroutine is waiting for this lock
Switch from normal to starvation
• The duration of waiting for locks is longer than 1ms

15. Comparison pthread_mutex, sync.Mutex
1. No owner member in Go mutex struct
It means goroutines can unlock the mutex locked by another goroutine
struct {
state int32
sema uint32
}
2. pthread_mutex: Hybrid Mutex (Type: PTHREAD_MUTEX_ADAPTIVE_NP)
sync.Mutex: Hybrid Mutex & normal / starvation mode

16. Comparison pthread_mutex, sync.Mutex
3. pthread_mutex: Other threads can get the lock when the lock owner is
terminated.
static pthread_mutex_t mtx;
static void * original_owner_thread(void *ptr)
{
pthread_mutex_lock(&mtx);
pthread_exit(NULL);
}
int main(int argc, char *argv[])
{
pthread_t thr;
pthread_mutexattr_t attr;
int s;
pthread_mutexattr_init(&attr);
pthread_mutexattr_setrobust(&attr, PTHREAD_MUTEX_ROBUST);
pthread_mutex_init(&mtx, &attr);
pthread_create(&thr, NULL, original_owner_thread, NULL);
sleep(2);
s = pthread_mutex_lock(&mtx);
if (s == EOWNERDEAD) {
pthread_mutex_consistent(&mtx);
pthread_mutex_unlock(&mtx);
exit(EXIT_SUCCESS);
}
}

17. Reader-Writer Locks
1. Reader preferred
2. Shared(read)–Exclusive(write) Locks
3. Use reader/writer or consumer/producer usage patterns.

18. Reader-Writer Locks
This reader-writer lock allow for maximum concurrency, but can lead to
write-starvation if contention is high. This is because writer threads will
not be able to acquire the lock as long as at least one reading thread
holds it.

19. Reader-Writer Locks in Go RWMutex
So, common implementation of reader–writer locks usually block
additional readers if a lock is already held in read mode and a thread is
blocked trying to acquire the lock in write mode.
This prevents a constant stream of readers from starving waiting writers.
Go sync.RWMutex use this implementation
as well.

20. Comparison pthread_rwlock, sync.RWMutex
pthread_rwlock
Different modes:
• PTHREAD_RWLOCK_PREFER_READER_NP (may lead write-starvation)
• PTHREAD_RWLOCK_PREFER_WRITER_NONRECURSIVE_NP
sync.RWMutex
1. In particular, it prohibits recursive read locking.
2. Prevent starving waiting writers.

21. Scalability Issues
type RWMutex struct {
w Mutex // held if there are pending writers
writerSem uint32 // semaphore for writers to wait for completing readers
readerSem uint32 // semaphore for readers to wait for completing writers
readerCount int32 // number of pending readers
readerWait int32 // number of departing readers
}
The default Go implementation of sync.RWMutex does not scale well to
multiple cores, as all readers contend on the same memory location
when they all try to atomically increment it.

22. Data Race Detection
ThreadSanitizer is a data race detector for C/C++
Data races are one of the most common and hardest to debug types of bugs in
concurrent systems. Go integrates ThreadSanitizer into CLI tools so that you can
trigger it by adding a race flag.
Tools: ThreadSanitizer
Go Command Option -race
go test -race
go run -race main.go

23. Data Race Detection
The Difference of Data Race and Race Condition
Race Condition
The system's substantive behavior is dependent on the sequence or timing of other
uncontrollable events.
Data race
occurs when two threads access the same variable concurrently and at least one of
the accesses is write.
ThreadSanitizer can not detect race condition without data race.

24. Data Race Detection
// Race condition without data race
var from int32 = 10
var to int32 = 20
func change(g *sync.WaitGroup, number int32, from, to *int32) error {
defer g.Done()
n := atomic.LoadInt32(from)
for i := 0; i < 100; i++ {} // busy business logic
if n < number {
return fmt.Errorf("not enough")
}
atomic.AddInt32(to, number)
atomic.AddInt32(from, -number)
return nil
}
func Test(t *testing.T) {
var group sync.WaitGroup
group.Add(3)
go change(&group, 7, &from, &to)
go change(&group, 4, &from, &to)
go change(&group, 1, &from, &to)
group.Wait()
}

25. References
Books
1. Linux Kernel Development
2. Perfbook
Posts
1. Race Condition vs. Data Race