Managing Linux network namespaces in Go 1.10 version and before.

1. Twitter: @prateekgogia
Github: @prateekgogia

2. Docker daemon
eth0 Linux machine
Docker
Daemon

3. Docker runtime adopted Go
• Promoted wider adoption of Go as a systems language
• Bug fixes in Docker produced more Go programmers
• Identifying systems level problem in Go runtime related to syscalls / threads

4. Isolation in Docker container: namespaces
• Process trees (PID Namespace)
• Mounts (MNT namespace)
• Network (Net namespace)
• Users / UIDs (User Namespace)
• Hostnames (UTS Namespace)
• Inter Process Communication (IPC Namespace)

5. Lets see this in action

6. Running a Go Process on Linux machine
eth0 Linux machine
Go Process
OS thread
Goroutine

7. Add new Network Namespace with lo
(loopback) only interface
eth0
net namespaces n
Linux machine
Go Process
OS thread
Goroutine

8. Adding worker goroutines to new network
namespaces
eth0
Worker
goroutines
net namespaces n
Linux machine
Go Process
OS thread
Goroutine
runtime.LockOSThread()

9. Adding worker goroutines to host network
namespace
eth0
Worker
goroutines
net namespaces n
Linux machine
Go Process
OS thread
Goroutine
runtime.LockOSThread()

10. Adding worker goroutines to host network
namespace
eth0
more
Worker
goroutines
net namespaces n
Linux machine Go Process
OS thread
Goroutine
runtime.LockOSThread()

11. Incorrect interfaces detected by goroutines
eth0
more
Worker
goroutines
net namespaces n
Linux machine Go Process
goroutine
with wrong
interface info

12. Go runtime scheduler
M P G G G G
M P G
M
P
G
OS thread
Processor (Logical)
Goroutine

13. Go runtime scheduler
M P G G G G
M P G G G
Global queue
M
P
G
OS thread
Processor (Logical)
Goroutine

14. Go runtime Scheduler
M P G G G G
M P G G G
M P G G G
M P G G G G G G
Global queue
M
P
G
OS thread
Processor (Logical)
Goroutine

15. Go runtime Scheduler
M P G G G G
M P G G G
M P G G G
M P G G G G G G
Global queue
New threads inherit
state from already
running threads

16. Go runtime model
eth0
more
Worker
goroutines
net namespaces n
Linux machine Go Process
OS thread
Goroutine
runtime.LockOSThread()

17. Go runtime model
eth0
more
Worker
goroutines
net namespaces n
Linux machine Go Process

18. How Go 1.10 Fixed it

19. Go runtime scheduler
M P G G G G
M P G
M
P
G
OS thread
Processor (Logical)
Goroutine

20. Go runtime scheduler
M P G
M P G
M
P
G
OS thread
Processor
(Logical)
Goroutine
runtime.LockOSThread()

21. Go runtime scheduler in go 1.10
M P G
M P G
startTemplateThread()
M
P
G
OS thread
Processor
(Logical)
Goroutine
runtime.LockOSThread()

22. LockOSThread() in go 1.10
M P G
M P G
Template thread
M
P
G
OS thread
Processor (Logical)
Goroutine
runtime.LockOSThread()

23. Go runtime scheduler in 1.10
M P G
M P G
Template thread
M
P
G
OS thread
Processor (Logical)
Goroutine
runtime.LockOSThread()
M P G
M P G

24. Template thread creation in Go Process
eth0
net namespaces n
Linux machine
Go Process
Template
thread

25. Adding worker goroutines to host network
namespace
eth0
net namespaces n
Linux machine
Go Process
Template
thread

26. Summary
• If creating / managing Linux network namespaces using Go, use Go
version >= 1.10

27. Thank You!!!

28. runtime.LockOSThread()
M P G
M P G
M P G G G
M P G G G G G G
Global queue
Goroutine called
LockOSThread()
M
P
G
OS thread
Processor
(Logical)
Goroutine
runtime.LockOSThread()

29. Docker runtime adopted Go ??
• Static compilation
• Asynchronous primitives
• Multi-arch builds
• Faster development
• Awesome Documentations

31. How to manage network namespaces
with Go 1.10 and beyond
ns := createNetNamespace()
runtime.LockOSThread()
defer runtime.UnlockOSThread()
ns.Do(func(_ ns.NetNS) error {
runtime.LockOSThread()
cb()
}
If the goroutine here changes the state of the
thread, let the thread exit with goroutine

32. Bonus feature
• OS thread terminates if goroutine exits
• runtime.LockOSThread() is nested.
• UnlockOSThread must be called the same number of times in order to unlock the thread.