When you drink a soda, do you recycle the can? When you allocate a slice, do you recycle the memory? Recycling cans is good for the planet and recycling slices can be good for your Go program. But how? The garbage collector? A sync.Pool? Something else? You’ll be surprised what a difference it makes!

1. Slice Recycling
Performance and Pitfalls
David Golden
@xdg • https://xdg.me/
Staff Engineer, MongoDB
Gotham Go 2019

2. Garbage!

3. Discarded memory

6. Go provides a
garbage collector

8. Does this really matter?

10. Call stack
(box per fcn)
Execution time

11. Call stack
(box per fcn)
Execution time
Left-to-right order is meaningless

12. Execution time

13. null

17. What’s going on?

18. What does this have to
do with slices?

19. ⚠
Gross simplification
ahead…

20. GC runs when live heap
memory doubles

21. GC steals time
when it can’t keep up

22. Allocate less often!

24. sync.Pool
recycles objects
between GC cycles

25. But…
Not designed for slices

26. Not size aware!

27. Slice capacities in
the pool can differ

28. Pool order is undefined!

29. ‘I have a
little
water’

30. ‘I have a
little
water’
Oops!
Pool gave me a
giant container

31. ‘I have a
lot of
water’

32. ‘I have a
lot of
water’
Oops!
Pool gave me a
tiny container

33. Think about append…

34. xs = append(xs, ys…)

35. xs = append(xs, ys…)
Not always the same slice!

36. xs = append(xs, ys…)

37. xs = append(xs, ys…)

38. xs = append(xs, ys…)
This is thrown away!
X

40. What can we do?

41. Consider a trivial
encoder

42. Consider a trivial
encoder
?!

43. string keys
integer values

44. Packed binary format
42“foo0” 99“bar0”

45. Variable length!

46. Pluggable memory
management

47. type BytePool interface {
Get() []byte
Put([]byte)
Resize(orig []byte, n int) []byte
}

48. type BytePool interface {
Get() []byte
Put([]byte)
Resize(orig []byte, n int) []byte
}
type poolEncoder struct {
pool BytePool
buf []byte
}

49. Uses the pool
for allocations

50. Uses the pool for resizing
(not append)

51. Releases the buffer
to the pool

52. Four strategies…

53. 1. ‘null’ – use the garbage collector

54. 1. ‘null’ – use the garbage collector
2. ‘sync’ – use a sync.Pool

55. 1. ‘null’ – use the garbage collector
2. ‘sync’ – use a sync.Pool
3. ‘power2’ – sync.Pool per size bucket

56. 1. ‘null’ – use the garbage collector
2. ‘sync’ – use a sync.Pool
3. ‘power2’ – sync.Pool per size bucket
4. ‘reserved’ – keep N buffers around forever

57. 1. null

58. Use the garbage collector

59. type nullPool struct{}

60. type NullPool struct{}
func (np NullPool) Get() []byte {
return make([]byte, 0, startCap)
}

61. type NullPool struct{}
func (np NullPool) Get() []byte {
return make([]byte, 0, startCap)
}
func (np NullPool) Put(bs []byte) {}

62. type NullPool struct{}
func (np NullPool) Get() []byte {
return make([]byte, 0, startCap)
}
func (np NullPool) Put(bs []byte) {}
func (np NullPool) Resize(orig []byte, size int) []byte {
}

63. type NullPool struct{}
func (np NullPool) Get() []byte {
return make([]byte, 0, startCap)
}
func (np NullPool) Put(bs []byte) {}
func (np NullPool) Resize(orig []byte, size int) []byte {
if size < cap(orig) {
return orig[0:size]
}
}

64. type NullPool struct{}
func (np NullPool) Get() []byte {
return make([]byte, 0, startCap)
}
func (np NullPool) Put(bs []byte) {}
func (np NullPool) Resize(orig []byte, size int) []byte {
if size < cap(orig) {
return orig[0:size]
}
temp := make([]byte, size, max(size, cap(orig)*2))
copy(temp, orig)
return temp
}

65. 2. sync

66. Use a sync.pool

67. type SyncPool struct {
pool *sync.Pool
}

68. func (sp SyncPool) Get() []byte {
}

69. func (sp SyncPool) Get() []byte {
bp := sp.pool.Get() // returns interface{}
}

70. func (sp SyncPool) Get() []byte {
bp := sp.pool.Get() // returns interface{}
if bp == nil {
return make([]byte, 0, startCap)
}
}

71. func (sp SyncPool) Get() []byte {
bp := sp.pool.Get()
if bp == nil {
return make([]byte, 0, startCap)
}
buf := bp.([]byte)
for i := range buf {
buf[i] = 0
}
return buf[0:0]
}

72. func (sp SyncPool) Get() []byte {
bp := sp.pool.Get()
if bp == nil {
return make([]byte, 0, startCap)
}
buf := bp.([]byte)
for i := range buf {
buf[i] = 0
}
return buf[0:0]
}
Optimized away by compiler!

73. func (sp SyncPool) Get() []byte {
bp := sp.pool.Get()
if bp == nil {
return make([]byte, 0, startCap)
}
buf := bp.([]byte)
for i := range buf {
buf[i] = 0
}
return buf[0:0]
}
func (sp SyncPool) Put(bs []byte) {
sp.pool.Put(bs)
}

74. func (sp SyncPool) Resize(orig []byte, size int) []byte {
}

75. func (sp SyncPool) Resize(orig []byte, size int) []byte {
if size < cap(orig) {
return orig[0:size]
}
temp := make([]byte, size, max(size, cap(orig)*2))
copy(temp, orig)
sp.Put(orig)
return temp
}

76. 3. power2

77. sync.pool
per size bucket

78. type Power2Pool struct {
pools []*sync.Pool
}

79. type Power2Pool struct {
pools []*sync.Pool
}
Skipping the details!

80. Pools organized by
power-of-2 capacities

81. Get() and Resize()
call size-aware getn()

82. Put() also size-aware

83. 4. reserved

84. Keep big containers around

85. type ReservedPool struct {
sync.Mutex
pool [20][]byte
size int
targetCap int
}

86. type ReservedPool struct {
sync.Mutex
pool [20][]byte
size int
targetCap int
}
Skipping the details!

87. Capped LIFO stack

88. Resizing grows
target capacity

89. Put() drops slices
smaller than target

90. 1. ‘null’ – use the garbage collector
2. ‘sync’ – use a sync.Pool
3. ‘power2’ – sync.Pool per size bucket
4. ‘reserved’ – keep N buffers around forever

91. Ways to analyze?

92. • Benchmarking
• Max heap analysis
• Profiling – flame graphs
• Trace browser

93. Workload
• N items to encode
• 20 goroutines encoding
• Up to 1000 keys per encoding
• Up to 200 characters per key

94. Isolation
• Pre-generate random data lists outside loops
• Turn on profiler after data generation
• Constant randSeed for repeatability
• Goroutines iterate data lists from random
starting points
• Profile/trace with standalone executable

95. xdg-go/zzz-slice-recycling

96. Benchmarking

97. go test -bench .

98. null 17354 ns/op
sync 5264 ns/op
power2 4333 ns/op
reserved 2945 ns/op

99. Maximum heap(?!)

100. GODEBUG=gctrace=1

101. Debug line per GC run

102. ... 13->26->16 MB ...
... 17->41->27 MB ...
... 28->36->10 MB ...
... 14->32->20 MB ...
... 21->24->5 MB ...
... 8->22->17 MB ...
... 18->29->14 MB ...
... 16->25->12 MB ...
... 16->35->22 MB ...
... 23->47->26 MB ...
... 27->45->20 MB ...

103. ... 13->26->16 MB ...
... 17->41->27 MB ...
... 28->36->10 MB ...
... 14->32->20 MB ...
... 21->24->5 MB ...
... 8->22->17 MB ...
... 18->29->14 MB ...
... 16->25->12 MB ...
... 16->35->22 MB ...
... 23->47->26 MB ...
... 27->45->20 MB ...
Max heap at GC end

104. null 17354 ns/op 141 MB
sync 5264 ns/op 35 MB
power2 4333 ns/op 9 MB
reserved 2945 ns/op 19 MB

105. Profiling

106. $ ./main -pooltype <type> -profile cpu
$ go tool pprof -http :8080
./main ./cpu.pprof

108. null

109. sync
null

110. power2
null

111. reserved
null

112. Tracing

113. $ ./main -pooltype <type> -profile trace
$ go tool trace -http :8080
./main ./trace.pprof

115. null

116. null
Type '?' for nav help

117. null

118. null

119. null

120. null

121. sync

122. power2

123. reserved

124. • Benchmarking
• Max heap analysis
• Profiling – flame graphs
• Trace browser

125. What about naive
approach?

127. func (sp LeakySyncPool) Resize(orig []byte, size int) []byte {
if size < cap(orig) {
return orig[0:size]
}
temp := make([]byte, size, max(size, cap(orig)*2))
copy(temp, bs)
sp.Put(bs)
return temp
}

128. null 17354 ns/op 141 MB
sync 5264 ns/op 35 MB
power2 4333 ns/op 9 MB
reserved 2945 ns/op 19 MB
leakysync 3183 ns/op 9 MB

129. What's going on?!?

137. Reserved!?

138. leaky → large containers

139. Larger starting capacity?

140. 1000x
startCap 256 bytes 256,000 bytes

141. 1000x
startCap 256 bytes 256,000 bytes
null 17354 ns/op
sync 5264 ns/op
power2 4333 ns/op
reserved 2945 ns/op
leakysync 3183 ns/op

142. 1000x
startCap 256 bytes 256,000 bytes
null 17354 ns/op 13633 ns/op
sync 5264 ns/op 2839 ns/op
power2 4333 ns/op 2975 ns/op
reserved 2945 ns/op 2659 ns/op
leakysync 3183 ns/op 3082 ns/op

143. Take aways

144. Don’t optimize early!

145. Don’t get clever!

146. Trade space for speed

147. Resizing hurts!

148. Allocate for the 99%tile

149. Happy recycling!

150. Thank you!
@xdg david@mongodb.com https://xdg.me
https://github.com/xdg-go/zzz-slice-recycling