1.
Go @ Tokopedia Qasim Zaidi

2.
About Tokopedia
• Launched in 2009
• 300K Active merchants
• 8 Million Products
• Indonesia’s biggest online marketplace
• Everyone pays shipping charges
• Shipping is a function of distance
• No discounting or cash-backs, neutral marketplace.
• Most payments are offline

3.
nginx
ssl termination and
proxy pass
nginx-mod_perl
nginx-mod_perl
nginx-mod_perl
The Stack
postgres
mongo
redis
proxy_pass

4.
What we already use go for
• Search & Discovery
• Image Uploads
• Analytics

5.
60% of the time, it works every time.

6.
nginx
ssl termination and
proxy pass
nginx-mod_perl
nginx-mod_perl
nginx-mod_perl
The Stack
postgres
mongo
redis
proxy_pass
mod_perl blocks nginx
isolation is hard
cancellation is harder

7.
Challenges of scaling with mod_perl
• Each request is
synchronously
processed in a worker
• If there is one bad
request, it ends up
queuing everything else.
• Cancellation is not easy
- so even when the user
moves away, the request
continues to run.

8.
In 2012, we bet on node.js

9.
scaled well too - from 50K transactions in 2012 to over a million in 2015

10.
node.js challenges
• Too easy to mess up (dynamic, no type checking)
• linting, code reviews
• Unbounded Concurrency is hard.
• async, promises
• Scale issues with http client for c10k
• even dns latencies can have big impact

11.
Picking a new language in 2015.

12.
the no QA philosophy
QA is not a different role.
Developers are responsible for
testing their code.

13.
Testing in Go
• Easy to write unit tests
func TestFoo(t *testing.T) {
...
}
• run as go test
• No separate frameworks
• Benchmarking is as easy as testing

14.
the no Documentation
philosophy
Code should be self explanatory.
Documentation outside of code is
never updated.

15.
Documentation with GoDoc
• Comments in code - e.g.
preceding function definition
• It extracts the comments
and presents them:
• $ godoc strings Join
func Join(a []string, sep
string) string
Join concatenates the
elements of a to create a
single string. The separator
string sep is placed between
elements in the resulting
string.
// Join concatenates the elements of a to create
a single string.
// The separator string sep is placed between
elements in the resulting string.
func Join(a []string, sep string) string {
//self explanatory blah blah blah
//more of the same
}

16.
godoc - examples and testing
func ExampleJoin() {
s := []string{"foo", "bar", "baz"}
fmt.Println(strings.Join(s, ", "))
// Output: foo, bar, baz
}
• Also integrated with the testing framework to provide testable
example functions.

17.
godoc - webview

18.
Concurrency should
be easy
And not yet forced.
Too much concurrency spoils the
code.

19.
Code should be hard
to mess with
A compiler that catches bugs
and is not too slow
statically compiled binaries - no
library mess

20.
Institutional knowledge
Go brings google’s institutional
knowledge to you, for free
- Groupcache
- Expvars
- Single-flights and Cancellations

21.
Object Caching
• In node.js - we built in an object caching solution in
house, that can easily make a function called cached
without changing much code.
• var wrapper = cache.wrap(original);
• Can chose whether to cache in memory or redis.
• Can avoid thundering herds - single flighting

22.
Enter group cache
• In memory
• distributed
• single flight
ELB
app
groupcache
app app
groupcachegroupcache
http http

23.
expvars - server metrics over http
• Export variables over http
• http://localhost:8123/debug/vars
{
"cmdline": ["/var/folders/bc/27hv15_d2zvcc3n3s9dxmfg00000gn/T/go-build421732654/command-
line-arguments/_obj/exe/main","-l","debug","-s","i.sock","-c","realtest"],
"counters": {"a": 10, "b": 10},
"memstats": {"Alloc":1076016,"TotalAlloc":1801544,"Sys":5966072,"Lookups":209,"Mallocs":
7986,"Frees":4528,"HeapAlloc":1076016,"HeapSys":2097152,"HeapIdle":327680,"HeapInuse":
1769472,"HeapReleased":0,"HeapObjects":3458,"StackInuse":212992,"StackSys":
917504,"MSpanInuse":21504,"MSpanSys":32768,"MCacheInuse":8800,"MCacheSys":
16384,"BuckHashSys":1441160,"GCSys":1183744,"OtherSys":277360,"NextGC":
1436032,"LastGC":1418102095002592201,"PauseTotalNs":2744531,"PauseNs":
[480149,171430,603839,288381,494934,522995,182803,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"NumGC":7,"EnableGC":true,"DebugGC":false}
}

24.
Cancellations
• User moves away, nginx sends 499, but queries keep
running.
• Not Abandoning work when user has moved away is
often wasteful
• Go’s has advisory cancellations - cancel across go-
routines
• facilitated by contexts

25.
moving from ‘dynamic’ to ‘compiled’
• Deployment changes
• from git clone to compile, build & deploy
• have a way of knowing binary version that is automatic
BUILDHASH=$(shell cd src/$(SRCDIR) && git rev-parse --verify HEAD | cut -c 1-7)
go build —ldflags -X $BUILDHASH
appname —version
• config and code is separate
• make sure everything you may ever need to configure is in config/flags, can’t change
binary
appname —port
appname —configtest
appname —debug
• have good debugging (we have debug() statements that are turned on by —debug flag)

26.
The go workflow
jenkins
dpkg-
buildpackagetriggers build
build dpkg using go get
ansible
srv1 srv1 srv1
elb
nginx
binary, run via upstart
deploy dpkg
restart
2
1
triggers deploy
3

27.
No templates, no exceptions, weird and sometimes inconsistent.
Go is an opinionated language
“Instructions, registers, and assembler
directives are always in UPPER CASE to
remind you that assembly programming is
a fraught endeavor.”
- Rob Pike

28.
-Dick Gabriel, from the golang FAQ
Who would have guessed sophistication
bought such noise?

29.
Golang - missing a package manager?
• go get - but its not complete
• No dependency management, unlike npm, no easy
hermetic builds
• gopkg.in is a solution - allows versioning
• if you have to change versions, edit every single import

30.
Deployment Challenges
• No Fork() - forking with subroutines ain’t easy
• Facebook grace (https://github.com/facebookgo/grace)
• PID changes - no relationship
• problems with upstart / job monitor
• log rotation

31.
Moving from a script to binary
• Deployment changes (from git clone to build, package and deploy)
• Need to know which version of code is running
• appname —version
• Need to debug easily
• appname —debug - enables debug() output
• Keep configuration flexible, because in case of issues, you can’t just edit and
deploy
• appname —configtest
• appname —port (over ride config)

32.
Go @ Tokopedia
• Go is a modern language, very suitable for web.
• Go is Performant, unless you are doing C.
• Go is mostly google still, and not 100% complete.
• Go is opinionated, like Plan9, and there will be a learning
curve.
• Go is slower for prototyping, e.g. when compared to
node.js

33.
Questions?