This document discusses optimizing and profiling Golang REST APIs. It explains that profiling measures program performance to aid optimization. The steps are to deploy an application, conduct profiling to identify slow code, analyze profiling data using tools like pprof, decide on solutions like using goroutines or caching, test the impact of changes, and repeat until performance goals are met before deployment. Short profiling demos are provided and load testing results show performance improvements from optimization solutions.

1. Optimizing
and Profiling
Golang
REST API

2. Overview
- What is Software Optimization? And why we need it?
- What is System Profiling?
- Profiling in Golang
- Steps to do Profiling in Golang API
- And Optimize it!

3. Software Optimization

4. Optimization is the action of making the best
or most effective use of a situation or
resource.

7. Why We Need to Optimize Our System?
- Know what our system’s needs
- Reduce any unused resources and process
- Increase our API Performance

8. Steps to Optimize (our) REST API
- #1 Have a real application that ready to deployed to production
- #2 Do profiling
- #3 Get data from profiling
- #4 Decision
- #5 Repeat to #2 until confident to deploy
- #6 Deploy

9. #1 Application: Study case - a simple REST API
- Sample application can be found at: https://github.com/bxcodec/aqua

10. #2 Do Profiling
Profiling: is a form of dynamic program analysis that measures, the space
(memory) or time complexity of a program, the usage of particular instructions, or
the frequency and duration of function calls. Most commonly, profiling information
serves to aid program optimization. ~ Source: ipfs.io

11. #2 Do Profiling… (1): Why we need profiling?
- Know what is our application needs from Memory and CPU
- Know how long is processing time and response time of our API
- Know exactly which code should be improved
- More confident before releasing into Production

12. Scope: Golang REST API to detect Slow Function a.k.a CPU Profiling
Steps
- Import pprof (https://golang.org/pkg/net/http/pprof/) in our projects
- Run and do load testing
- Load testing tools
- AB (Apache Benchmark)
- Hey (https://github.com/rakyll/hey)
- Anything that can do many request at once.
- Run the pprof tool
#2 Do Profiling… (2): Profiling in Golang API

14. Short Demo

15. #3 Get data from profiling
- Detect the slow code with pprof
- Get a graphical data of all the process
- Using pprof
- Using 3rd party-app, current case I use: speedscope.app

16. #3 Get data from profiling … (2)
- Load Testing Result
- https://gist.github.com/bxcodec/ecef947c98c8c44a2be46d3137a37555
- pprof results

17. pprof
- Invented by Google for profiling
- List available command can be seen in
- Most common used
- top N [sort by]
- e.g: top 10 -cum
- list [function keyword]
- e.g: list FetchArticles
$ go tool pprof -help

18. pprof … (2)
- Something good to know about pprof syntax
- cum: total of time needed to execute the whole function
- flat: total of time needed to execute the function excluding external function
- sum: is just a coverage percentage from each line

19. pprof … (3)
cum for callMe(): 5s
flat for callMe(): 2s

20. Flame Graph Rendering
- You can use any Flame Graph tools
- I used speedscope.app

22. #4 Decision
- We have data from Load Testing
- We have a profiling result
- Is it okay? Is it acceptable?
- If not, what’s is the solutions?

23. Any Idea?

24. First Solution
- Using goroutine

25. First Solution … (1)
- Load Testing Result
- https://gist.github.com/bxcodec/48094754383a31b8d835ec32036d461e
- pprof

26. Second Solution
- Using fetch by IDs for the Authors

27. Second Solution … (1)
- Load Testing Results
- https://gist.github.com/bxcodec/ced0e5c28131c977ed02a2c02ae60429
- The result pretty impressive: could handle up to 170K request in 5 minutes

28. Takeaways
- Use pprof to know which code should be improved
- Do a Load testing before and after optimized
- Optimize after profiling a.k.a avoid premature optimizing
- E.g: directly using cache
- Looks for the problem, and choose the right solution.
- E.g: for study case previously,
using goroutine might not be the best solutions
- For the better experience, do load testing and
profiling between servers

30. Discussion Time!

31. Thank You
Thanks to:
- https://gopherize.me for the cool Gopher Avatar
- filldb.info for the dummy data generator
- Xendit and Golang Jakarta for the chance
Find me on:
- https://github.com/bxcodec
- https://medium.com/@imantumorang
- Xendit -- We’re hiring ^_^