Presented at Singapore Gophers Meetup - 20 Jan 2015 Working with the Go http package: - Customising handlers - Writing middleware - Ecosystem Source: https://github.com/jonog/customising-go-web Slides: http://go-talks.appspot.com/github.com/jonog/customising-go-web/customising-go-web.slide

1. Customising Your Own Web
Framework in Go
20 January 2015
Jonathan Gomez
Engineer, Zumata

2. This Talk
Overview
- Intro to serving requests with http/net
- Customising Handlers
- Writing Middleware
- Ecosystem
Key takeaways
- Compared with Ruby/Node.js, mainly using the standard library is considered normal
- Interfaces and first-class functions make it easy to extend functionality
- Ecosystem of libraries that work alongside http/netis growing

3. Intro to Serving Requests with http/net

4. Serving Requests via Standard Lib (1/4)
packagemain
import"net/http"
funchandlerFn(whttp.ResponseWriter,r*http.Request){
w.Write([]byte(`Helloworld!`))
}
funcmain(){
http.HandleFunc("/",handlerFn)
http.ListenAndServe("localhost:4000",nil)
}
ListenAndServe - creates server that will listen for requests
Each request spawns a go routine: goc.serve()

5. Serving Requests via Standard Lib (2/4)
ServeMux matches incoming request against a list of patterns (method/host/url)
ServeMux is a special kind of Handlerwhich calls another Handler
Handler interface
typeHandlerinterface{
ServeHTTP(ResponseWriter,*Request)
}

6. Serving Requests via Standard Lib (3/4)
Request handling logic in ordinary function func(ResponseWriter,*Request)
funcfinal(whttp.ResponseWriter,r*http.Request){
w.Write([]byte("OK"))
}
Register the function as a Handler on DefaultServeMux
http.Handle("/",http.HandlerFunc(final))
Also can:
http.HandleFunc("/",final)

7. Serving Requests via Standard Lib (4/4)
func(ResponseWriter,*Request)
ResponseWriter interface
typeResponseWriterinterface{
Header()Header
Write([]byte)(int,error)
WriteHeader(int)
}
Request struct
typeRequeststruct{
Methodstring
URL*url.URL
HeaderHeader
Bodyio.ReadCloser
ContentLengthint64
Hoststring
RemoteAddrstring
...
}

8. Customising Handlers

9. Demo: Customising Handlers - DRY Response Handling (1/3)
typeappHandlerstruct{
hfunc(http.ResponseWriter,*http.Request)(error)
}
func(ahappHandler)ServeHTTP(whttp.ResponseWriter,r*http.Request){
err:=ah.h(w,r)
iferr!=nil{
switcherr:=err.(type){
caseErrorDetails:
ErrorJSON(w,err)
default:
ErrorJSON(w,ErrorDetails{"InternalServerError","",500})
}
}
}
In app code we might extend this further:
Add error types and respond differently.
e.g. warn vs error-level log, send alerts, increment error metrics

10. Demo: Customising Handlers - DRY Response Handling (2/3)
typeErrorDetailsstruct{
Messagestring`json:"error"`
Detailsstring`json:"details,omitempty"`
Statusint`json:"-"`
}
func(eErrorDetails)Error()string{
returnfmt.Sprintf("Error:%s,Details:%s",e.Message,e.Details)
}
funcErrorJSON(whttp.ResponseWriter,detailsErrorDetails){
jsonB,err:=json.Marshal(details)
iferr!=nil{
http.Error(w,err.Error(),500)
return
}
w.Header().Set("Content-Type","application/json")
w.WriteHeader(details.Status)
w.Write(jsonB)
}

11. Demo: Customising Handlers - DRY Response Handling (3/3)
Use of special struct and special handler function to satisfy Handlerinterface
http.Handle("/",appHandler{unstableEndpoint})
Reduce repetition, extend functionality.
funcunstableEndpoint(whttp.ResponseWriter,r*http.Request)(error){
ifrand.Intn(100)>60{
returnErrorDetails{"Strangerequest","Pleasetryagain.",422}
}
ifrand.Intn(100)>80{
returnErrorDetails{"Seriousfailure","Weareinvestigating.",500}
}
w.Write([]byte(`{"ok":true}`))
returnnil
} Run

12. Demo: Customising Handlers - Avoiding Globals
Allows injecting dependencies rather than relying on global variables.
typeApistruct{
importantThingstring
//db*gorp.DbMap
//redis*redis.Pool
//logger...
}
typeappHandlerstruct{
*Api
hfunc(*Api,http.ResponseWriter,*http.Request)
}
func(ahappHandler)ServeHTTP(whttp.ResponseWriter,r*http.Request){
ah.h(ah.Api,w,r)
}
funcmyHandler(a*Api,whttp.ResponseWriter,r*http.Request){
w.Write([]byte("2015:Yearofthe"+a.importantThing))
} Run

13. Writing Middleware

14. Middleware: Why?
Abstract common functionality across a set of handlers
Bare minimum in Go:
func(nexthttp.Handler)http.Handler
Typical uses of middleware across languages/frameworks:
- logging
- authentication
- handling panic / exceptions
- gzipping
- request parsing

15. Demo: Middleware Example (Panic Recovery)
funcrecoveryMiddleware(nexthttp.Handler)http.Handler{
returnhttp.HandlerFunc(func(whttp.ResponseWriter,r*http.Request){
deferfunc(){
iferr:=recover();err!=nil{
log.Println("Recoverfromerror:",err)
http.Error(w,http.StatusText(500),500)
}
}()
log.Println("ExecutingrecoveryMiddleware")
next.ServeHTTP(w,r)
})
}
funcfinal(whttp.ResponseWriter,r*http.Request){
log.Println("ExecutingfinalHandler")
panic("walau!")
w.Write([]byte("OK"))
} Run

16. Demo: Chaining Middleware
funcmiddlewareOne(nexthttp.Handler)http.Handler{
returnhttp.HandlerFunc(func(whttp.ResponseWriter,r*http.Request){
log.Println("->ExecutingmiddlewareOne")
next.ServeHTTP(w,r)
log.Println("->ExecutingmiddlewareOneagain")
})
}
Calling chain of middleware
http.Handle("/",middlewareOne(middlewareTwo(http.HandlerFunc(final))))
funcmiddlewareTwo(nexthttp.Handler)http.Handler{
returnhttp.HandlerFunc(func(whttp.ResponseWriter,r*http.Request){
log.Println("--->ExecutingmiddlewareTwo")
next.ServeHTTP(w,r)
log.Println("--->ExecutingmiddlewareTwoagain")
})
} Run

17. Chaining Middleware - Alternate Syntax
3rd Party Library: Alice
Manages middleware with the standard function signature
Nice syntax for setting up chains used in different endpoints
chain:=alice.New(middlewareOne,middlewareTwo)
http.Handle("/",chain.Then(finalHandler))
Our example
noAuthChain:=alice.New(contextMiddleware,loggerMiddleware)
authChain:=alice.New(contextMiddleware,loggerMiddleware,apiKeyAuthMiddleware)
adminChain:=alice.New(contextMiddleware,loggerMiddleware,adminAuthMiddleware)

18. Demo: Creating Configurable Middleware
e.g. Pass the dependency on *AppLogger
varlogger*AppLogger=NewLogger()
loggerMiddleware:=simpleLoggerMiddlewareWrapper(logger)
http.Handle("/",loggerMiddleware(http.HandlerFunc(final)))
funcsimpleLoggerMiddlewareWrapper(logger*AppLogger)func(http.Handler)http.Handler{
returnfunc(nexthttp.Handler)http.Handler{
returnhttp.HandlerFunc(func(whttp.ResponseWriter,r*http.Request){
startTime:=time.Now()
next.ServeHTTP(w,r)
endTime:=time.Since(startTime)
logger.Info(r.Method+""+r.URL.String()+""+endTime.String())
})
}
} Run

19. Demo: Customising ResponseWriter (1/3)
typeResponseWriterinterface{
Header()http.Header
Write([]byte)(int,error)
WriteHeader(int)
}
ResponseWriter as an interface allows us to extend functionality easily
Example:
Step 1: Create a struct that wraps ResponseWriter
typeresponseWriterLoggerstruct{
w http.ResponseWriter
datastruct{
statusint
size int
}
}
Record data that would be otherwise be untracked.

20. Demo: Customising ResponseWriter (2/3)
Step 2: Define methods required for implicit satisfaction
func(l*responseWriterLogger)Header()http.Header{
returnl.w.Header()
}
func(l*responseWriterLogger)Write(b[]byte)(int,error){
//scenariowhereWriteHeaderhasnotbeencalled
ifl.data.status==0{
l.data.status=http.StatusOK
}
size,err:=l.w.Write(b)
l.data.size+=size
returnsize,err
}
func(l*responseWriterLogger)WriteHeader(codeint){
l.w.WriteHeader(code)
l.data.status=code
}

21. Demo: Customising ResponseWriter (3/3)
funcspecialLoggerMiddlewareWrapper(logger*AppLogger)func(http.Handler)http.Handler{
returnfunc(nexthttp.Handler)http.Handler{
returnhttp.HandlerFunc(func(whttp.ResponseWriter,r*http.Request){
startTime:=time.Now()
w2:=&responseWriterLogger{w:w}
next.ServeHTTP(w2,r)
logger.Info(r.Method+""+r.URL.String()+""+
time.Since(startTime).String()+
"status:"+strconv.Itoa(w2.data.status)+
"size:"+strconv.Itoa(w2.data.size))
})
}
} Run

22. Growing Middleware Ecosystem
Excerpt from Negroni Github page
graceful:(https://github.com/stretchr/graceful)graceful HTTP Shutdown
oauth2:(https://github.com/goincremental/negroni-oauth2)oAuth2 middleware
binding:(https://github.com/mholt/binding)data binding from HTTP requests into structs
xrequestid:(https://github.com/pilu/xrequestid)Assign a random X-Request-Id: header to each request
gorelic:(https://github.com/jingweno/negroni-gorelic)New Relic agent for Go runtime
Mailgun's Oxy
stream:(http://godoc.org/github.com/mailgun/oxy/stream)retries and buffers requests and responses
connlimit:(http://godoc.org/github.com/mailgun/oxy/connlimit)Simultaneous connections limiter
ratelimit:(http://godoc.org/github.com/mailgun/oxy/ratelimit)Rate limiter

23. Other Web Framework Components
Routing & Extracting URL Params
- standard library can be inflexible
- regex for extracting url params can feel too low level
- plenty of third party routers, e.g. Gorilla mux
funcShowWidget(whttp.ResponseWriter,r*http.Request){
vars:=mux.Vars(r)
teamIdStr:=vars["team_id"]
widgetIdStr:=vars["widget_id"]
...
}
Request-specific context
- sharing data between items in middleware chain and final handler
- solutions involve either global map, or per-request map/structs using custom
handlers/middleware

24. Web frameworks vs Build on top of standard library?
Time/expertise to build what you need? Too much re-inventing?
Your optimisation vs framework optimisation?
Performance? Does performance order of magnitude matter?
How much magic do you want?
Compatibility with net/http/ ecosystem? Framework interchangeability?
Martini -- 6.1k(https://github.com/go-martini/martini)
Revel -- 4.7k(https://github.com/revel/revel)
beego -- 3.7k(https://github.com/astaxie/beego)
goji -- 1.9k(https://github.com/zenazn/goji)
gin -- 1.9k(https://github.com/gin-gonic/gin)
negroni -- 1.8k(https://github.com/codegangsta/negroni)
go-json-rest -- 1.1k(https://github.com/ant0ine/go-json-rest)
Gorilla/mux -- 1.1k(https://github.com/gorilla/mux)
Tiger Tonic -- 0.8k(https://github.com/rcrowley/go-tigertonic)
Gocraft/web -- 0.6k(https://github.com/gocraft/web)

25. Thank you
Jonathan Gomez
Engineer, Zumata
jonathanbgomez@gmail.com(mailto:jonathanbgomez@gmail.com)
@jonog(http://twitter.com/jonog)