11. How to search âGopherâ type
â grep can search only by text
â The text must be understood as Go source
code
â We need âStatic Analysisâ
11
13. Static Analysis & Dynamic Analysis
â Static Analysis
â Analyzing a program WITHOUT execution
â Analyzing structure of a program from source codes
â e.g. linter, code complement, code formatter
â Dynamic Analysis
â Analyzing a program WITH execution
â Investigating variables and result of functions
â e.g. race detector
13
14. Reflection
â Reflection
â Analyzing values and types at runtime.
â Reflection in Go
â reflect package
â It is only way to get struct tags at runtime
â encoding package uses reflection to
encode/decode JSONs, XMLs and so on.
14
15. Static Analysis Tools for Go
â There are many static analysis tools for Go
gofmt/goimports code formatter
go vet/golint code checker, linter
guru code comprehension tool
gocode code complement tool
errcheck error handlings checker
gorename/gomvpkg refactoring tools
15
16. Go for Static Analysis
â Go is easy to static analyze
â Static typing
â Simple syntax
â Type inference
â No Implicit type conversion
â go package is provided as a standard package
Static Analysis gives
a lot of information
16
17. Sub packages of go package
ast Package ast declares the types used to represent syntax trees for Go packages.
build Package build gathers information about Go packages.
constant Package constant implements Values representing untyped Go constants and their corresponding operations.
doc Package doc extracts source code documentation from a Go AST.
format Package format implements standard formatting of Go source.
importer Package importer provides access to export data importers.
parser Package parser implements a parser for Go source files.
printer Package printer implements printing of AST nodes.
scanner Package scanner implements a scanner for Go source text.
token
Package token defines constants representing the lexical tokens of the Go programming language and basic
operations on tokens (printing, predicates).
types Package types declares the data types and implements the algorithms for type-checking of Go packages.
17
19. Flow of Static Analysis
Source Code
Token
Abstract Syntax Tree
(AST)
Type Info
Parse
Tokenize
Type Check
go/scanner
go/token
go/parser
go/ast
go/types
go/constant
19
21. Parsing - go/parser,go/ast
â Converting tokens to abstract syntax tree
v + 1
IDENT ADD INT
Source Code:
+
v 1
BinaryExpr
Ident BasicLit
Tokens:
Abstract Syntax Tree:
(AST)
21
22. AST of âHello, Worldâ
package main
import "fmt"
func main() {
fmt.Println("Hello, äžç")
}
Run on Playground
*ast.File
[]ast.Decl
*ast.GenDecl *ast.FuncDecl
22
23. Type Check - go/types,go/constant
â Extract type infomation from AST
â Identifier Resolution
â Type Detection
â Constant Evalution
n := 100 + 200
m := n + 300
Constant Evalution
= 300
Type Detection
-> int
Identifier Resolution
23
24. Getting AST from source code
â Use parser.Parse* function
â ParseExpr,ParseExprFrom
â Parsing expression
â ParseExpr is simple version of ParseExprFrom
â ParseFile
â Parsing a file
â ParseDir
â Parsing files in a directory
â Calling ParseFile in the function
24
25. Getting AST from an expression
expr, err := parser.ParseExpr(`v + 1`)
if err != nil {
/* handling the error */
}
/* use expr */
â Parsing an expression
25
26. Getting AST from a file
const src = `
package main
var v = 100
func main() {
fmt.Println(v+1)
}`
fs := token.NewFileSet()
f, err := parser.ParseFile(fs, "my.go", src, 0)
if err != nil {
/* handling the error */
}
/* use f */
file name which is opened when
src is nil
Source Code
Parsing Mode
26
27. token.FileSet
â Recording positions on files
â The position (token.Pos) is represented by integer
â The value is unique among multiple files
â token.FileSet holds offset of each files
â The offsets are recorded at parsing
â token.FileSet is passed to parsing function as
an output parameter
type Pos int
27
28. Demo 1: Parse and Dump an AST
28
https://youtu.be/lM1Pj6xYxZs
29. Traversing AST - ast.Inspect
â Using ast.Inspect
expr, _ := parser.ParseExpr(`v + 1`)
ast.Inspect(expr, func(n ast.Node) bool {
if n != nil { fmt.Printf("%Tn", n) }
return true
})
Traverse the AST
*ast.BinaryExpr
*ast.Ident
*ast.BasicLit
Run on Playground
ast.Walk is more powerful and complex
+
v 1
BinaryExpr
Ident BasicLit
29
30. Traversing AST - Recursively
func traverse(n ast.Node) {
switch n := n.(type) {
case *ast.Ident:
fmt.Println(n.Name)
case *ast.BinaryExpr:
traverse(n.X)
traverse(n.Y)
case *ast.UnaryExpr:
traverse(n.X)
default:
fmt.Println(n)
}
}
print idetifyerâs name
traverse each terms recursively
switching by types
Run on Playground
traverse a term recursively
30
32. Type Checking
/* initialize configs for type checking */
cfg := &types.Config{Importer: importer.Default()}
info := &types.Info{
/* TODO: initialize maps which will hold results */
}
pkg,err := cfg.Check("main", fs, []*ast.File{f}, info)
if err != nil {
/* handling the error */
}
/* TODO: use pkg or info */
â (*types.Config).Check do a type checking
32
33. types.Info holds results of type checking
type Info struct {
// Types maps expressions to their types, and for constant
// expressions, also their values.
Types map[ast.Expr]TypeAndValue
// Defs maps identifiers to the objects they define.
Defs map[*ast.Ident]Object
// Uses maps identifiers to the objects they denote.
Uses map[*ast.Ident]Object
// Implicits maps nodes to their implicitly declared objects, if any.
Implicits map[ast.Node]Object
// Selections maps selector expressions (excluding qualified identifiers)
// to their corresponding selections.
Selections map[*ast.SelectorExpr]*Selection
// Scopes maps ast.Nodes to the scopes they define.
Scopes map[ast.Node]*Scope
// InitOrder is the list of package-level initializers in the order in which
// they must be executed.
InitOrder []*Initializer
}
33
37. Tests for requests and responses
â Requests (Responses) can be test by
specifying fields and expected values,
document for each fields
validator.RequestBody(t, []httpdoc.TestCase{
{"Name", "tenntenn", "User Name"},
{"Attribute.Email", "tenntenn@example.com", "e-mail address"},
}, &createUserRequest{})
37
38. github.com/tenntenn/gpath
â Simplefy reflection of struct fields
â Reflecting by expression of selectors
â Easy to reflect complex struct value
â Supports maps and slices (but restricted)
type Bar struct { N []int }
type Foo struct { Bar *Bar }
f := &Foo{ Bar: &Bar{ N: []int{100} }}
v, _ := At(f, `Bar.N[0]`)
fmt.Println(v)
$ go run main.go
100
38
39. Create own static analysis tools
â Reduce costs of code reviews by own tools
â Custimized linter for your project
â Detecting typical bugs with own tools
â You should pay your time for more important things
â algorithm, design, performance and so on
39
41. Banner Tool
â A management tool of in-app banners
In-App Banner
A screenshot of our app (Mercari Atte)
41
42. Banner Tool
â A management tool of in-app banners
Banner Tool
â Delivery Conditions
â Response
Getting Banners
w/ OS, API Version
List of Delivered Banners
w/ Image URL, Destination URL, etc...
Apps
42
43. Evaluation of Delivery Conditions
Getting Banners
GET /banner/?os=1
String(os) == "1"
Banner Image, etc...
Delivery Condition:
Banner Tool
"1" == "1"
true
BIND
EVAL
â Describing delivery conditions by a Go like expression
â Eval expressions with go package
â Using a function calling for describing a variable with a type
43
44. Conclutions
â Static Analysis is EASY in Go
â static typing, simple syntax, go package, etc...
â go package can
â tokenize, parse and type check
â Static analysis in production
â own static analysis tools (e.g. gpath, go-httpdoc)
â web apps (e.g. Banner Tool)
44