From System Engineer to Gopher

From system engineer
to GOPHER
I-Fan@XRSPACE
2021/07/28
coscup.ifan.tw

About ME
Back-end engineer @ XRSPACE,
Newbie to Golang and back-end
Android system software engineer @ XRSPACE,
Developed world first 5G VR HMD

Outline
• System Software Engineer vs Back-end Engineer
• Why Golang
• Clean Architecture with Golang
• Scratch an API server

System Software Engineer
vs
Back-end Engineer

ID
Android Device(Phone, HMD)
Hardware
Bootloader
Android Framework
Tools
Linux & Device Driver
HAL
Android Libs
Android Runtime
Android APP
From device bring-up to
software tool for device,
The scope of my work.
What’s system software engineer
(android)

A “Working” System
New request
Tracing lot of source code
Add little piece of codes
Customized System
DRM Playback.
Plain android system.
Tracing android/kernel + DRM module + base non-hlos
source code.
Enable/Modify related source code to integrate DRM
module into system.
A android system capable of DRM playback

What’s software engineer in my
imagination
Silicon Valley S01E05
A Mac with fancy tools that used to code at Starbucks

Discussing and Prototyping
New idea
Write some code
Test and validation
Developed a system
Back-end
Engineer
More time for discussing and prototyping.
More coding and I am building something
from scratch,
it’s delightful.

• One of the newest language
• Developer community
• Performance
• Statically typed
• Great for back-end development
• Effective Go
• Modern grammar
• Build-in test
• Go fmt
• Go vet
Why Golang

Clean Architecture with Golang
• Independent of Frameworks
• Testable
• Independent of UI
• Independent of Database
• Independent of any external agency

• The Dependency Rule
• Dependencies can only point inwards,
nothing in an inner circle can know
anything at all about something in an
outer circle.
• Dependency inversion principle

• Pros
• Same concept as DDD(Domain)
• Flexibility
• Testability
• Maintenance
• Cons
• Learning curve
• encapsulation, inheritance and
polymorphism everywhere!
• Heavy(compare to Monolithic)

Scratch an API server
User
API
SERVER
Database
AWS S3
File
Access URL
Save Meta
Save File
coscup.ifan.tw

Controller
Storage
Client
Domain
Application
REST
MongoDB
AWS S3
Repository
coscup.ifan.tw

Controller
Storage
Client
Domain
Application
REST
MongoDB
AWS S3
Repository

type AssetID string
type AssetType int
type Asset struct {
FileSize int64
Type AssetType
UploadedTime time.Time
TransferredTime time.Time
ID AssetID
Status AssetStatus
FileName string
Label string
Description string
hashedFileName string
}
Domain
- Define asset data structure
domain/asset.define.go

The order of fields in struct matters
type myStruct struct {
myInt bool // 1 byte
myFloat float64 // 8 bytes
myBool int32 // 4 bytes
// this takes 24 byte
}
type myStructOptimized struct {
myFloat float64 // 8 bytes
myInt int32 // 4 bytes
myBool bool // 1 byte
// this takes 16 byte
}
Memory
Allocation

type Repository interface {
Save(asset Asset) error
}
type StorageClient interface {
Save(ctx context.Context, fileName string)
(string, error)
}
Domain
- Define interfaces for external
layers to follow it
- Interfaces for repository
- Interface for storage
client

package application
package domain package repository
Dependency inversion principle? How?
type Repository {} struct
type Asset {} struct
package repository
type Repository struct {}
func (repo *Repository)Save(a Asset){}
package domain
type Asset struct {}
References
package application
asset := Asset{}
repo := Repository{}
repo.Save(asset)

package application
package domain package repository
Dependency inversion principle? How?
type Repository {} struct
type Asset {} struct
package repository
type Repository struct {}
func (repo *Repository)Save(a Asset){}
package domain
type Asset struct {}
type Repository interface{
Save(a Asset)
}
package application
asset := Asset{}
repo := domain.Repository
repo.Save(asset)
type Repository interface {
Save(a Asset)
}
References
Inherits

type Asset struct {
FileSize int64 `bson:"file_size,omitempty"`
Type int64 `bson:"type,omitempty"`
UploadedTime time.Time `bson:"uploaded_time,omitempty"`
ModifyTime time.Time `bson:"modify_time,omitempty"`
TransferredTime time.Time `bson:"transferred_time,omitempty"`
ID string `bson:"id,omitempty"`
Status string `bson:"status,omitempty"`
FileName string `bson:"file_name,omitempty"`
XRID string `bson:"xrid,omitempty"`
Label string `bson:"label,omitempty"`
Description string `bson:"description,omitempty"
HashedFileName string `bson:"hashed_file_name,omitempty"`
}
Repository
- Define data
structure to
save to
MongoDB
repository/mongodb/asset.go

type assetRepo struct {
collection *mongo.Collection
ctx context.Context
}
func (repo *assetRepo) Save(asset domain.Asset) error {
result := Asset{}
err := copier.Copy(&result, &asset)
if err != nil { … }
result.HashedFileName = asset.HashedFileName()
filter := bson.M{
"id": asset.ID,
}
update := bson.M{"$set": &result}
opts := options.Update().SetUpsert(true)
_, err = repo.collection.UpdateOne(repo.ctx, filter,
update, opts)
return err
}
Repository
repository/mongodb/asset.go
- Implement Save()

type S3StorageClient struct {
channel chan io.Reader
…
}
func (s S3StorageClient) Save(ctx context.Context, fileName string) (string,
error) {
…
file, ok := <-s.channel
if ok != true { … }
res, err := s.uploader.UploadWithContext(ctx, &s3manager.UploadInput{
Body: file,
Bucket: aws.String(s.bucket),
Key: aws.String(fileName),
})
if err != nil { … }
return res.Location, nil
}
Storage Client
storageclient/asset.go
- Implement Save()

type UploadFileCommand struct {
FileSize int64
FileName string
Label string
Description string
TransferredTime time.Time
UploadedTime time.Time
}
type UploadFileResult struct {
HashedFileName string
AssertURL string
AssetStatus string
ID string
}
Application
application/command.go
- Define upload command and response

type AppService struct {
repo domain.AssetRepository
storageClient domain.AssetStorageClient
}
func NewAssetAppService(repo domain.Repository,
storageClient domain.StorageClient)
*AppService {
return &AppService{
repo: repo,
storageClient: storageClient,
}
}
Application
application/application.srv.go
- Define service struct and constructor

func (assetApp *AppService) UploadFile(command UploadFileCommand) (*UploadFileResult, error) {
ts := time.Now()
asset, err := domain.NewAsset(domain.NewAssetParam{
ID: assetApp.repo.NewID(),
FileName: command.FileName,
FileSize: command.FileSize,
Status: domain.AssetStatusUploaded,
UploadedTime: command.UploadedTime,
Description: command.Description,
Label: command.Label,
})
if err != nil {…}
err = assetApp.repo.Save(*asset)
if err != nil {…}
urlAsset, err := assetApp.storageClient.Save(context.Background(), asset.HashedFileName())
if err != nil {…}
asset.Transferred(asset.UploadedTime.Add(time.Now().Sub(ts)))
err = assetApp.repo.Save(*asset)
if err != nil {…}
return &UploadFileResult{
HashedFileName: asset.HashedFileName(),
AssertURL: urlAsset,
ID: string(asset.ID),
AssetStatus: string(asset.Status),
}, nil
}
Application
application/application.srv.go
- Implement
function for
upload usecase

Controller
Storage
Client
Domain
Service
REST
MongoDB
AWS S3
Repository
coscup.ifan.tw

func (a *Asset) Upload(ctx *gin.Context) {
label := ctx.PostForm("label")
description := ctx.PostForm("description")
fileAsset, headerAsset, err := parseUploadFiles(ctx,
"asset")
if err != nil {}
ch := make(chan io.Reader, 1)
defer close(ch)
ch <- fileAsset
client, err := storageclient.NewStorageClient(config,
ch, time.Second*5)
if err != nil {}
repoAsset := a.sys.GetRepository().Asset()
appSrv := application.NewAssetAppService(repoAsset,
client)
cmd := application.UploadFileCommand{
FileName: headerAsset.Filename,
Label: label,
FileSize: headerAsset.Size,
Description: description,
}
result, err := appSrv.UploadFile(cmd)
if err != nil {}
ctx.JSON(200, gin.H{"success": result.AssertURL
,})
}
Controller
- Translate input request to
internal data format
- Wake application up for work

Save
Repository
Upload
Application
Save
Storage Client
POST /upload
Controller
Asset
Domain
Used by
Used by
Used by
Used by Injected interface{}
Independent test to
database/storage
Independent test to
repository/storage
Independent test to
application
Top layer can be used
every where

Demo
Source code:
coscup.ifan.tw

What I have learned
• The working model as is totally different from system
engineer.
• Never hesitate to refactor or even abandon your code, always
refine it.
• It’s important to learn it with fine architecture or design
pattern, it might to tough but it’s worth.

Thank you.
linkedin.com/in/ifan-wang
w07fan@gmail.com

From System Engineer to Gopher

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie From System Engineer to Gopher

Ähnlich wie From System Engineer to Gopher (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

From System Engineer to Gopher