5. • We have been building software for 30 years
• 95% of all software we use is built in-house
• Use mainly own datacenters
• First steps in the cloud
IT @ Sixt
6. • We have a lot of highly customizable workflows and
a lot of interdependencies
• We want to bring products and features to the market fast
• High service availability while constantly innovating
Our challenges
10. What is a service?
https://github.com/micro/go-micro
11. What is a service?
Making requests
To other services
Handlers
Middleware
…
Pub/Sub JSON-RPC
PROTO-RPC
GRPC
…
Consul
ETCD
Zookeeper
ECS
…
Client
load balancing
HTTP
Websockets
RabbitMQ
NATS
…
12. • We do not count lines of code - single responsibility is the most important rule for us
• We follow domain driven design (DDD) approach to identify responsibilities
• Each service may be a client and a server, a publisher and a subscriber at the same time
Size?
Cut?
Reservation
API
Reservation Branch Location
For example:
13. How does it all fit together?
Mobile API Web API
Domain Service Domain Service Domain Service
Aggregation layer
(client specific)
Business Domain
Specific
(generic / multitenant)
Clients
14. How does it all fit together?
eu-west-1a eu-west-1b eu-west-1c
API Gateway API Gateway API Gateway
A B C A C A B C
Event Stream Event Stream Event Stream
Traffic routing
Microservices (
in Go, Java…)
Kafka
Persistence Layer
B
16. • Captures any change in your system ( business events, app telemetry, logs…)
• Source for all reactive workflows ( think personalization )
• Feeds data to your machine learning models
• Powers up your near real-time analytics
• Pipes all information to your data warehouse and batch-like
workloads
The Event Stream
17. Fleet serviceRDMS
Elastic
search
Zoning
service
{
"lat": 53.5621,
"lng": 10.0652,
"car_id": ”edwqde-dw31w-dewqde-123”
}
{
"lat": 53.5621,
"lng": 10.0652,
"car_id": ”edwqde-dw31w-dewqde-123”,
"car_model": “BMW 320d”,
"car_color”: “white” …
}
{
"lat": 53.5621,
"lng": 10.0652,
"car_id": ”edwqde-dw31w-dewqde-123”,
"car_model": “BMW 320d”,
"car_color”: “white”,
”city" : “Hamburg”
}
Stream processing primer
1 A stream of telematics events
2
Enrich events with vehicle info
using a fleet service
3
Enrich events with location info
using a geofence service
18. Stream processing @ Sixt
Say hello to
• Open-sourced by LinkedIn
• Publish-subscribe messaging rethought as a distributed commit log
• Supports data replication and is scalable by design
• Can reply events from the past
• Supports event ordering
19. • Rely on event schema to ensure data quality
( no unknown or wrongly formatted events in the
Pipeline )
• Control topics to ensure contextual event
ordering & partitioning
Our stream processing architecture
21. Local cache for Foo and Bar
Async event enrichment
primer
Enriching
service
Service Foo
Topic A
Service Bar
Event A
Changes Foo
Changes Bar
Event B1 2
Topic BTopic Foo Topic Bar
Publish changes Publish changes
22. Sync vs Async
Sync
• Can be hard and more involved to implement
• Harder to debug and complexity
may steeply increase
• Keeps services independent
• Increased performance
( no additional network calls )
• Fast to implement and easy to understand
• Easy to debug and extend
• Introduces dependencies between services
• Latency becomes a problem
if upstream services are slow
Async
Implementation
Maintenance and Ops
Coupling
Performence
We actually use both depending on the use case and the effort required
24. The “source of truth” for many
personalized workflows
is now in the stream
Many small services that
do one thing onlyUse-case driven
persistence options
Each service should be able
to scale up or down based
on usage
Our version of the modern
enterprise stack
A modern self-service data warehouse
populated near real time
25. Learnings so far
• Building small services allows us to manage their lifecycle easier
• Tooling and automation is never enough
• Embracing open-source solutions and contributing back made a big difference
• Working more closely with startups and open-source tech helped us gain momentum