What we've built at Zalando is complex. Supporting – profitably – a publicly traded e-commerce company that does business in 15 diverse European markets, with more than 15 million active users who all speak different languages, use different payment methods, prefer different shipping methods, and have different product tastes, has required nonstop innovation. Until recently we've focused on building a unified, comprehensive retail system, quickly, that solves just our problems. But to truly fight against complexity – particularly the accidental complexity that slows down our development process – we have adopted a microservices architecture. And when it comes to DevOps, we’ve gone a step beyond the "You build it, you run it" motto--working in autonomous teams with DevOps treated as a "first-class entity.” In this talk, Henning Jacobs (Software Architect) and Jan Löffler (Head of Platform Engineering) will share their experience implementing “Radical Agility” from a DevOps perspective. “Radical Agility” is the Zalando technology team’s multi-pronged approach to managing the complexity that results from building an architecture of massive size. Henning and Jan will focus on how microservices enable Zalando’s engineers to move faster and build systems that scale, at scale, and avoid dependencies. They will show how microservices, in conjunction with a cloud infrastructure, support teams as they try strive for autonomy. Finally, they will draw upon their experiences to show how this all works in practice, and discuss what is organizationally and architecturally necessary to make DevOps a top priority for all members of your tech organization.

1. Radical Agility
with Autonomous Teams and Microservices in the Cloud
DevOps Con 2015 - Berlin - Germany

2. ARCHITECTURE
LEADERSHIP
ABOUT US
RADICAL AGILITY
INFRASTRUCTURE
AGENDA

3. Jan Löffler
● Head of Platform Engineering
● Twitter: @jlsoft2
● jan.loeffler@zalando.de
ABOUT US

4. Henning Jacobs
● STUPS Hacker
● Twitter: @try_except_
● henning.jacobs@zalando.de
ABOUT US

5. 15 countries
3 fulfillment centers
15+ million active customers
2.2+ billion € revenue 2014
130+ million visits per month
8.000+ employees
ONE of EUROPE’S LARGEST ONLINE FASHION RETAILERS
Visit us: tech.zalando.com

9. ENVIRONMENT

10. DATACENTER ENVIRONMENT
DataCenter I
Gütersloh, Germany
DataCenter II
Berlin, Germany
DataCenter III
Berlin, Germany
Global Traffic Management
AWS
Dublin, Ireland

11. THE LOST
HIGHWAY

12. CLOUD PROJECTS
2013/14 2014
Pequod
2013
Noah’s ARKzCloud

13. TOPIC 1
WHERE
TO GO
THIS NEEDS TO STOP
Doing it yourself is not the
most sensible thing.
Amazon invested already
thousands of engineering
hours… we must utilize this.
(Eric Bowman)

14. RADICAL
AGILITY

15. GOAL
DELIVER AMAZING
PRODUCTS
EFFICIENTLY AT
SCALE, AND
FEELING GREAT
ABOUT IT.

16. 3 PRINCIPLES

17. PURPOSE

18. AUTONOMY

19. MASTERY

20. LEADERSHIP
FROM
CONTROL &
COMMAND
TO PURPOSE
AND TRUST

21. NEW LEADERSHIP
DELIVERY
LEAD
PEOPLE
LEAD

22. GLOBAL
REGRESSION
INNOVATION LAB
TECH
SERVICE
PRODUCT
OWNER
PRODUCT
SPECIALIST
DELIVERY
LEAD
PEOPLE
LEAD
BUSINESS
ASSURANCE
PRODUCT
OVERARCHING
ADMIN & SUPPORT
CONTROLLING
EXECUTIVE
SUPPORT
COMPLIANCE
RISK ,
SECURITY &
STRATEGY
ONBOARDING &
TECHADEMY
AGILE
COACHING PROJECT
MANGEMENT
ENGINEERING
PRODUCTIVITY
DELIVERY
LEAD
PEOPLE LEAD
DELIVERY

23. STEERING
ALIGNING
TARGETS VIA
OKR: WORK
TOWARDS
COMPANY
PURPOSE

24. ARCHITECTURE
AN
ARCHITECTURE
FOR
INNOVATION

25. API FIRST

26. REST

27. SAAS

28. MICRO
SERVICES

29. CLOUD

30. INFRASTRUCTURE
ROBUST
PLATFORM
SERVICES TO
ENABLE
AUTONOMY

31. STUPS
STUPS To Unleash Penguin Swarms

32. AWS
STUPS
DOCKER
DEPLOY
SSH
ACCESS
AUDIT
REPORTS
FULL AWS
ACCESS
A PLATFORM ON TOP OF AMAZON WEB SERVICES

33. AUTONOMY AND COMPLIANCE
STUPS offers
maximum freedom for developers
while enabling
near-real-time audit compliance
for every single application.

34. One AWS account per Team
Deployment with Docker
Managed SSH Access
REST/OAuth 2.0 mandatory
Supports Traceability of Changes
STUPS IN A NUTSHELL

35. Public Internet
*.foo.example.org *.bar.example.org
Team “Foo” Team “Bar”ELB ELB
EC2
Instance
EC2
InstanceEC2
InstanceEC2
Instance
EC2
InstanceEC2
Instance
Data Center LB
EC2
InstanceEC2
InstanceLegacy
Instances
ISOLATED AWS ACCOUNTS

37. IMMUTABLE STACKS

38. AWS
DEPLOYMENT WITH SENZA
Senza CLI
Pier One
docker pull
docker push
Taupage

39. SENZA: DEFINITION YAML

40. SENZA: BOOTSTRAP NEW CLOUD FORMATION STACK

41. SENZA: MANAGE STACKS

43. APPLICATION LOGS: TAUPAGE SUPPORTS LOGENTRIES AND SCALYR

45. SSH ACCESS: TIME-LIMITED ACCESS TO ANY TEAM SERVER

47. TODO: Screenshot
ZMON

48. ZMON APPLIANCE
*.foo.example.org *.bar.example.org
Team “Foo” Team “Bar”
EC2
Instance
EC2
InstanceEC2
Instance
EC2
Instance
ZMON
Appliance
ZMON
Appliance
KairosDB
EC2
Instance
EC2
Instance
ZMON
Controller
ELB ELB

49. HYSTRIX TURBINE

51. OAUTH: APPLICATION REGISTRATION IN YOUR TURN

52. OAUTH: CREDENTIAL DISTRIBUTION VIA S3 BUCKETS
AWS
YOUR TURN
get access
token
Taupage
Kio Mint
OAuth
Provider
store
passwords
get password
S3
rotate
passwords

53. STUPS Frontpage
http://stups.io
STUPS Documentation
http://docs.stups.io
GitHub Repositories
https://github.com/zalando-stups
Trying out Senza and Taupage
http://docs.stups.io/en/latest/user-guide/standalone-deployment.html
LINKS

54. QUESTIONS?

55. BACKUP

56. ● ELB for
inbound traffic
● NAT instances
for outbound
● HTTPS Only
● Internal subnets
for app instances
DMZ DMZ DMZ
internalinternal
eu-west-1a eu-west-1b eu-west-1c
ELB
EC2
internal
EC2
NAT
STUPS: AWS ACCOUNT VPC SETUP

57. Pier One Docker Reg.
build
approve
EC2 Instances
Docker
Container
Application “myapp”
issue_management: Jira
Application Version “1.0”
artifact: docker/myart:1.0
Taupage AMI
Ticket System
Kio Application Registry
Ticket System
SCM
Image “docker/myart:1.0”
commit: afb123Issue “ABC-123”
spec: [...]
Commit “afb123”
msg: ABC-123..
✓ specs approved
✓ artifact tested
✓ artifact approved
STUPS: TRACEABILITY

58. ZALANDO TECH CONSTITUTION PT. 1

59. ZALANDO TECH CONSTITUTION PT. 2

60. ENGINEER
DELIVERS CUTTING EDGE SOFTWARE PRODUCTS END TO
END
MAINTAINS AND OPERATES THE SOFTWARE PRODUCTS
IS ACCOUNTABLE FOR HIGH QUALITY OF SOFTWARE
PRODUCTS AND THEIR SPECIFICATION

61. PRODUCER
ACCOUNTABLE FOR NON-CORE ENGINEERING TASKS
NECESSARY FOR SUCCESSFUL E2E DELIVERY AND
OPERATION
ORGANIZES TEAM EXTERNAL PROJECT DEPENDENCIES,
EXTERNAL DELIVERABLES AND ROADMAPS
TAKES CARE AND GETS RID OF IMPEDIMENTS

62. DELIVERY LEAD
DELIVER
TRUST BASED LEADERSHIP
UNDERSTAND THE BUSINESS
DRIVE PURPOSE
SUPPORT AUTONOMY

63. DELIVERY LEAD
DEVELOPS VISION AND ENSURES HIGH QUALITY
CUTTING EDGE PRODUCT DELIVERY
BUILDS POWERFUL TEAMS
ENABLES TEAMS AND CHALLENGES DECISIONS
OVERSEES ALL TECHNICAL TOPICS

64. PEOPLE LEAD
ENABLES PEOPLE TO CONTINUOUSLY GROW AND
DEVELOP THEIR CAREER THROUGH ROTATIONAL TOUR
OF MASTERY
ALIGNS COMPANY AND TEAM PURPOSE WITH PEOPLE’S
DRIVERS
CONTRIBUTES TO ALL PEOPLE RELATED TOOLS AND
PROCESSES AND ENSURES THAT THEY ARE STATE OF
THE ART

65. PEOPLE LEAD
COACH
SUPPORT
SUPPORT AUTONOMY
DRIVER OF MASTERY
TOURS OF MASTERY
LOTS OF DIRECTS

66. What we've built at Zalando is complex. Supporting – profitably – a publicly traded e-commerce company that does
business in 15 diverse European markets, with more than 15 million active users who all speak different languages, use
different payment methods, prefer different shipping methods, and have different product tastes, has required nonstop
innovation. Until recently we've focused on building a unified, comprehensive retail system, quickly, that solves just our
problems. But to truly fight against complexity – particularly the accidental complexity that slows down our development
process – we have adopted a microservices architecture. And when it comes to DevOps, we’ve gone a step beyond the
"You build it, you run it" motto--working in autonomous teams with DevOps treated as a "first-class entity.”
In this talk, Henning Jacobs (Software Architect) and Jan Löffler (Head of Platform Engineering) will share their
experience implementing “Radical Agility” from a DevOps perspective. “Radical Agility” is the Zalando technology
team’s multi-pronged approach to managing the complexity that results from building an architecture of massive size.
Henning and Jan will focus on how microservices enable Zalando’s engineers to move faster and build systems that
scale, at scale, and avoid dependencies. They will show how microservices, in conjunction with a cloud infrastructure,
support teams as they try strive for autonomy. Finally, they will draw upon their experiences to show how this all works
in practice, and discuss what is organizationally and architecturally necessary to make DevOps a top priority for all
members of your tech organization.
ABSTRACT