Technical presentation around the challenges and technical concerns, you need to plan when you try to make the jump of using containers in a production environment with Amazon Elastic Container Service (ECS)

1. Containers @ iZettle
A story of love, ruin & peril in AWS for the sake of containers
Alberto Lorente Leal
Software Engineer @ Horizon, iZettle

2. The journey to
production

3. Simplifying Deployments ● Shipping features is always key for your business
success
● Faster release cycles always good
● Deploy in test, production should be seamless

4. Resource
utilization
● Increase resource utilization
of shared instance
● Better cost management

5. Monitoring & Logging
You want to be ready when stuff breaks.

6. Pet vs Cattle
● How do you handle complexity as you
grow?
● Different servers, configurations..
○ Things do not change right?

7. Containers at scale

8. Amazon’s ECS
● Proprietary architecture
○ Partially open-sourced with blox
● Underlying Distributed Scheduler
ref: http://www.allthingsdistributed.com/2015/07/under-the-hood-of-the-amazon-ec2-container-service.html

9. The setup

10. Immutable Infrastructure
Everything stateless bottom-up
● Home baked AMIs with ECS agent
● Container per service
● Infrastructure as Code
○ Terraform
○ Version control of Infrastructure
○ Test and Prod. Infra are exact replicas

11. Monitor &
Logging
Partial usage of Influx’s data TICK metrics
stack
● Grafana visualizes influx metrics
● Telegraf for host metrics
● InfluxDB reporter at service level
● Splunk
○ Log forwarding
○ Log aggregation per service

12. InfluxDB
dropwizard
reporter

13. Immutable State & Containers

14. Cracks on the road

15. When things go sideways
Some AWS pain along the road...
● Security and Access control
● AWS SDK compatibility issues
○ Check you run the version you think
● Fixed ports
○ One container per instance…
● Autoscaling
○ Multiple alarms cancelling each other
○ Reservation metrics

16. When things go sideways
Debugging infrastructure issues...
● Patching
● Logs missing
○ Syslog driver
● Docker breaks
○ New upgrade version, things break
● Misconfigurations
○ Wrong values on wrong environment
○ Rebuild AMI take time..

17. Questions?
iZettle

18. Thank You!
iZettle