devopsdays.istanbul 2020

1. Cloud Native Dünyada CI/CD
Mustafa AKIN SRE Architect @Atlassian, Opsgenie

2. Cloud Native Definition
“Cloud native technologies empower organizations to build and run scalable
applications in modern, dynamic environments such as public, private, and hybrid
clouds. Containers, service meshes, microservices, immutable infrastructure, and
declarative APIs exemplify this approach.
These techniques enable loosely coupled systems that are resilient, manageable, and
observable. Combined with robust automation, they allow engineers to make
high-impact changes frequently and predictably with minimal toil.”
https://github.com/cncf/toc/blob/master/DEFINITION.md

5. Containers Become Key Enablers for Many
Abilities some consider to be unnatural
Subsecond start times
Lightweight on system resources
Ease of packaging
Ability to easily share container images
Running same container in production
Isolation from system and clean throwaway environments

6. Tools Are Means For an End Goal
Containers Docker
Workload Scheduler Kubernetes
Service Meshes Istio, Linkerd
Immutable Infrastructure Terraform, Cloudformation, Pulumi
Monitoring Prometheus, Tracing

7. Cloud Native is a bit about Standardization
Containers Packaging is easy, ability to run in isolation, subsecond boot times
Workload Scheduler Declare your workload and deployments can be automated
Service Meshes Ease S2S comms, free metrics, policy enforcement, security
Immutable Infrastructure Don’t do manual changes that are not version tracked
Monitoring Export metrics and traces to find problem

8. What actually is Cloud Native?
Cloud enables modern, and dynamic environments.
You can get a fleet of server provisioned in a time you need to fill a cup of coffee.
You can create a managed database, object storage, block storage to scale virtually
unlimited.
You can manage workflows, tests, deployment, monitoring, by using automations, not
clicking buttons manually.
Modern workflows allows you reduce the Lead Time for Changes and Mean Time to
Recovery to minutes, compared to hours or even days for traditional companies.

9. What actually is Cloud Native?
If you do not properly architecture your applications, workflows to fit in the cloud, not
making use of any advancements other than creating virtual machines,
You are just deploying the same old software to Cloud-based
Virtual Machines using the same slow methods.
Cloud can also be on-premise*

10. Also Designs Should Adapt
Microservices Architecture
Asynchronous Data Flows
Isolated Failures
Microtransactions
Scalability
High Availability

11. Stages of CI & CD Adoption
1. Let’s test it when we deploy to production
2. A local test environment, then deploy to production if it’s safe
3. Write some unit tests to validate and run them before prod
4. Extend the test suite with Integration & External Tests
5. Disable merging to master without pull requests & tests passing
6. A remote test environment to deploy the application to run functional tests
7. Parallelize all the tests to reduce master merge queues
8. Slack notifications for who broke the tests
9. Reject Pull Request if the metrics go down
10. Automated Deployment
11. Rollback the Deployment based on alerts & degraded metrics
Try to convert repo to
Microservices & Embrace
DevOps in some of
the phases by distributing
the ownership

13. The Cycle for Changes
Develop Test Locally
Deploy to
Test
Environment
Run Tests
Automated
Merge to
Master
Deploy
5 min 3 min 5 min 5 min
18 minutes in Total
~26.6 per day (8h)

14. The Cycle for Changes
Develop Test Locally
Deploy to
Test
Environment
Run Tests
Automated
Merge to
Master
Deploy
10 min 25 min 40 min 30 min
105 minutes in Total
~4.5 per day (8h)

15. The Cycle for Changes
Develop Test Locally
Deploy to
Test
Environment
Run Tests
Automated
Merge to
Master
Deploy
10 min 25 min 40 min 30 min
105 minutes in Total
~4.5 per day (8h)
OPERATE

16. Rising Ops Need with Increased Velocity of Dev
How to run tests properly for each Pull Request?
How to deploy an application in a test environment?
How do you monitor an application to understand it’s not working as desired?
You want to divide your project to multiple services, how to properly orchestrate?
How to ensure you can merge to master in a distributed microservices environment?
How can 10+ people work on same repository?

17. Cycle: Develop
Need external software like Redis, Postgres? Deploy it as a container.
Docker-Compose can be helpful to run multiple containers and tear down easily.
Extract the metrics and traces the same way you would do in production to see impact.

18. Docker-Compose

19. Prometheus Metrics

20. Cycle: Test Locally
Use containers for single use, isolated, deterministic environments.
Ability to test for multiple versions by just changing the Docker container.
Deploy other microservices in containers without caring how to build/compile them.

21. Various JDK Versions as Docker Images

22. Various Microservices
https://github.com/GoogleCloudPlatform/microservices-demo

23. Localstack, Minio

24. Cycle: Deploy to Test Environment
Package your application as a container.
Make use of Kubernetes to schedule your container somewhere. Don’t care where it
runs. Use Declarative APIs to just describe your service.
Or use Google CloudRun, AWS Fargate for managed solutions.
Make use of Infrastructure as a Code, like Terraform & Cloudformation to ensure you
don’t do manual changes to your infrastructure.

25. Infrastructure as a Code
resource "aws_elb" "bar" {
name = "foobar-terraform-elb"
availability_zones = ["us-west-2a", "us-west-2b", "us-west-2c"]
access_logs {
bucket = "foo"
bucket_prefix = "bar"
interval = 60
}
listener {
instance_port = 8000
instance_protocol = "http"
lb_port = 80
lb_protocol = "http"
}
instances = ["${aws_instance.foo.id}"]
cross_zone_load_balancing = true
tags = {
Name = "foobar-terraform-elb"
}

26. A Kubernetes YAML

27. Knative YAML

28. Cycle: Run Automated Tests
Docker Images & Containers Provide Great Single Use Environments
Multiple Providers like Bitbucket Pipelines, Github, CircleCI, TravisCI supports Docker
Test for multiple versions in parallel, i.e run java:8, java:9
Running multiple instances of application become easy with containers

29. Jenkins

30. Bitbucket Pipelines Example

31. Tekton

32. Tekton

33. Cycle: Deploy
The tests should mark the Pull Request successful that blocks deployment otherwise.
The techniques for deploying to test environment should be similar in how you deploy
in production.
Kubernetes can be a good candidate. But what you probably need is an orchestrator.
Always think about your actual needs and consider your scale.
Ability to rollback is important for Mean Time To Recovery (MTTR)
Keeping track of what actually is deployed is also important

34. Various Deployment Types
Just Replace the Old Version
Blue-Green State for Safer Rollback
Canary Deployment with increasing traffic
A-B Testing for enabling certain deployment to certain users only

35. Spinnaker
https://devops.com/cd-foundation-touts-spinnaker-cd-progress/

36. GoCD

37. Bitbucket Deployment

38. There ain't no such thing as a free lunch
Every tool you introduce is a technical debt you need to maintain.
Can you afford 1-3 person DevOps Engineers? Person-months?
DevOps & Ownership has benefits, i.e frontend team knows Webpack better than you
How to keep balance between freedom/centralization?
Are all teams are capable of defining their own pipelines?
Continuous Delivery also implies owning after deployment, getting alerts, reverting
back.

39. Goal is to Squeeze the Cycle
Develop Test Locally
Deploy to
Test
Environment
Run Tests
Automated
Merge to
Master
Deploy
10 min 25 min 40 min 30 min
105 minutes in Total
~4.5 per day (8h)
OPERATE

40. Goal is to Squeeze the Cycle
Develop Test Locally
Deploy to
Test
Environmen
t
Run Tests
Automated
Merge to
Master
Deploy

41. Teşekkürler!
Mustafa Akın
@mustafaakin
www.mustafaakin.dev