In this presentation, I will recount the challenges my team and I faced over the past year and how we overcame obstacles and achieved our goals. Even for greenfield projects, our journey had its share of surprises. Despite living in 2023, we still encountered common problems that many think are long-solved. I'll delve into topics such as defining DNS zones across multiple environments, dealing with ineffective build artifacts from mature development teams, GitHub Actions for CI/CD, versioning, cost optimization, and the potential pitfalls of adopting GitOps in combination with a "reuse as much as possible" mentality. So, join me as we explore the trade-offs and lessons learned from these real-world scenarios. With this presentation, you'll gain valuable insights to help you navigate similar challenges quickly and confidently.

2. Who Am I?
 Engineer<T> where T : Azure | DevOps | C#
 Microsoft® Most Valuable Professional
 Solutions Architect @ SoftServe, Inc.
 Spend time in the cloud
 More… go to https://ifesenko.com

3. Agenda  DNS
 Versioning
 GitOps & ArgoCD
 GitHub Actions for CI/CD

4. Context

5. DNS

6. It’s Always DNS

7. DNS. Lessons Learned
 Do NOT use {env}.{app}.company.com as a naming convention
 Do {app}.{env}.company.com
 Use {internal}.{env}.company.com DNS zone if you have resources
that are not exposed to public internet
 CNAME records will help during potential migration if you use them

8. Versioning

9. Versioning

10. Semantic Versioning (SemVer) & Branch Strategy
 Given a version number MAJOR.MINOR.PATCH, increment the:
 MAJOR version when you make incompatible API changes
 MINOR version when you add functionality in a backwards compatible manner
 PATCH version when you make backwards compatible bug fixes

11. Calendar-Based Versioning (CalVer)
 YYYY.MM.Sequence(.Patch)
 Breaking changes are for changelog
 Ship feature and fix as soon as possible
 Gregorian calendar and UTC time are only dependencies
 If you have support cycle each user can easily check if it is supported

12. Versioning. Lessons Learned
 Ensure calendar-based version is generated per a run and only once
 Always leave traces to correlate build number with commit id
 Git tag
 Version metadata
 You should be able to generate new version number on any platform

13. GitOps & ArgoCD

14. GitOps

15. GitOps. Repository Layout
 .github/
 azure/ - definitions related to policy as code
 cluster/ - all files related to AKS
 argocd/ - Application of Applications (cluster bootstrapping)
 non-prod/
 apps/
 projects/
 prod/
 apps/
 projects/
 rabbitmq/ - RabbitMQ Helm chart
 stunnel/ - stunnel Helm chart
 docs/ - runbooks, scripts, decision records, etc.
 terraform/ - Terraform files to manage infrastructure

16. ArgoCD. App Of Apps Pattern
 When ArgoCD is deployed
 create a new app “root”
 that consists other apps and projects
 PATH: cluster/argocd/non-prod
 Apps
 definitions of 1st party and 3rd party apps
 Projects
 we use ArgoCD projects as environments

17. projects/dev.yaml
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: dev
spec:
clusterResourceWhitelist:
- group: "*"
kind: "*"
description: Dev Environment
destinations:
- name: "*"
namespace: "*-dev"
server: "*"
sourceRepos:
- "*"
status: {}

18. apps/rabbitmq.yaml
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: rabbitmq
spec:
generators:
- list:
elements:
- env: dev
- env: sqa
- env: uat
template:
metadata:
name: "rabbitmq-{{env}}"
spec:
project: "{{env}}"
source:
repoURL: git@github.com:org/ops-product.git
targetRevision: HEAD
path: cluster/rabbitmq
helm:
valueFiles:
- common/values-common.yaml
- variants/non-prod/values-non-prod.yaml
- "envs/{{env}}-eastus/values-settings.yaml"
- "envs/{{env}}-eastus/values-replicas.yaml"
destination:
namespace: "rabbitmq-{{env}}"
server: https://kubernetes.default.svc
syncPolicy:
syncOptions:
- CreateNamespace=true

19. ArgoCD. Helm Chart Layout
 common/ - configuration which is common to all envs
 envs/ - holds environment specific configuration
 templates/ - Helm chart files
 variants/ - holds characteristics between similar envs

20. GitHub Actions for CI/CD

21. GitHub Actions or “Count to 10”

22. GitHub Actions Layout
 Build
 Code Analysis
 Deploy
 PRs
 *Reusable workflows to build,
test and publish

23. “CommitOps” in Action
skip-ci skip-tests

24. “EmojiOps” in Action
 Use as a symbol system to highlight important steps

25. GitHub Actions & Release Management

26. deploy.yaml
deploy:
runs-on: ubuntu-latest
name: deploy to ${{ inputs.env-id }}
environment:
name: ${{ inputs.env-id }}
env:
VERSION_FILE_PATH: cluster/product-services/envs/${{ inputs.env-id }}-eastus/values-version.yaml
steps:
- name: Checkout org/ops-product repository
uses: actions/checkout@v3
with:
repository: org/ops-product
path: ops-product
ssh-key: ${{ secrets.OPS_PRODUCT_SSH_PRIVATE_KEY }}
- name: Update container image version to ${{ inputs.build-version }}
uses: fjogeleit/yaml-update-action@main
with:
valueFile: ${{ env.VERSION_FILE_PATH }}
propertyPath: "image.tag"
value: "${{ inputs.build-version }}"
commitChange: false
updateFile: true
workDir: ops-product
branch: deployment
masterBranchName: main
targetBranch: dev
repository: org/ops-product
- name: Commit and push changes
run: |
git config user.name "${GITHUB_ACTOR}"
git config user.email "${GITHUB_ACTOR}@users.noreply.github.com"
git add .
git commit -m "Bump image version to ${{ inputs.build-version }} for ${{ inputs.env-id }} environment"
git push
working-directory: ops-product

27. Keep Up To Date Your Actions & Charts
 Dependabot or Renovate

28. GitHub Actions. Lessons Learned
 Be careful when sharing state between Jobs via GitHub Artifacts
 Do not publish build artifacts to GitHub Artifacts
 Allow a subsequently queued workflow run to interrupt previous
runs. GitHub Docs
 Limit number of open pull requests for version updates
 Cache dependencies
 Nested workflows and secrets scope

29. Questions?
@ky7m | ifesenko.com