Lachlan presented these slides for the ongoing Weave Online User Group series. You can view the recording here: https://youtu.be/_YZRbS5GaYE Adjusting your spyglass and looking out over the water, you can see how useful a package manager like Helm is. Perhaps you’ve used it to manage the fractal complexity of packages on your Kubernetes clusters (without losing track of versions stashed in the hold). But Helm 3 is rumored to be different, and you’re ready to get started on this exciting voyage – as soon as you have some idea of what’s port and what’s starboard! In this story-fueled session, we’ll take you through differences from the Helm of yore, tips for a successful rollout or upgrade, and opportunities to shape the project’s future. The cloud native waters can be choppy, but a technical deep dive powered by open source tooling will steer you right! Speaker: Lachlan Evenson, Principal Program Manager (Container Compute Team), Azure Bio: Lachlan is a Principal Program Manager on the Container Compute team at Azure. He has spent the last few years working with Kubernetes and enabling cloud native journeys. Lachlan is a cloud native ambassador, emeritus Kubernetes release lead, Helm charts maintainer, and has deep operational knowledge of many Cloud Native projects. @LachlanEvenson Host: Tamao Nakahara, Head of Developer Experience, Weaveworks Bio: Tamao is passionate about Developer Experience and co-organizes devxcon.com. She has over 20 years of DevEx, ecosystem alliances, and event experience, including as Director of Developer Relations at New Relic, running open source community programs at VMware and Pivotal for Cloud Foundry, Spring, Hadoop, RabbitMQ, and Redis, and helping customers with Oracle virtualization at VMware. To learn more about GitOps, check out: The Practical Guide to GitOps (eBook): http://bit.ly/gitops_guide

1. @LachlanEvenson
Helm 3: Navigating to Distant
Shores
Image Credits: Bridget Kromhout

2. @LachlanEvenson
1.why Helm?
2.v3 overview
3.breaking changes
4.new features
5.increasing reliability
6.what’s next?

3. @LachlanEvenson
Who am I?
- Open Source Program Manager at Azure
- Helm Charts maintainer
- Kubernetes 1.16 release lead
- CNCF Ambassador

4. @LachlanEvenson

5. @LachlanEvenson
Find, share, and use software built for k8s
Manage complexity Easy updates
Simple sharing Rollbacks

6. @LachlanEvenson
v3 overview
- based on community best practices
- dramatic simplification
- architectural changes (security as priority)

7. @LachlanEvenson
Major refactor needed
- Helm is almost as old as k8s
- Predating CRDs and k8s RBAC
- Simpler, more secure: focus on production

8. @LachlanEvenson
Making Helm more k8s-native
- Inheriting security controls from kubeconfig
- Using k8s RBAC to limit access & resources
- Replacing custom APIs for charts and
deployments with secrets

9. @LachlanEvenson
Farewell, Tiller
- simpler & more flexible architecture, security, &
upgrades
- now using Kubernetes API directly
- rendering Charts client-side; storing in release
- lowering the barrier of entry for contributors

10. @LachlanEvenson
CLI changes
helm delete ——-> helm uninstall
helm inspect ——-> helm show
helm fetch ——-> helm pull
helm search ——-> helm search repo (vs helm
search hub)
--purge is now default (to override: helm uninstall --keep-history)

11. @LachlanEvenson
Warning! Breaking Changes!
(see https://v3.helm.sh/docs/faq/ for full details)

12. @LachlanEvenson
Don’t worry!
- We intend to support Helm 2 charts
- You *should* be able to replace the Helm 2 binary with
the Helm 3 binary IF you take the following
considerations into account….

13. @LachlanEvenson
Namespace changes
- Release metadata is stored in the same namespace as the
release
- Templated resources with namespace set will be installed into
said namespace prior to the application of `--namespace` flag
- Helm will no longer create a namespace if it doesn’t already
exist

14. @LachlanEvenson
Chart dependency management
- Old style: requirements.yaml and requirements.lock
- New style: Chart.yaml and Chart.lock
(breaking change if you use helm dependency
subcommands)

15. @LachlanEvenson
- `crd-install` ignored in Helm 3.
Useful warning message if
CRDs are present in templates
directory
- Replaced with crds directory at
chart root
CRD installation

16. @LachlanEvenson
Release metadata
- No longer stored in Tiller namespace
- Stored as secret in release namespace
- Double base64 encoded JSON blob
- Not backwards compatible with Helm 2 release
metadata https://github.com/helm/helm-2to3

17. @LachlanEvenson
Deprecated Function(s)
- {{ .Release.Time }} deprecated in favor for `now`
function

18. @LachlanEvenson
--generate-name
- Old style: auto-generated unless overridden
- New style: error unless auto-generation requested

19. @LachlanEvenson
DEMO: Helm 3 end-to-end

20. @LachlanEvenson

21. @LachlanEvenson
Adventure! Excitement! Features!

22. @LachlanEvenson
Chart Repository API
- working towards compat with OCI standard
- available now with a feature gate!
(eventually: pluggable auth! novel artifact types!
host on container registries of your choice!)

23. @LachlanEvenson
Chart API Version 2
- Let 3rd party tools identity v2 charts
- Dependency changes
- Library Charts

24. @LachlanEvenson
Library chart support
- shared by other charts
- does not create any release artifacts of its own
- A library chart’s templates can only
declare define elements
- allows simpler code reuse

25. @LachlanEvenson
Validating Chart Values with JSONSchema
- Chart values can now have
JSON Schema
- Chart maintainers may provide
typed schema for values.yaml
- Better error reporting
- https://v3.helm.sh/docs/topics/
charts/#schema-files

26. @LachlanEvenson
@tek_sauce

27. @LachlanEvenson
three-way merge for upgrade
- three-way merge patch during helm upgrade
- much like kubectl apply
- old config, new config, and current state

28. @LachlanEvenson
Increasing reliability: pre-release checks

29. @LachlanEvenson
Top 3 ways Helm Releases fail
- Invalid Kubernetes resources
- Denied by Policy
- Role Based Access Control

30. @LachlanEvenson
invalid k8s resources
$ helm install stable/nginx-ingress --set
controller.replicaCount=two
Error: release estranged-arachnid failed:
Deployment in version "v1beta1" cannot be handled
as a Deployment: v1beta1.Deployment.Spec:
v1beta1.DeploymentSpec.Replicas: readUint32:
unexpected character: , error found in #10 byte
of ...|eplicas":"two","revi|..., bigger
context ...|default"},"spec":
{"minReadySeconds":0,"replicas":"two","revisionHi
storyLimit":10,"strategy":{},"temp|...

31. @LachlanEvenson
resources don’t work!?
(…on this k8s version)
$ helm install stable/nginx-ingress
Error: validation failed: unable to
recognize "": no matches for kind
"Deployment" in version "extensions/
v1beta1"
https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/

32. @LachlanEvenson
kubeval: install as Helm plugin
$ helm plugin install https://
github.com/instrumenta/helm-kubeval
@garethr - kubeval.instrumenta.dev

33. @LachlanEvenson
kubeval: find invalid deployments
$ helm kubeval stable/nginx-ingress --set
controller.replicaCount=two
[…]
The file nginx-ingress/templates/controller-
deployment.yaml contains an invalid Deployment
---> spec.replicas: Invalid type. Expected:
[integer,null], given: string
The file nginx-ingress/templates/default-backend-
deployment.yaml contains a valid Deployment
[…]
Error: plugin "kubeval" exited with error

34. @LachlanEvenson
kubeval: will a chart work with a given version?
$ helm kubeval stable/nginx-ingress -v 1.15.0
The file nginx-ingress/templates/controller-
serviceaccount.yaml contains a valid
ServiceAccount
The file nginx-ingress/templates/default-
backend-serviceaccount.yaml contains a valid
ServiceAccount
[…]

35. @LachlanEvenson
$ helm conftest stable/nginx-ingress
FAIL - nginx-ingress-controller in the Deployment
release-name-nginx-ingress-controller does not have
a memory limit set
FAIL - nginx-ingress-controller in the Deployment
release-name-nginx-ingress-controller does not have
a CPU limit set
[…]
Error: plugin "conftest" exited with error
conftest: fail if non-compliant with policy

36. @LachlanEvenson
$ helm conftest stable/nginx-ingress/ —set
controller.resources.limits.cpu=100m,contr
oller.resources.limits.memory=64Mi
$
conftest: fail if non-compliant with policy

37. @LachlanEvenson
… and spoiler alert: in Helm 3, with Tiller
gone, you won’t have the “cluster admin”
permissions anymore!
Helm 3 and RBAC

38. @LachlanEvenson
$ for i in `helm template stable/nginx-ingress | grep
-i Kind | awk -F: '{print $2}' | sort -u`; do echo "$i:
`kubectl auth can-i create $i`"; done
Warning: resource 'clusterroles' is not namespace
scoped in group 'rbac.authorization.k8s.io'
ClusterRole: no
Warning: resource 'clusterrolebindings' is not
namespace scoped in group 'rbac.authorization.k8s.io'
ClusterRoleBinding: no
Deployment: yes
Role: yes
RoleBinding: yes
Service: yes
ServiceAccount: yes
kubectl can-i

39. @LachlanEvenson
multiple options:
kubectl auth can-i
(https://kubernetes.io/docs/reference/access-
authn-authz/authorization/#checking-api-
access)
who-can
(https://github.com/aquasecurity/kubectl-
who-can)

40. @LachlanEvenson
What’s next?

41. @LachlanEvenson
Security Audit
- Went great. No major vulns found!
- Preparing Helm for graduation
https://github.com/helm/community/blob/master/
security-audit/HLM-01-report.pdf

42. @LachlanEvenson
Support
After Helm 3 release
- 6 months of bugfixes and security issues
-12 months end-of-life, only security issues will
be accepted.

43. @LachlanEvenson
get involved!
- Upgrade to Helm 3
- https://v3.helm.sh/docs/faq/
- helm.sh for community calls
- feedback on new use cases & workflows
- test for backwards-compat with existing charts!

44. @LachlanEvenson
v3.helm.sh/docs/faq
Thanks!