A look at how containerd and Kubernetes CRI could work together by Tim Hockin at the containerd summit

1. Containerd and CRI

2. Kubernetes: Container Runtime Interface (CRI)
● A new plugin interface for container runtimes
○ RuntimeService, ImageService
● A refactoring of organically evolved code
● Make Kubernetes more extensible
○ Empower arbitrary 3rd party runtimes without sending us a PR
kubelet
CRI
shim container
runtimegrpc
client
containercontainer
container
container
containergrpc
server

3. Pod Sandbox
● Pod is composed of a group of application containers in an isolated
environment with resource constraints
● Pod Sandbox is the environment
● Isolation: interpreted flexibly by container runtimes
○ Network namespace
○ Virtual machine
● Resource constraints
○ Pod-level cgroup
● Container runtime is completely responsible for network setup

4. Imperative Container Operations
● CRI provides knobs for kubelet to implement higher-level features
○ lifecycle hooks
○ liveness/readiness checks
○ container restarts & backoff
● Why imperative container-level operations?
○ Flexibility vs. Feature velocity
○ User-facing Kubernetes API is still declarative

5. Container stdout/stderr Logs
● Requirements
○ Better log/disk management
■ Decouple the lifecycle of logs and containers
○ Support kubernetes logging features (e.g., kubectl logs --since)
■ Need understanding of log formats
● Solution
○ Instruct the runtime to store logs at a given path
■ /var/log/pods/<podUID>/<containerName>_<instance#>.log
○ Ask runtime to decorate the logs with a standard format
2016-10-06T00:17:09.669794202Z stdout The content of the log entry 1
2016-10-06T00:17:10.113242941Z stderr The content of the log entry 2
2016-10-06T00:17:11.241390016Z stderr The content of the log entry 3

6. Image Service
● Kubernetes supports only Docker images (as of release 1.6)
● Basic image management operations
○ ListImage, ImageStatus, PullImage, RemoveImage
● Refer to the images by name or digest
● Runtime Service needs to be able to locate and use the image to create a
root filesystem

7. Pod and Container Lifecycle
● PodSandbox rpc calls
○ RunPodSandbox, StopPodSandbox, RemovePodsandbox
● Container rpc calls
○ CreateContainer, StartContainer, StopContainer, RemoveContainer, Exec,...
● Life of a pod
○ Create the Pod Sandbox first, then add containers to it
○ RunPodSandbox -> CreateContainer -> StartContainer -> StopContainer ->
StopPodSandbox -> RemoveContainer -> RemovePodSandbox

8. Recap
● CRI is …
○ a plugin interface to support a wide variety of container runtimes in kubernetes
● CRI is NOT…
○ an interface for a full-fledged, all-inclusive container runtime
○ a user-facing API

9. CRI and Docker Engine
● CRI had to build on top of the Docker engine API
○ Docker is the default, and the only container runtime supporting all kubernetes features
○ Ensuring a smooth transition
● This brings extra constraints and limitations

10. CRI and Docker Engine - Challenges
● Adding one more hop from kubelet to runc
○ kubelet -> CRI shim -> docker daemon -> containerd -> runc
● Extra bells and whistles that are not used by kubernetes
○ Volumes, networking, etc.
○ Larger surface to validate and maintain
● Limited support for other image formats
○ OCI

11. CRI and Docker Engine - Challenges (cont.)
● Stdout/stderr logs
○ The lifecycle of logs are coupled with containers
○ CRI log format is not supported
○ Log path is not configurable
○ Temporary solution: continue using the JSON log driver
■ Support the Docker JSON format in kubelet
■ Use symbolic links to access the logs

12. A Better Match: containerd
● Provides only the core functionality we need
● Designed to be embedded into a larger system
● No tight dependence between images and containers
● Supports OCI images
● Eliminates one hop (docker daemon)
kubelet
CRI
shim
containerd
containercontainer
container
container
container

13. Near-Term plans
● Figure out the mapping of containerd to CRI
○ Make sure it has everything kubernetes needs
○ Help fill any gaps
● Proof-of-concept CRI shim
○ Likely challenges: logs, exec
● Community presentation and discussion

14. Current Status of CRI
● Kubernetes 1.5: v1alpha1 API
○ Still evolving
○ Missing container stats API, etc.
● Kubernetes 1.6 (Mar. 2017): Kubelet using CRI by default
○ Deprecating the old Docker implementation in the next release
● Ongoing CRI-compatible runtime projects
○ cri-o: oci-conformant runtimes
○ dockershim: the built-in docker-CRI integration
○ frakti: hypervisor-based runtimes
○ rktlet: the rkt container runtime
○ virtlet: a VM (QCOW) runtime