This presentation was given during Benevol 2020. https://benevol2020.github.io/ Abstract: Container-based solutions, such as Docker, have become increasingly relevant in the software industry to facilitate deploying and maintaining software systems. Little is known, however, about how outdated such containers are at the moment of their release or when used in production. We address this question, by measuring and comparing five different dimensions of technical lag that Docker container images can face: package lag, time lag, version lag, vulnerability lag, and bug lag. We instantiate the formal technical lag framework from previous work to operationalise these different dimensions of lag on Docker Hub images based on the Debian Linux distribution. We carry out a large-scale empirical study of such technical lag, over a three-year period, in a large dataset of Debian images. We compare the differences between official and community images, as well as between images with different Debian distributions: OldStable, Stable or Testing. The analysis shows that the different dimensions of technical lag are complementary, providing multiple insights. Our research offers empirical evidence that developers and deployers of Docker images can benefit from identifying to which extent their containers are outdated according to the considered dimensions, and mitigate the risks related to such outdatedness.

1. On the Evolution of Technical Lag in
Debian-based DockerHub Images
Ahmed Zerouali, Tom Mens, Alexandre Decan,
Jesus Gonzalez-Barahona and Gregorio Robles.
THE 19TH BELGIUM-NETHERLANDS SOFTWARE EVOLUTION
WORKSHOP
LUXEMBOURG, 3/4 DECEMBER 2020
1

2. About Docker container images
- A Docker image is a read-only template that contains a set of instructions for
creating a container.
- A container is a lightweight, standalone, executable package of software.
2

3. Motivation
ClusterHQ, Inc
3

4. Other main concerns for
container adoption:
• Dependencies (required
packages)
• Bugs in third-party software
• Outdated third-party software
Motivation
Anchore, Inc
4

5. A method to assess how vulnerable,
buggy and outdated Docker images are.
Goal
5

6. Technical lag
Technical lag: the increasing difference between deployed software
packages and the ideal available upstream packages.
➢ Ideal: stability, security, functionality, etc.
➢ Difference: version updates, bugs, vulnerabilities, line of
code, commits, etc.
6

7. Technical Lag
7

8. ● is a set of component releases
● is a set of possible lag values
● ideal : → is a function returning the “ideal” component release
● delta : x → is a function computing the difference between
two component releases
● agg : is a function aggregating the results of a set of lags
A Framework of Technical Lag
8

9. Given a technical lag framework , we define:
Aggregated Technical lag:
Technical lag:
Let be a set of components, then:
A Framework of Technical Lag
9

10. How does technical lag evolve in DockerHub
images?
Research Question
10

11. Technical Lag in DockerHub images
➢ Ideal: Highest available version
11

12. Case study
Type of data Data source
Package metadata Debian Archive
Security vulnerabilities Debian Security Tracker
Bugs Ultimate Debian Database
12

13. Results
/Package lag
Community images have higher package lag than official ones.
Only < 3% of packages are outdated in community images.
13

14. Testing images have higher package lag, because they are
frequently updated in the Debian repository.
Results
/Package lag
14

15. Results
/Time lag
The median time lag of community images is well over a year,
and it is highest for OldStable images.
15

16. Results
/Version lag
The median version lag of community images is 7 missed
versions. Testing images have a higher version lag.
16

17. Results
/Vulnerability lag
Community images have a median vulnerability lag of 10 vulnerabilities.
OldStable images have a higher vulnerability lag than other images.
17

18. Results
/Bug lag
Testing images have a higher bug lag than Stable images
because they tend to come with bug fixes.
18

19. Discussion
Package lag
Time lag
Version lag
19

20. Vulnerability lag
Bug lag
Discussion
20

21. Technical lag should be measured in different ways, offering
complementary information.
The technical lag could help Docker users to keep their images
and containers in a healthy shape.
Conclusion
21

22. 22

23. Technical Lag in DockerHub images
➔ package lag indicates whether a given package release is outdated;
➔ time lag quantifies the time difference between two release dates;
➔ version lag quantifies the number of missed versions between releases;
➔ vulnerability lag measures the difference in number of vulnerabilities;
➔ bug lag measures the difference in number of bugs.
23