Software library packages are constantly evolving and increasing in number. Not updating to the latest available release of dependent libraries may negatively affect software development by not benefiting from new functionality, vulnerability and bug fixes available in more recent versions. On the other hand, automatically updating to the latest release may introduce incompatibility issues. We introduce a technical lag metric for dependencies in package networks, in order to assess how outdated a software package is compared to the latest available releases of its dependencies. We empirically analyse the package update practices and technical lag for the npm distribution of JavaScript packages. Our results show a strong presence of technical lag caused by the specific use of dependency constraints, indicating a reluctance to update dependencies to avoid backward incompatible changes

1. An Empirical Analysis of
Technical Lag in npm Package
Dependencies
Ahmed Zerouali, Eleni Constantinou, Tom Mens,
Gregorio Robles and Jesus M. Gonzalez-Barahona
The 17th International Conference on Software Reuse
May 21-23, 2018 - Madrid

2. /background
/aims
/method
/results
/limitations
/conclusion
Outline

3. /background
Packages Releases Dependencies (runtime)
+700K +4.5M +20M
+145K +825K +2M
+130K +840K +2.3M
Libraries.io by March 2018
RubyGems

4. /background
Open PRs Active Bugs
+2.3M -
+2M -
- +120K
by January 2018 - https://octoverse.github.com/

5. /background
Technical lag*: the increasing difference between
deployed software packages and the available
upstream packages
Measurement: version updates, bugs, vulnerabilities,
line of code, commits, etc.
(*) Gonzalez-Barahona, et al. "Technical Lag in Software Compilations: Measuring How Outdated a Software Deployment Is."
IFIP International Conference on Open Source Systems. Springer, Cham, 2017.
Gold standard: stability, security, functionality, etc.

6. /background
Example: different kinds of “gold standards” for Debian
Gold standard Scenario Candidate
Stability Isolated system, stable
functionality
Debian Stable
Functionality Cloud application Latest upstream
Security Reused containers Stable upstream

7. /background

8. /aims
Decan A, Mens T, Grosjean P. An empirical comparison of dependency network evolution in seven software packaging ecosystems. EMSE2017.

9. /aims
Not update
Update
“How one developer just broke Node, Babel and thousands of projects in 11 lines of JavaScript” -
https://www.theregister.co.uk/2016/03/23/npm_left_pad_chaos/

10. /aims
Goal: Analyze technical lag in a wide ecosystem of
reused of packages.

11. /aims
/ research questions
RQ0: How do packages manage their dependencies?
RQ1: How often do packages release new versions?
RQ2: What is the technical lag induced by outdated
dependencies?
RQ3: How often do dependencies inducing technical lag release
new versions?
RQ4: What is the appropriate moment to update dependencies?

12. /method
/dataset

13. /method
/dataset
Open Data:
- Libraries.io gathers data from 36 package managers and 3 source code
repositories.
- They track over 2.7m unique open source packages, 33m repositories and
235m interdependencies between them.

14. /method
/dataset
- 610K packages
- 4.2M releases
- 44.9M dependencies
by Nov 2017
from

15. /method
/semantic versioning
Examples: 0.0.1, 1.0.0, 1.2.3, 1.2.3-beta

16. /method
/semantic versioning
Other: *, ==1.2.3, >1.2.3, <1.2.3, 1.2.x, 1.x.x

17. /method
/technical lag
- Measurement = version updates, time
- version lag : version updates difference
- time lag: time difference
- Gold standard = being up to date.

18. /method
/technical lag
1.0.1 1.1.0 2.0.01.2.1 2.1.0
Dependency: D
npm package
version
Technical lag

19. /method
/technical lag
- time lag = date(latest) - date(used)
- version lag = (∆Major, ∆Minor, ∆Patch)

20. /method
/technical lag
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependency: D
npm package
version
Technical lag
- time lag= date(2.1.0) - date(1.1.0)

21. /method
/technical lag
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependency: D
npm package
version
Technical lag
1 minor
- time lag= date(2.1.0) - date(1.1.0)

22. /method
/technical lag
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependency: D
npm package
version
Technical lag
- time lag= date(2.1.0) - date(1.1.0)
1 minor
1 major

23. /method
/technical lag
1.0.1 1.1.0 2.0.01.2.0 2.0.1
Dependency: D
npm package
version
Technical lag
1 minor
1 major 1 patch
- time lag= date(2.1.0) - date(1.1.0)
- version lag= (1,1,1)

24. /method
/technical lag
1.0.1 1.2.0 2.0.1
3.6.0 4.1.04.0.0 5.0.0
2.0.0 2.1.0
npm package: P
dependency: D
^1.0.0
Technical lag
*
^1.0.0 ^2.0.0
^1.0.0 = [ 1.0.0, 2.0.0 [
allowed

25. /method
/technical lag
1.0.1 1.2.0 2.0.1
3.6.0 4.1.04.0.0 5.0.0
2.0.0 2.1.0
npm package: P
dependency: D
^1.0.0
Technical lag
*
^1.0.0 ^2.0.0
allowed
^1.0.0 = [ 1.0.0, 2.0.0 [

26. /method
/technical lag
1.0.1 1.2.0 2.0.1
3.6.0 4.1.04.0.0 5.0.0
2.0.0 2.1.0
npm package: P
dependency: D
^1.0.0
Technical lag = 0
*
^1.0.0 ^2.0.0
allowed
^1.0.0 = [ 1.0.0, 2.0.0 [

27. /results
/RQ0: How do packages manage their dependencies?
68.2%
15.7%
7.8%
4.1%
- Developers are concerned with backward compatibility
- There is a potential of too strict dependency constraints leading to
technical lag.
4.3%

28. /results
/RQ0: How do packages manage their dependencies?
- New dependencies are mainly added in major and minor releases.
- Dependencies are removed almost exclusively in major releases.
- Most packages do not appear to change their dependencies over time.

29. /results
/RQ0: How do packages manage their dependencies?
Number of updated dependencies between package releases,
classified by release type of the update.

30. /results
/RQ1: How often do packages release new versions?
- Patch: 80% - Minor: 16% - Major: 04%
- Dependent packages in npm mainly benefit from patch releases.
- Technical version lag is mainly occurring at the patch level.

31. /results
/RQ1: How often do packages release new versions?
Time needed to update a package to a patch, minor or major release
- The average time to release a patch, minor and major versions
corresponds to 13 days, 1 month and 2 months respectively.

32. /results
/RQ2: What is the technical lag induced by outdated dependencies?
- 27% of 44.1M dependencies are outdated.
- The outdated dependencies are used by 60% of all considered packages.
57%
28%
12%
3%

33. /results
/RQ2: What is the technical lag induced by outdated dependencies?

34. /results
/RQ2: What is the technical lag induced by outdated dependencies?
- Outdated dependencies induce a median of time lag of three months
and a half, and median version lag of one minor and two patch versions.

35. /results
/RQ3: How often do dependencies inducing technical lag release new versions?
- Packages that are required as dependencies and are outdated have
more frequent releases than other required packages.

36. /results
/RQ4: What is the appropriate moment to update dependencies?
- Developers should not start using newly available packages
immediately because they may still contain bugs that need new
patches.

37. /limitations
- If the libraries.io dataset is incomplete, then there is a risk of underestimating technical lag.
- We did not differentiate between package characteristics, such as age, size, type, etc.
- The results are related to the measurement used to quantify for technical lag.
- npm semver had some issues in the past.

38. /conclusion/
summary
Analyzed technical lag induced by outdated dependencies:
- A large number of packages suffer from technical lag.
- Outdated dependencies are several months behind the latest release.
- Technical lag caused by the specific use of dependency constraints,
- Maintainers should wait a few days before updating to the new patch dependency release.

39. /conclusion/
future work
- Consider other measurements of technical lag and other gold standards.
- Validate the results with bug fixes, vulnerabilities and issues.
- Consider other ecosystems.
- Carry out cross-ecosystem comparisons.

41. Questions