Free and Open Source Software distributions raise difficult problems both for distribution editors and system administrators. Distributions evolve rapidly by integrating new versions of software packages that are independently developed. System upgrades may proceed on different paths depending on the current state of a system and the available software packages, and system administrators are faced with choices of upgrade paths, and possibly with failing upgrades. Mancoosi develops mechanisms that provide for rollbacks of failed upgrade attempts, allowing the system administrator to revert the system to the state before the upgrade, and better algorithms and tools to plan upgrade paths based on various information sources about software packages and on optimization criteria. The consortium as a whole is entirely committed to the free software movement, and the project sets up virtuous cycles associating users, industry and researchers that will outlive the project itself. Mancoosi is a European research project in the 7th Research Framework Programme (FP7) of the European Commission, which has started February 1st, 2008, and has a duration of 3 years.

1. Mancoosi: bridging communities to improve
package-based systems
Roberto Di Cosmo
University Paris Diderot Paris 7 and INRIA
Fossa 2009
November 18th, 2009

2. The promise of GNU/Linux distributions
GNU/Linux distributions are intermediaries between FOSS projects
and their users
They factor the work needed to mainain, configure, test, package
together tens of thousands of different software components.

3. Package-based distributions: a very successful idea . . .
Central notion : package, together with package management
software
A main innovations of GNU/Linux distributions: it opens the door
to flexible configuration and management of complex systems via
reusability and modularity.

4. Package management: a key industrial issue
A few facts:
In 2011, 80% of commercial software will contain Open
Source code. . . 49,7% of critical application come from
the Open Source world...
Mark Driver (Gartner) sept.07
A key industrial issue is to build faster, stronger, harder, better,
more user oriented packaging technology, to master the challenge
of the Open Source success.
And yet, surprisingly little has been done in terms of R&D
specifically related to package technology (as opposed to
components or web services).

5. The Challenge of Distributions
Mandriva, Debian, Ubuntu, and all other distribution editors are
responsible for maintaining a free software distribution. It is a
challenging task! (smooth upgrades, automatic dependency
solving, up to date software . . . )
Help is needed to build: better infrastructure for package
maintainers and better package managers for final users.
Two projects to the rescue:
EDOS [2004–2007] aim: provide FOSS distribution editors
with better QA tools
Mancoosi [2008–2011] aim: provide with better package
managers to improve their “platform management”
experiences (upgrade, downgrade)

6. Packages, metadata, installation
some files
Package = some scripts Example
metadata Package: aterm
Version: 0.4.2-11
Identification
Section: x11
Inter-package rel. Installed-Size: 280
Dependencies Maintainer: G¨ran Weinholt ...
o
Conflicts Architecture: i386
Feature declarations Depends: libc6 (>= 2.3.2.ds1-4),
libice6 | xlibs (>> 4.1.0), ...
Other
Conflicts: suidmanager (<< 0.50)
Package maintainer Provides: x-terminal-emulator
Textual descriptions ...
...
a package is the elemental component of modern distribution
systems (not GNU/Linux specific)
a working system is deployed by installing a package set
(≈ 1000/2000 for GNU/Linux distro)

7. Sample installation
Phase Trace
User request # apt-get install aterm
8Reading package lists... Done
>
>Building dependency tree... Done
>
>
>
>The following extra packages will be installed:
>
>
>
> libafterimage0
>
>
>
>The following NEW packages will be installed
<
Constraint resolution aterm libafterimage0
>
>0 upgraded, 2 newly installed, 0 to remove and 1786 not upgraded.
>
>
>Need to get 386kB of archives.
>
>
>
>After unpacking 807kB of additional disk space will be used.
>
>
>
>Do you want to continue [Y/n]? Y
>
:
8
>Get: 1 http://debian.ens-cachan.fr testing/main libafterimage0 2.2.8-2 [301kB]
<
Package retrieval Get: 2 http://debian.ens-cachan.fr testing/main aterm 1.0.1-4 [84.4kB]
>Fetched 386kB in 0s (410kB/s)
:
Pre-configuration {
8Selecting previously deselected package libafterimage0.
>
>(Reading database ... 294774 files and directories currently installed.)
>
>
<
Unpacking libafterimage0 (from .../libafterimage0_2.2.8-2_i386.deb) ...
Unpacking
>Selecting previously deselected package aterm.
>
>Unpacking aterm (from .../aterm_1.0.1-4_i386.deb) ...
>
:
(
Setting up libafterimage0 (2.2.8-2) ...
Configuration Setting up aterm (1.0.1-4) ...
each phase can fail (it actually happens quite often . . . )
efforts are needed to identify errors as early as possible

8. One of EDOS outcomes: edos-debcheck
EDOS has brought some major results for distribution editors:
theory : checking if the dependencies of a package can be
satisfied is NP-complete
practice : several tools have been developed during EDOS (≈
110’000 OCaml LOCs), some examples:
edos-debcheck command line checker for package
installability
pkglab interactive, console-based environment
for repository inspection
ceve parser/converter between package list
formats
tart optimised algorithm to cut a repository
into slices (e.g. media), so that
packages available on the i-th slice are
installable using only slices up to i

9. Focus on edos-debcheck
Written by Jerˆme Vouillon, edos-debcheck takes as input an
o
APT package list (e.g. /var/lib/apt/lists/*) and checks
whether one, several, or all packages in it are installable wrt that
repository.
Customized SAT solver, quite fast: checking installability of all
package in main testing/amd64 takes 5 seconds on an entry-level
machine.
Example
edos-debcheck </var/lib/apt/lists/... main binary-amd64 Packages
Parsing package file... 1.2 seconds 21617 packages
Generating constraints... 2.3 seconds
Checking packages... 1.5 seconds
acx100-source (= 20070101-3): FAILED
alien-arena (= 7.0-1): FAILED
alien-arena-browser (= 7.0-1): FAILED
alien-arena-server (= 7.0-1): FAILED
alsa-firmware-loaders (= 1.0.16-1): FAILED
amoeba (= 1.1-19): FAILED
...
# explanation can be required as well

10. The Mancoosi project [ http://www.mancoosi.org ]
project strep FP7 (2008–2011)
focus coherence and maintenance of a FOSS distribution
installation (user point of view)
Tools for user-side package management (package managers /
meta-installer)
installation
removal
upgrade
downgrade
...

11. An upgrade problem example
# sudo apt-get install debhelper
Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
armagetron armagetron-common autoconf bonobo-activation codebreaker debconf
debconf-i18n debconf-utils dialog esound-common fb-music-high fontconfig
frozen-bubble-data grepmail gv intltool-debian libaiksaurus-data
libaiksaurus0c102 libatk1.0-0 libatk1.0-dev libbonobo-activation4 libbonobo2-0
libbonobo2-common libdb3 libdbd-mysql-perl libdbi-perl libeel2-data libesd0
...
The following packages will be REMOVED:
autoconf2.13 frozen-bubble frozen-bubble-lib gconf2 gnomemeeting itk3.1-dev
libbonoboui2-0 libbonoboui2-common libdigest-md5-perl libforms0.89 libgconf2-4
libgnome2-0 libgnome2-common libgnomeui-0 libgnomevfs2-0 libgnomevfs2-common
libgtk1.2-dev libgtk2.0-0png3 libgtk2.0-dev libmime-base64-perl
libpango1.0-dev libsdl-mixer1.2-dev libsdl-perl libsdl-ttf1.2-dev
libsdl1.2-dev libsmpeg-dev libstorable-perl nautilus tk8.3-dev tktable-dev
x-window-system x-window-system-core xaw3dg-dev xlib6g xlib6g-dev xlibmesa-dev
xlibmesa3 xlibosmesa3 xlibs-dev xlibs-pic xpdf xpdf-reader
The following NEW packages will be installed:
armagetron-common debconf-i18n fb-music-high fontconfig intltool-debian
libaiksaurus-data libaiksaurus0c102 libeel2-data libfilehandle-unget-perl
libfontconfig1 libforms1 libgdbm3 libgnutls7 libgsf-1 libice-dev libice6
libidl0 liblzo1 libmagick5.5.7 libmail-mbox-messageparser-perl
libmysqlclient12 libncursesw5 libnet-daemon-perl libnewt0.51 libpaper1
libplrpc-perl libsdl-console ...
75 packages upgraded, 80 newly installed, 42 to remove and 858 not upgraded.
Need to get 67.1MB of archives. After unpacking 26.9MB will be used.
Do you want to continue? [Y/n] Abort.

12. Goals, approach and strategy
Mancoosi’s goal:
enable safe, efficient maintainability of the Open Source
software infrastructure built out of software packages
A twofold approach to reach this objective:
design algorithms and tools to tackle the upgradeability
problem;
this allows to find good upgrade paths w.r.t. the user’s needs.
conceive and implement a transactional update process;
this allows to roll-back an unsatisfactory upgrade.
A long term, ecosystem strategy to reach this objective:
bring users, distributions, developers and researchers together!
set up virtous cycles

13. All in a picture
Mancoosi architecture
In Mancoosi, there are really two main activities, corresponding to
the architecture of our design, and they are loosely coupled
User request WP4,WP5
↓
Platform management interface
Optimal upgrade path computation WP4, WP5
Transactional component manager WP2, WP3
↓
Software Platform

14. Upgrade resolution
Abstract view of packages through the metadata lens.

15. Transactional updates
Concrete view of packages through the maintainer scripts lens.

16. Mancoosi technology preview: strong dependencies
GNU/Linux software distributions: probably the most complex
system of systems on the planet
HOW can you visualize such a system of systems?

17. Draw a graph, no deep knowledge
kde-amusements and kde-toys in Debian 3.1 (aka sarge, 2005)
were unnecessary components, overlapping kde-games and
kde-edu.They are waste, and are gone today (2009).
Should be easy to spot, only 244 components are concerned!
Can you see this fact here?

18. Let’s try a more systemic approach
Here is a graph of strong dominators for the KDE subsystem:
See the problem, now?
More on this at http://www.mancoosi.org

19. Concluding remark
Everything developed in Mancoosi is already relevant for some, and
will be increasingly relevant to all component based technologies
Linux : packages
Eclipse : plugins
Firefox : extensions
Java : beans
Web services : predictable service assembly
... ...

20. Project Partners

21. Family pictures. . .
the Mancoosi team, Feb 2008

22. Questions
Thank you for your attention.