This document discusses improving rollback capabilities in Linux via a domain specific language (DSL) approach and distributing data. It presents an overview of rollback, different types of rollback, and challenges. The authors propose using a DSL to model package maintainer scripts to better support rollback. The DSL would be logged and used to drive an automated rollback. Distributed data storage via peer-to-peer networks is also proposed to share rollback logs and package state information. The overall goal is to build a framework enabling transactionally protected package management including reliable rollback.
Improving Rollback in Linux via DSL approach & distributing
1. Energia Open Source
Improving Rollback in Linux
via DSL approach & distributing
Sunday, 6th February 2011
Presented by:
John Thomson: John.Thomson@caixamagica.pt
Researcher
Paulo Trezentos: Paulo.Trezentos@caixamagica.pt
http://twitter.com/PauloTrezentos
R&D Director
2. Energia Open Source
Overview
An overview of what Caixa Mágica does.
– MANCOOSI project: European 7th Framework Programme
Roll-back
– Definition
– Types of roll-back
– As part of a bigger system
DSL
– Definition
– Example
Approach
Implementation
Distributing data (kad/p2p)
Conclusion
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 2
3. Energia Open Source
A brief summary of Caixa Mágica Software
and an overview of what we do:
Caixa Mágica is a FOSS software and solutions
provider, based in Lisbon, Portugal
Linux- Caixa Mágica 15 is the main Linux
Distribution available in Portugal
Caixa Mágica work with national companies,
Government and with European partners to promote Open
Source in all aspects of work
Over 800,000 Linux CM systems installed (single & dual boot)
Interesting development → CM16 moving to Debian base
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 3
4. Energia Open Source
European based research projects
Caixa Mágica works in many multi-national European based
research projects. After EDOS (EU FP6 STREP) based project
it is now actively working on MANCOOSI-FP7 Project
Also working on ULOOP and Timbus projects
Work with a multitude of top-tier Universities and research
institutions
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 4
5. Energia Open Source
About MANCOOSI
MANCOOSI - Managing the complexity of open-
source software. www.mancoosi.org
Solving package management issues that have
been identified through EDOS project
Other talks
Talks yesterday by Ralf and Jaap about solvers + tools
André Guerreiro presents Gumby in room H.1302 at 15:30
– Visualise package meta-data
Paulo Trezentos presents APT-PBO in room H.1302 at 16:00
– A better class of solver
Aim is to provide better tools for package management
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 5
6. Energia Open Source
Roll-back as simple as traversing time?
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 6
7. Energia Open Source
What does roll-back mean, really?
In terms of package configuration,
“roll-back is the process of inverting the changes to the
system made by package upgrades to get back to a
particular system state.”
Many other mechanisms out there that work on using file
system snapshots/saving the state, (next slide)
S1 S2
PkgFoo v 1.00 Roll-back, PkgFoo v 2.00
Time: 10.00pm possible? Time: 10.20pm Installation Timeline
The mechanism is one part of Transactionally Protected
Package Management as spoken about by Jeff Johnson last
year at FOSDEM 2010.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 7
8. Energia Open Source
Different types of roll-back
Conary, a 2nd Generation Package Manager, aims to
improve current meta-installers. Used in Linux
distributions such as Foresight Linux.
Augeas, is a configuration management tool that
makes manipulating config files much easier
ZFS, used by Nexenta is an example of a file-system,
snapshot mechanism to snapshot several system states.
NixOS, a revolutionary system that re-thinks how files
and resources are used to make them purely functional
Other mechanisms e.g. etckeeper being developed by
Fedora also try and capture configuration files into a
VCS. BTRFS – another snapshot based mechanism.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 8
9. Energia Open Source
Drawbacks of the various approaches
File-system Snapshot based mechanisms
– Capture state of the whole system → down to individual files
– Agnostic of packages
– Granularity is not well matched for packages
Package Based
– Same as installing an older version
– Maintain configurations and settings
System Call based
– Syscall trapping - records changes to the file-system
– Capturing files alone is not always enough
Aim is to combine the complimentary parts of these systems
to cover a wide range of situations and events
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 9
10. Energia Open Source
Difficulties of Roll-back
Package maintainers and developers tend to think in the
forward direction of upgrading packages
Working in the reverse direction is a relatively unheard of
concept. Think of a toy slinky
Downgrade is counter-intuitive
Already have been done if it was necessary?
Rolling-back changes is 'only' needed when a package fails
to work on the system
There may be cases where roll-back is impossible using
the techniques that we have investigated, or possibly at all
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 10
11. Energia Open Source
Roll-back is one part of a bigger picture
Ability to undo package upgrade/install is just one benefit of
having a system that can capture the package configuration
state and maintain a deterministic, state transition model of
the system.
By examining the current maintainer scripts and templates
provided by deb/rpm-helper, we defined a language that
assists with problems not addressed now by meta-installers.
Transactionaly Protected Package Management (TPPM) is
what we are aiming to move towards.
TPPM
roll-back roll-back
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 11
12. Energia Open Source
Advantages and benefits of Roll-back/TPPM
By building a framework in which R/B + TPPM is possible we
can capture the functional elements of maintainer scripts
Error Detection
Simulate/test the results of
an upgrade on a model of system
Detect errors a-priori or
potential 'slow' errors after upgrades
Detect/(correct) sequencing orders in package installations
Roll-back
Framework → drive roll-back. Now provide more information
to a roll-back utility than available to current meta-installers
Identify which packages can/cannot perform roll-back
Allow individual packages to change versions and show the
dependencies for those changes
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 12
13. Energia Open Source
Domain Specific Language (DSL) -package upgrades
Our DSL is a language used to abstract from the system and
model to solve particular, identified problems
In our case, the DSL is focused on analysing package
maintainer scripts
Designed not to be a Turing Complete Language like BASH
but rather something which can resolve problems
It is an evolutionary language designed to capture the details
of the vast majority of common maintainer scripts
Increase the coverage of DSL by examining new scripts
Many thanks to University of L'Aquila for their creation of the
language and for their support in using it
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 13
14. Energia Open Source
System Integration
For roll-back we will have a log of
Apt the DSL commands executed in-
sequence
model_simulator (( ))
model_simulator
Perform roll-back → run inverse
statements associated with those
DSL_rollback_pre (( ))
DSL_rollback_pre commands in 'reverse' order
Have the simulator to pre-check if
run_transaction ()
run_transaction () the package configuration can be
rolled-back; whether it will leave it
DSL_rollback_post (( ))
in an erroneus state
DSL_rollback_post
Performing LIFO style roll-back we
run post commands before we run
the pre statements
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 14
15. Energia Open Source
Domain Specific Language (DSL) Example
The DSL has been created to assist with some of the
problems discovered and analysed by prior research
Using cups.spec %post example from CUPS-1.4.2
%post
dslstart postinst_init(cups)
/sbin/chkconfig add cups Matched DSL Pair
/sbin/chkconfig cups on
dslstop postinst_init(cups)
# Restart cupsd if we are upgrading...
dslstart post_init_restart(cups)
if test $1 gt 1; then
/sbin/service cups stop Matched DSL Pair
/sbin/service cups start
fi
dslstop post_init_restart(cups)
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 15
16. Energia Open Source
Example continued, Log
rbHist dslID TID parentID DSL_CMD bhINVERSE
1 1 1 start postinst_init(cups) TRUE
2 1 1 stop postinst_init(cups) TRUE
3 1 1 start post_init_restart(cups) TRUE
4 1 1 stop post_init_restart(cups) TRUE
pkgHist id parent op pkgName pkgVer1 pkgVer2 dateTime
1 1 inst cups 0 1.4.2 2010-01-30
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 16
17. Energia Open Source
Transactions- what happens if a maintainer
script fails?
If a maintainer script fails in the middle of one of the
operations will have a log like this
dslID TID parentID DSL_CMD bhINVERSE
1 1 1 start postinst_init(cups) TRUE
2 1 1 stop postinst_init(cups) TRUE
3 1 1 start post_init_restart(cups) TRUE
Transaction has quite obviously failed. No matching end for a
DSL command reached. Odd number of elements etc.
Perform a roll-back for all matching sub-transaction ID
elements, but in the reverse order with certain constraints.
If a set of script elements cannot perform roll-back in the
middle of operating, then don't create a dsl tag.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 17
18. Energia Open Source
Performing the roll-back
Identify the sub-transaction/package upgrade to roll-back.
Check that all commands in the database have an inverse for
the particular transaction.
Run them in reverse-operation order.
dslstart postinst_init(cups) [1]
dslstop postinst_init(cups) [2]
dslstart post_init_restart(cups) [3] //Restart not needed postrm
becomes
postrm_init_restart(cups) [3] //Remove nonneeded element
prerm_init_stop(cups) [4] //From lookup table.
prerm_init(cups) [1+2]
In other cases the order might be [3,2,1]. Identified by tags.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 18
19. Energia Open Source
Architecture
Modified Package as input → DSL extracted → Log storage
and simulator → Mechanism for executing roll-backs →
Maintenance of scripts etc.
User I/P
Inject DSL
dslstart postinst_init(cups)
dslend postinst_init(cups) Logs
Roll-back
SQLite DB
.spec file Apt-rpm
%post
/sbin/chkconfig --add cups Modified System State
/sbin/chkconfig cups on WP2
...
Simulator
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 19
20. Energia Open Source
Overall Approach
Analyse maintainer scripts in standard systems
Identify common themes in maintainer scripts
Create first version of the DSL
Modify maintainer scripts to contain DSL
Log DSL into SQLite DB
Create roll-back mechanism
Logged Data + Stored Info → Perform Roll-back
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 20
21. Energia Open Source
A state/time perspective of roll-back
apt-get [Maintainer Scripts]
T0 install cups In terms of DSL
apt-get DSL cups pkg
rollback apt-rpm DSL stmts
Log
cups 1.2
apt-get
T0+n rollback
TID-028 [Maintainer Scripts]-1
apt-get
rollback
roll-back
2010-02-06
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 21
22. Energia Open Source
Distributing Rollback-Information
Rollback data can be distributed
– Transceive data onto Kademlia network.
– Distribute rpms/rollback data/packages state to network
– Private data submission to network? Opt-in mechanism
Kad/P2P Network
Logs Config cups pkg
SQLite DB data
Normal upgrades
Initial System State Modified System State
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 22
23. Energia Open Source
Todo list:
Completed:
– Single package rollback with DSL
– Supporting framework e.g. SQLite3 and script injectors.
Now in the process of:
– Integrating Simulator/Failure Detector into package upgrade
– Generating tool suite for automatic detection/generation of DSL
commands for a given set of packages and their scripts
– Creating web interface for changing the auto-created DSL pkgs
– Completing rollback mechanism for multiple packages txns.
– Testing
– Documentation
– Release + revisions
– Integration into build systems
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 23
24. Energia Open Source
Conclusion
The need for Roll-back of packages is growing
Not a trivial problem
Need to consider the problems of
complex upgrade paths
Using the DSL we aim to remove
a lot of the failure cases in
package upgrades.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 24
25. Energia Open Source
Questions?
Thank you for listening. I hope that it was interesting and
that you have some questions to ask.
06.02.2011 Improving Rollback in Linux via DSL approach & distributing 25