Free software is pervasive, and is being used in some form in practically all software developments. We now face the challenge, and have the opportunity, of creating and using tools that will allow us to accompany the growth of Free Software over the next years.
How to Troubleshoot Apps for the Modern Connected Worker
Free Software: Challenges and opportunities for the next decades, Roberto di cosmo
1. Free/Open Source Software: some challenges and
opportunities for the next 10 years
Roberto Di Cosmo
IRILL Director
www.dicosmo.org - www.irill.org
Universit´ Paris Diderot - IRILL - INRIA
e
France
November 14th, 2013
OW2Con
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
1 / 28
2. Short Bio
15 years of Free and Open Source Software
1998 Pi`ge dans le Cyberespace
e
1999 DemoLinux
2004 EDOS
2007 Free Software Thematic Group
http://www.
systematic-paris-region.
org/fr/logiciel-libre
100 members (SMEs, Labs)
30 projects ( 150Me)
2008 Mancoosi project
www.mancoosi.org
2010 IRILL www.irill.org
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
2 / 28
5. Reminder: FOSS
free (as in free beer, or gratuit) software which has not to be payed
(today)
free (as in free speech, or libre) software granting 4 freedoms to its users:1
0
1
2
3
freedom to use the software
freedom to study the source code of the software and to
adapt it to user needs
freedom to distribute software copies
freedom to distribute modified software copies
two points of view: the freedom one as outlined above (“free software”),
the technical one pivoting around source code availability (“open source”)
1
there are of course also obligations, which vary according to the license: GPL, BSD,
Mozilla, MIT/X, AGPL, . . .
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
5 / 28
6. Free software is everywhere
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
6 / 28
7. Some numbers that say it all
IT Market
FOSS in France: 2.5 billions euros in 2012 (PAC)
RedHat: 1 billion dollars in 2012
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
7 / 28
8. Some numbers that say it all
IT Market
FOSS in France: 2.5 billions euros in 2012 (PAC)
RedHat: 1 billion dollars in 2012
OS Market
There Are Now 1.3 Million Android Device Activations Per Day
Eric Schmidt, September 2012
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
7 / 28
9. Some numbers that say it all
IT Market
FOSS in France: 2.5 billions euros in 2012 (PAC)
RedHat: 1 billion dollars in 2012
OS Market
There Are Now 1.3 Million Android Device Activations Per Day
Eric Schmidt, September 2012
Number of FOSS projects
2,091,753 FOSS projects
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Antepedia, May 2013
Free Software Challenges
November 2013 / OW2
7 / 28
10. Some numbers that say it all
IT Market
FOSS in France: 2.5 billions euros in 2012 (PAC)
RedHat: 1 billion dollars in 2012
OS Market
There Are Now 1.3 Million Android Device Activations Per Day
Eric Schmidt, September 2012
Number of FOSS projects
2,091,753 FOSS projects
Antepedia, May 2013
Across all layers
Linux, Apache, OpenStack, Hadoop, Postrgres, Alfresco, Zimbra,
LibreOffice, Talend, ...
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
7 / 28
11. Some numbers that say it all
IT Market
FOSS in France: 2.5 billions euros in 2012 (PAC)
RedHat: 1 billion dollars in 2012
OS Market
There Are Now 1.3 Million Android Device Activations Per Day
Eric Schmidt, September 2012
Number of FOSS projects
2,091,753 FOSS projects
Antepedia, May 2013
Across all layers
Linux, Apache, OpenStack, Hadoop, Postrgres, Alfresco, Zimbra,
LibreOffice, Talend, ...
FOSS is or will be part of your software project: get ready!
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
7 / 28
13. Challenges and opportunities
FOSS is radically changing the way software is conceived, developed,
maintained, deployed, tested, proven, marketed and sold.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
9 / 28
14. Challenges and opportunities
FOSS is radically changing the way software is conceived, developed,
maintained, deployed, tested, proven, marketed and sold.
This is a tidal change with disruptive power:
challenges build transparent software (re)using a multitude of
components available with their source code
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
9 / 28
15. Challenges and opportunities
FOSS is radically changing the way software is conceived, developed,
maintained, deployed, tested, proven, marketed and sold.
This is a tidal change with disruptive power:
challenges build transparent software (re)using a multitude of
components available with their source code
opportunities huge new market opening up for new tools and expertise
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
9 / 28
16. Challenges and opportunities
FOSS is radically changing the way software is conceived, developed,
maintained, deployed, tested, proven, marketed and sold.
This is a tidal change with disruptive power:
challenges build transparent software (re)using a multitude of
components available with their source code
opportunities huge new market opening up for new tools and expertise
Let’s see a few aspects of this revolution.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
9 / 28
18. FOSS as Lawyer’s nightmare
Proprietary software
There are as many licences as products, but the lawyer’s life is easy: the
basic answer to the question
May I copy/modify/adapt/distribute/etc. software X?
is almost always a very simple NO.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
11 / 28
19. FOSS as Lawyer’s nightmare
Proprietary software
There are as many licences as products, but the lawyer’s life is easy: the
basic answer to the question
May I copy/modify/adapt/distribute/etc. software X?
is almost always a very simple NO.
Free software
Lawyer’s life is more complex: there are only a few dozens licenses for
millions of FOSS components, but the basic answer to the question
May I do X with software Y?
is almost always It depends....
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
11 / 28
21. Bringing Tools In
You can do much more with FOSS than proprietary software, but not
everything: famous lawsuits concerning Busybox have shown that FOSS
licence obligations are real.
The gpl-violations.org project actively pursues violators.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
13 / 28
22. Bringing Tools In
You can do much more with FOSS than proprietary software, but not
everything: famous lawsuits concerning Busybox have shown that FOSS
licence obligations are real.
The gpl-violations.org project actively pursues violators.
This created a business opportunity for companies that sell tools to spot
legal bugs (sic!) in your software:
Blackduck (30% annual growth recently!)
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
13 / 28
23. Bringing Tools In
You can do much more with FOSS than proprietary software, but not
everything: famous lawsuits concerning Busybox have shown that FOSS
licence obligations are real.
The gpl-violations.org project actively pursues violators.
This created a business opportunity for companies that sell tools to spot
legal bugs (sic!) in your software:
Blackduck (30% annual growth recently!)
Palamida
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
13 / 28
24. Bringing Tools In
You can do much more with FOSS than proprietary software, but not
everything: famous lawsuits concerning Busybox have shown that FOSS
licence obligations are real.
The gpl-violations.org project actively pursues violators.
This created a business opportunity for companies that sell tools to spot
legal bugs (sic!) in your software:
Blackduck (30% annual growth recently!)
Palamida
There are also FOSS players:
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
13 / 28
25. Bringing Tools In
You can do much more with FOSS than proprietary software, but not
everything: famous lawsuits concerning Busybox have shown that FOSS
licence obligations are real.
The gpl-violations.org project actively pursues violators.
This created a business opportunity for companies that sell tools to spot
legal bugs (sic!) in your software:
Blackduck (30% annual growth recently!)
Palamida
There are also FOSS players:
FOSSology (HP, free software)
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
13 / 28
26. Bringing Tools In
You can do much more with FOSS than proprietary software, but not
everything: famous lawsuits concerning Busybox have shown that FOSS
licence obligations are real.
The gpl-violations.org project actively pursues violators.
This created a business opportunity for companies that sell tools to spot
legal bugs (sic!) in your software:
Blackduck (30% annual growth recently!)
Palamida
There are also FOSS players:
FOSSology (HP, free software)
Ninka (Daniel German, free software)
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
13 / 28
27. Bringing Tools In
You can do much more with FOSS than proprietary software, but not
everything: famous lawsuits concerning Busybox have shown that FOSS
licence obligations are real.
The gpl-violations.org project actively pursues violators.
This created a business opportunity for companies that sell tools to spot
legal bugs (sic!) in your software:
Blackduck (30% annual growth recently!)
Palamida
There are also FOSS players:
FOSSology (HP, free software)
Ninka (Daniel German, free software)
But...
This is just the tip of the iceberg...
and the least useful and/or interesting one!
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
13 / 28
29. Bringing Tools In, reloaded
Tools are needed to trace software origin, evolution, copies and forks; this
allows to
have a complete view of a full bill of materials (Antelink, Blackduck)
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
15 / 28
30. Bringing Tools In, reloaded
Tools are needed to trace software origin, evolution, copies and forks; this
allows to
have a complete view of a full bill of materials (Antelink, Blackduck)
identify code flows, and vulnerability duplications (Antelink, ...)
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
15 / 28
31. Bringing Tools In, reloaded
Tools are needed to trace software origin, evolution, copies and forks; this
allows to
have a complete view of a full bill of materials (Antelink, Blackduck)
identify code flows, and vulnerability duplications (Antelink, ...)
profile community contributions (Bitergia)
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
15 / 28
32. Bringing Tools In, reloaded
Tools are needed to trace software origin, evolution, copies and forks; this
allows to
have a complete view of a full bill of materials (Antelink, Blackduck)
identify code flows, and vulnerability duplications (Antelink, ...)
profile community contributions (Bitergia)
... you name it
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
15 / 28
33. Bringing Tools In, reloaded
Tools are needed to trace software origin, evolution, copies and forks; this
allows to
have a complete view of a full bill of materials (Antelink, Blackduck)
identify code flows, and vulnerability duplications (Antelink, ...)
profile community contributions (Bitergia)
... you name it
This is starting to show the way of the future...
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
15 / 28
34. Who contributes? how much? to what? ...
Answer
Bitergia’s dashboards www.bitergia.org
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
16 / 28
35. Who contributes? how much? to what? ...
Answer
Bitergia’s dashboards www.bitergia.org
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
16 / 28
36. Where does this code come from?
Answer
Antepedia Reporter’s dashboards www.antelink.com
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
17 / 28
37. Where does this code come from?
Answer
Antepedia Reporter’s dashboards www.antelink.com
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
17 / 28
38. Complex software in industry, ten years from now
Free Software is making software artefacts transparent:
no artificial barriers among architectural layers. . .
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
18 / 28
39. Complex software in industry, ten years from now
Free Software is making software artefacts transparent:
no artificial barriers among architectural layers. . .
theoretical possibility to follow the flow of computation from the more
abstract layers down to the moving bits in the hardware
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
18 / 28
40. Complex software in industry, ten years from now
Free Software is making software artefacts transparent:
no artificial barriers among architectural layers. . .
theoretical possibility to follow the flow of computation from the more
abstract layers down to the moving bits in the hardware
umprecedented levels of quality assurance and certification can be
expected
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
18 / 28
41. Complex software in industry, ten years from now
Free Software is making software artefacts transparent:
no artificial barriers among architectural layers. . .
theoretical possibility to follow the flow of computation from the more
abstract layers down to the moving bits in the hardware
umprecedented levels of quality assurance and certification can be
expected
this is not easy: we need new theory and new tools
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
18 / 28
42. Complex software in industry, ten years from now
Free Software is making software artefacts transparent:
no artificial barriers among architectural layers. . .
theoretical possibility to follow the flow of computation from the more
abstract layers down to the moving bits in the hardware
umprecedented levels of quality assurance and certification can be
expected
this is not easy: we need new theory and new tools
it will take time, effort and good cooperation between industrial
partners, higher education and research actors
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
18 / 28
43. Complex software in industry, ten years from now
Free Software is making software artefacts transparent:
no artificial barriers among architectural layers. . .
theoretical possibility to follow the flow of computation from the more
abstract layers down to the moving bits in the hardware
umprecedented levels of quality assurance and certification can be
expected
this is not easy: we need new theory and new tools
it will take time, effort and good cooperation between industrial
partners, higher education and research actors
Industry need to invest in Free Software fundamental research: the sooner,
the better.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
18 / 28
44. Complex software in industry, ten years from now
Free Software is making software artefacts transparent:
no artificial barriers among architectural layers. . .
theoretical possibility to follow the flow of computation from the more
abstract layers down to the moving bits in the hardware
umprecedented levels of quality assurance and certification can be
expected
this is not easy: we need new theory and new tools
it will take time, effort and good cooperation between industrial
partners, higher education and research actors
Industry need to invest in Free Software fundamental research: the sooner,
the better.
This is the kind of research we promote at IRILL: let’s see a concrete
example of what can be done.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
18 / 28
46. Coccinelle: flexible abstractions to master collateral
evolutions
Thanks to Gilles Muller and Julia Lawall for the following slides. See
http://coccinelle.lip6.fr/ for more information.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
20 / 28
47. Coccinelle: flexible abstractions to master collateral
evolutions
Thanks to Gilles Muller and Julia Lawall for the following slides. See
http://coccinelle.lip6.fr/ for more information.
The collateral evolution problem:
Library functions change.
Client code must be adapted.
– Change a function name, add an argument, etc.
Linux context:
– Many libraries: usb, net, etc.
– Very many clients, including outside the Linux source tree.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
20 / 28
48. An example
Evolution: A new function: kzalloc
=⇒ Collateral evolution: Merge kmalloc and memset into kzalloc
fh = kmalloc(sizeof(struct zoran fh), GFP_KERNEL);
if (!fh) {
dprintk(1,
KERN_ERR
"%s: zoran open(): allocation of zoran fh failedn",
ZR_DEVNAME(zr));
return -ENOMEM;
}
memset(fh, 0, sizeof(struct zoran fh));
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
21 / 28
49. An example
Evolution: A new function: kzalloc
=⇒ Collateral evolution: Merge kmalloc and memset into kzalloc
fh = kzalloc(sizeof(struct zoran fh), GFP_KERNEL);
if (!fh) {
dprintk(1,
KERN_ERR
"%s: zoran open(): allocation of zoran fh failedn",
ZR_DEVNAME(zr));
return -ENOMEM;
}
memset(fh, 0, sizeof(struct zoran fh));
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
22 / 28
50. A simple semantic patch
Abstract away the details of a patch
@@
expression x;
expression E1,E2;
@@
- x = kmalloc(E1,E2);
+ x = kzalloc(E1,E2);
...
- memset(x, 0, E1);
A single snippet ... updates 355/564 files!
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
23 / 28
51. Practical results
Collateral evolutions
Semantic patches for over 60 collateral evolutions.
Applied to over 5800 Linux files from various versions, with a success
rate of 100% on 93% of the files.
Bug finding
Generic bug types:
– Null dereference, initialization of unused variables, etc.
Bugs in the use of Linux APIs:
– Incoherent error checking, memory leaks, etc.
Over 450 patches created using Coccinelle accepted into Linux
Starting to be used by other Linux developers
Probable bugs found in gcc, postgresql, vim, amsn, pidgin, mplayer
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
24 / 28
53. IRILL in the big picture
IRILL (Innovation et Recherche en Informatique sur le Logiciel
Libre):
a center to host researchers, teachers, engineers, and free
software developers to work together on the new scientific,
technological and educational challenges of Free Software
See more info on http://www.irill.org.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
26 / 28
54. IRILL in the big picture
IRILL (Innovation et Recherche en Informatique sur le Logiciel
Libre):
a center to host researchers, teachers, engineers, and free
software developers to work together on the new scientific,
technological and educational challenges of Free Software
See more info on http://www.irill.org.
IRILL has an industry partnership program: join today!
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
26 / 28
55. Conclusions
Free Software is here to stay, and is disrupting the traditional organization
of the software industry.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
27 / 28
56. Conclusions
Free Software is here to stay, and is disrupting the traditional organization
of the software industry.
Developing, maintaining, marketing, integrating, and qualifying software
built with or in the FOSS world is the source of new challenges that are
new opportunities for disruptive innovation.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
27 / 28
57. Conclusions
Free Software is here to stay, and is disrupting the traditional organization
of the software industry.
Developing, maintaining, marketing, integrating, and qualifying software
built with or in the FOSS world is the source of new challenges that are
new opportunities for disruptive innovation.
The time has come to stop asking whether FOSS is relevant... and start
getting ready to use it properly.
Roberto Di Cosmo (Paris Diderot / Irill / INRIA)
Free Software Challenges
November 2013 / OW2
27 / 28