An open source approach to software as a service at the European Commission.

1. The OpenEuropa Initiative
An open source approach to software as a service
at the European Commission
Antonio De Marco
nuvole.org - info@nuvole.org - @nuvoleweb

2. About me
Hello, I’m Antonio.
● Director and co-founder of Nuvole
● From 2015 consulting part-time for Nuvole at the European Commission
● From 2018 co-founder and team leader of the OpenEuropa Initiative at
the Directorate-General for Informatics (DIGIT)
Disclaimer: the use case we are presenting today is built and maintained by a
large team of consultants, working for different companies.
2

3. About Nuvole
● From 2010, a proud Drupal company
● Complex Drupal projects
Interfacing with other systems or featuring complex front-end requirements
● Working with NGOs and public institutions
An international team rooted in NGOs and comfortable working with large NGOs and public
institutions (World Food Programme, European Commission, OECD, World Bank)
https://nuvole.org
3

4. Nuvole in the Drupal Community
● Long time Drupal contributors
Our open-source Drupal modules were downloaded more than 1 million times
● Drupal core contribution
Co-leading the official Drupal 8 Configuration Management Initiative
● Drupal events
Invited as presenters at several annual EU/US Drupal conferences (DrupalCon)
● Drupal trainers
10+ times official DrupalCon trainers + large organizations (HP, European Commission, etc.)
4

5. The European Union web presence
5

6. The European Union web presence
● 2 main institutional sites:
○ The European Commission: https://ec.europa.eu
○ The European Union: https://europa.eu
● 40 European agencies, such as:
○ EFSA, the European Food Safety Authority
○ Frontex, the European Border and Coast Guard Agency
● 27 Commission's Representations sites, such as
○ Italy: https://italy.representation.ec.europa.eu
○ Belgium: https://belgium.representation.ec.europa.eu
● Hundreds more: the Europass site, the COVID-19 response site, etc.
6

7. 7

8. 8

9. 9

10. 10

11. Managing institutional content is hard
● Multilingualism, supporting the 24 official languages in the EU
● Integration with the EU internal translation services
● Advanced content moderation workflow
● Scheduling content publication for sensitive news
● Harmonization of content categorization via shared taxonomies
● Centralized authentication and authorization system
● Integration with ECAS, the European Single Sign-On Service.
● Integration with the European Union Webtools widgets
● etc...
11

12. Drupal at the European Commission
12

13. Drupal at the European Commission
● 2012: a pilot project, the Multisite Platform, based on Drupal 7:
○ A common web management infrastructure
○ Rationalise the web presence of the EC
○ A coherent corporate image
○ Cost sharing amongst the different Directorate Generals
● 2014: the Europa platform, based on Drupal 7:
○ A Drupal 7 distribution built on the Multisite Platform experience
○ Aimed at migrating away from the then outdated web publishing platform
● 2015-2017: Migration of hundreds of websites to the Europa platform
○ Including the two main institutional sites
13

14. The Europa platform: a monolithic approach
● A SaaS built as a Drupal 7 distribution
● Organised in Drupal features, using the Features module
● Possibility for site owners to enable and disable features from the UI
● A first, genuine, attempt to business requirement orchestrations
● Allowed for extra customizations, on the final client’s expense
14

15. The Europa platform: a monolithic approach
Drupal 7 Core
Installation Profile
Feature A Feature B Feature C Feature D
Site custom code
15

16. The reality of a monolithic approach
● Large code base, difficult to maintain
● Long tests runs, not 100% covered
● New features might break existing sites
● Hard to involve people to contribute
● High deployment time
● Used more as a framework than a service
16

17. Other drawbacks
● A single codebase with tens of developers contributing to it
● Clients were hiring their own teams to manage the site
● 100 sites / 100 ways of implementing the same thing
○ Clients could extend it by hiring their own development team in house
○ Custom functionalities would require significant overrides
○ Development would then happen on a separate repository
● 100 ways of implementing the same thing / 1000 developers
○ Hard to maintain technical governance
○ Hard to monitor security
○ High cost of maintainability and quality assurance processes
17

18. The OpenEuropa Initiative
18

19. Drupal 8: an opportunity
In 2018 the migration of the Europa platform to Drupal 8 presented an
opportunity for a paradigm shift by leveraging:
● The maturity of a proper package management system
● The adoption of semantic versioning
● The raise of containerised applications
19

20. The OpenEuropa Initiative
OpenEuropa is a European Commission initiative aimed at strengthening the
adoption of open source tools and practices, particularly focused to:
● Build and maintain loosely-coupled, reusable software components,
● Build fully-fledged solutions for the European Institutions
● Establish reusable technical governance guidelines
● Contribute back to upstream open source projects, such as Drupal
https://github.com/openeuropa/documentation
20

21. 21

22. A “component-based” approach (1/2)
Each component (e.g. a Drupal module, or a PHP library):
● Has its own repository
● Enjoys an independent release cycle
● Is tested separately
● Has a dedicated continuous integration pipeline
● Is semantically versioned (x.y.z)
● Describes its own dependencies
22

23. A “component-based” approach (2/2)
Each component has everything that a developer needs to kickstart
development, bundled right in its repository.
For example, in case of a Drupal module:
● Running composer install builds a fully functional site, right within
the component root directory, allowing to kickstart development
● Running docker-compose up -d spins off a fully functional
environment, with all needed services (web server, database, etc.)
● Behat and functional PHPUnit tests are run against the target site
● Context-switching for developers is reduced to the minimum
23

24. Project setup
Fully functional
development site
Continuous Integration
and deployment 24

25. Advantages
A component-based approach:
● Leverages modern dependency package manager, like Composer
● Leverages Drupal's modularity as a point of strength
● Makes it easier to write and maintain tests
● Allows for a shorter “time-to-market” per component
● Facilitates intra-team contributions
● Facilitates external contributions
● Facilitates the adoption of open source tools and practices
25

26. Breaking up the monolith
Core
Profile
A B Site specific
Site A
Core
Profile
C Site specific
Site B
A
1.1.0
A
1.1.1
A
2.0.0
C
0.0.1
C
0.0.2
C
0.1.0
B
0.1.0
B
0.2.0
B
0.2.1
B
1.0.0
B
1.0.1
C
1.0.0
Time
26

27. 27

28. 28

29. 29

30. 30

31. 31

32. Building and maintaining OpenEuropa
32

33. An example: the EU’s visual identity
The Europa Component Library (ECL) is a comprehensive style guide
containing the design elements and visual standards to harmonize the
European Union web presence.
https://ec.europa.eu/component-library
33

34. EC EU
34

35. Aim: provide an ECL Drupal theme
The OpenEuropa Theme component must:
● Allow Drupal sites to feature an ECL look and feel
● Allow a site owner to choose between the EC and the EU branding
● Reuse the components in their native Twig implementation
35

36. Theme architecture
Three main components working together:
● ECL Twig Loader: loads ECL components using a custom Twig namespace
● OpenEuropa Theme: themes Drupal frontend using ECL components and
exposes them as UI Patterns
● Composer Artifacts: allows projects to use a fully built theme artifact
which contains ECL templates and compiled frontend assets
Other than “the usuals”: the Task Runner and the Code Review component
36

37. Europa
Component Library
ECL
Twig Loader
Provides reusable components
for an harmonized look and feel
Imports ECL templates in any PHP
application using Twig
{% include '@ecl/...' with {
'variant' : 'default' ,
'title': title,
'detail' : detail,
} %}
{% include '@ecl/...' with {
'Primary_image' : image,
'title': title,
'detail' : detail,
} %}
{% include '@ecl/...' with {
'secondary_image' : image,
'title': title,
'detail' : detail,
} %}
OpenEuropa
Theme
Drupal theme providing ECL
components integration
UI
Patterns
Field formatters
View displays
Entity view modes
37

38. Using the theme’s artifact
ECL ECL Loader Theme
Site builders
Site developers
Core components
38

39. Building a SaaS on top of OpenEuropa
39

40. A new SaaS for the EU web presence
OpenEuropa allows for a streamlined way of building and maintaining SaaS
platforms. Since 2018 we have been building a SaaS that is powering the next
generation of the European Union web presence.
The SaaS platform is:
● A private Drupal distribution
● Built using OpenEuropa components
● Tweaked by custom modules and themes, shipped within the distribution
40

41. Stitching it all together
Public
components
Private
distribution
Theme
Corporate
Content
Multilingual
support
Editorial
Workflow
OpenEuropa Modules
Translation
client
Composer
artifacts
OpenEuropa Libraries
Search API
Paragraphs
Contributed modules
Drupal profile
Custom module 1 Custom module 2 Custom theme
41

42. An economy of scale
A few non-functional requirements were clear since the beginning:
● The new SaaS platform is built to serve hundreds of sites
● Updates need to be rolled out all at the same time
● Each site needs to be built and managed by the site owners, without
employing any dedicated developer
● Eventually, a site owner can decide to “break free” from the flock, and
start its own development, while still keeping the possibility of upgrading
to newest SaaS versions
42

43. How we did it
● We have one repository called the master site:
○ A Composer-based Drupal site repository
○ It requires the SaaS Drupal profile
● Each other site instance gets its very own repository
● Each site instance repository is a git-fork of the master site
● A new release of the master site triggers a waterfall release pipeline,
which releases all the other site instances
43

44. Release one, deploy hundreds
44
Master site
Release manager
tags a release
Site A
Site B
Site C
Site D
Site A
Site B
Site C
PROD
PROD
Deployment pipeline
Test pipeline

45. Where we stand now
45

46. OpenEuropa in numbers
● More than 70 components, including Drupal modules, PHP libraries,
Docker images, development tools, templates, etc.
● More than 100 developers, divided in 20 teams, contribute daily to the
initiative
● Around 100 sites on the SaaS platform, out of which 50+ are currently
live, in production
● Tens of custom (non-SaaS) sites have been built using the OpenEuropa
components
46

47. Thank You
nuvole.org - info@nuvole.org