Software Architecture Erosion, Modernization and what you can do about it

1. Stop the Software
Architecture Erosion
Bernhard Merkle Frederic Madiot
R&D Software Products Manager
SICK AG, Germany OBEO, France
Contact us on linkedin.com or xing.com Page: 1

2. Agenda
– Software Architecture
– Detect Architectural Erosion
– Eclipse + Open Source Projects
– Model-Driven Reverse-Engineering
Page: 2

3. Architecture, Erosion
Page: 6

4. Software-Architecture: Definitions
IEEE 1471-2000:
The fundamental organization of a system,
embodied in its components,
their relationship to each other and the
environment,
and the principles governing its design and
evolution.
Page: 7

5. Architectural Erosion
“Sometimes the developers manage to
maintain this purity of design through the
initial development and into the first release.
More often something goes wrong.
The software starts to rot like a
piece of bad meat”.
Uncle Bob: “Agile Software Development” Page: 9

6. Architectural Erosion
Why should we care ?
– In (lots of) Projects, Architecture decay happens
– We are not alone, as we‘ve some good representatives… ;-)
There is/was a plan Page: 10

7. X-raying Software…
Page: 11

8. Findbugs: the first years
0.7.2 0.8.6
(03/2004) (10/2004)
Page: 12

9. Findbugs: first erosion
0.8.7 (05/2005)
Page: 13

10. Tangle increase…
0.8.8 1.0.0
(05/2005) (06/2006) Page: 14

11. Tangle increase…
1.3.0
Page: 15
(07/2007)

12. ONE BIG Tangle…
1.3.8 (03/2009)
Page: 16

13. Page: 17

14. Modeling Subsystems:
Page: 20

15. Fixing Architectural Violations
Page: 21

16. Fixing Architectural Violations
Page: 22

17. Fixing Architectural Violations
Page: 23

18. Fixing Architectural Violations
Page: 24

19. Fixing Architectural Violations
Page: 25

20. Fixing Architectural Violations
Page: 26

21. Fixing Architectural Violations
Page: 27

22. Tools for Architecture-Analysis
– SotoArc SonarJ
– Bauhaus Lattix
– Structure101 Klocwork, Coverity
– http://code.google.com/p/architecturerules/
– ODASA, CodeCrawler, Codecity, SeeSoft, XRadar, XDepend,
SonarSource, Kalistick, Sqoring, …
– Eclipse-Based: MoDisco (+Modernization of Applications)
http://se-radio.net/
episode-115-architecture-analysis Page: 28

23. Basic Approaches (Dependencies)
– makedepend, jdepend
– Eclipse (Java Build Path)
– OSGI (Dependencies)
Page: 29

24. Basic Approaches
PDE Dependency Visualization
Page: 30

25. Missing in basic approaches:
Architecture Analysis (Deviation)
Drill-Down + Aggregation
Displaying results
Monitoring changes, trends
Rating of Architecture
Requirements for Architectural Analysis Tools
Page: 31

26. Architecture Analysis (deviation)
Requirements
Architecture- Should-
Design Architecture
Comparison “Diff-” Actions
Architecture
Extraction Is- •Violations
•Conformance
Architecture
Existing Code
Page: 32

27. Displaying results:
Page: 33

28. Architecture: critical for OS-projects
How to augment How to replace
the development key people ?
and support load ?
Open-Source
Project
Marketplace
How to maintain
How to develop confidence with users?
a market ?
Page: 34

29. Openess limits Erosion
Developers expose their reputation
Names are associated to the architecture
Community can provide feedback
Warnings, Recommendations, ...
Page: 35

30. Risks of Erosion in FOSS
Contributors from several organizations
Different cultures, processes, tools, …
Lower pressure from management
Indirect Business
Hazardous funding
Difficulties to calculate costs and benefits
Page: 36

31. Eclipse: Architectural Analysis
Page: 37

32. Eclipse: 10 Years Legacy System
Eclipse Helios = 33M lines of code
Page: 38

33. Eclipse Architecture
Page: 39

34. Eclipse Architecture
Page: 40

35. Eclipse Architecture
Page: 41

36. E3.4: Plattf:Ant JDT:*
Page: 42

37. E3.4: Plattf:Ant JDT:UI
Page: 43

38. Plattform: CVS Workb (internal)
Page: 44

39. Team-UI UI-workbench (internal)
https://bugs.eclipse.org/bugs/show_bug.cgi?id=90582
Page: 45

40. Antipatterns
Page: 46

41. Dependent BaseClass
– Type:
• Design Problem
– Problem:
• one of more Methods shall implement different
behavior, depending on the type, passed in
– Context:
• make “extensible” systems, frameworks
– Forces:
• Programming languages offer, instanceof/typeid funcs.
– Antipattern:
• Methods of the baseclass, depend on derived classes, e.g. accessing
their members, doing switch/case depending on type information
Page: 47

42. AntiPatterns / Bad Smells:
Metrics/1000 Classes Eclipse JDK
Dependent Baseclass: 1 16
Multiple Interface Inher. 4 18
Abstractable Methods 80 60
Abstractable Attributes 45 170
Unused Classes 50 150
Unused Methods 950 2500
Unused Attributes 30 20
Page: 48

43. Code
Code Clones
Clones
Page: 49

44. Code Clones
– identical Files
• E2.0 JDT,CDT
– jdtdebuginternaluidialogfieldsListDialogField.java
– cdtdebuginternaluidialogfieldsListDialogField.java
• E3.4 CDT: identical packages
– cdtdebuginternaluidialogfields,
cdtdebugmiinternaluidialogfields
– variation of former identical Files
• E3.4 JDT,CDT
– jdtdebuginternaluidialogfieldsListDialogField.java
– cdtdebuginternaluidialogfieldsListDialogField.java
Page: 50

45. Rating Eclipse Architecture
Minor Erosions, Code Duplication
– large codebase,
– OSGI helps a lot
– API Police,
– Guidelines
– upcoming PDE tools
– Processes and Tools
Page: 51

46. Architecture Council Recommend.
•Architectural Recommendations •Community Practices
• Be asynchronous • Engage your user community
• Think API
• Long-running operations should be
cancelable •Software Development Practices
• Separate policy and mechanism • Unit Test, Unit Test, Unit Test
• Keep simple things simple • Continuous Integration
• Create Unittests early • Integrate in small steps
• Minimize plug-in dependencies
• Be aware of the deployment context
• Package coherence
• Putting only related things together
http://wiki.eclipse.org/Architecture_Council/Top_Ten_Project_Development_Practices
http://wiki.eclipse.org/Architecture_Council/Top_Ten_Recommendations
Page: 52

47. Yearly Simultaneous Releases Rules
Projects should leverage only published APIs of
dependencies
Version number ends with « qualifier »
Source code must use ICU4J classes
The project must contain an « about.html » file
Packages name should start with the plug-in ID
Plug-in must not contain JARs files
Plug-in should contain only one
« message.properties » and « Message.java » files
Etc…
http://www.eclipse.org/indigo/planning/EclipseSimultaneousRelease.php
Page: 53

48. New Eclipse projects …
driving to new architectures
Components that are more reusable + customizable
Service-oriented programming model
GUI represented by a model and configurable with CSS
Enabling Javascript app to be executed by Eclipse runtime
Framework to build SWT app declarativeley
Ajax-enabled web applications by using a subset of RCP APIs
Eclipse development model
Plug-ins & Eclipse workbench extension points
Widget toolkit using SWT API
Browser-based tools to develop for the web, in the web
Client: loosely coupled components written in JavaScript
Server: services exposed via REST-oriented HTTP APIs
Page: 54

49. Manage architectural transition?
?
Architecture A Architecture B
Page: 55

50. Evolution to e4
Backward Compatibility Layer
• Eclipse 3.x plug-ins run on e4
Challenges
• Not clean APIs usage?
Refactoring
• Using e4 development model?
Migration
• Single sourcing?
Forward Compat. Layer
(Session: Singlesourcing for Eclipse 4.x and Eclipse 3.x)
Page: 56

51. Evolution to RAP
Single sourcing with RAP is not automatic
• API differences
• Missing extension points
• Application startup and Activator scope
• API Differences
• Field validation Refactoring
• SWT ressources
• Singletons and Scopes
• Jobs and background threads
• Internationalization and localization
(http://eclipsesource.com/fileadmin/doc/2009_product/single_sourcing_guide.pdf)
Page: 57

52. How to modernize
existing Eclipse plug-ins?
Page: 58

53. Architectural Modernization Process
Transformation Reference
Tooling Tests
Modernization Transformation
Strategy & Integration
Non-Regression
Audit
Tests
Legacy Modernized
Software Software
System System
Page: 59

54. Anatomy of an Eclipse 3.x Plug-in
folders
files
.project
.classpath
!
ity
MANIFEST.MF
e ne
g
Eclipse Plug-in build.properties
te ro
He
plugin.xml
plugin.properties
Source code
Page: 60

55. Model-Driven
Reverse-Engineering
of Eclipse plug-ins
Page: 61

56. MoDisco
use Models to represent and manipulate
artifacts of existing systems
Discover ExistingUnderstand Transform
Software System
Software artifacts :
- source code
New
Models Viewpoints
- configuration files Software System
- tests
- database
-…
http://www.eclipse.org/MoDisco/
Page: 62

57. MoDisco Architecture
Supported Technologies
Java JSP XML EclipsePlugin
Metamodel Metamodel Metamodel
Discoverer Discoverer Discoverer Metamodel
Generator Generator Generator Discoverer
Transfo. to KDM
Discovery Model Customization OMG/ADM
Manager Browser & Extensibility Standards
Plug and orchestrate Navigation Definition of Pivot
transformations through specific Metamodels
complex models Viewpoints (SMM & KDM)
Infrastructure
Eclipse Modeling projects
Page: 63

58. Using EMF to describe a Plug-in
folders Project’s structure
(KDMSource)
files
.project (XML)
.project
.classpath (XML)
.classpath
it y!
manifest e
MANIFEST.MF
en plugin
build.properties
og (eclipseplugin)
om (XML)
Eclipse Plug-in build.properties (KDMCore)
Hextensions
plugin.xml
plugin.properties
plugin.properties (KDMCore)
Java source code
Source code (Java)
Page: 64

59. Using EMF to describe a Plug-in
Page: 65

60. What can you do with
the EMF model
of a plug-in ?
Page: 66

61. Leverage Eclipse Modeling components
Inspect
Inspect
(MoDisco Browser, EEF)
(MoDisco Browser, EEF)
Query
Query
(EMF Query)
(EMF Query)
Vizualize
Vizualize
(GMF, Graphiti)
(GMF, Graphiti)
EMF Model
of an Eclipse plug-in Compare
Compare
(EMF Compare)
(EMF Compare)
Transform
Transform
(ATL, QVTo)
(ATL, QVTo)
Generate code
Generate code
(Acceleo, Xpand, Jet)
(Acceleo, Xpand, Jet) Page: 67

62. Architectural Modernization Process
Transformation
Transformation Reference
Reference
Tooling
Tooling Tests
Tests
Modernization
Modernization Transformation
Transformation
Strategy
Strategy & Integration
& Integration
Non-Regression
Non-Regression
Audit
Tests
Tests
• Quality Analysis
• Volumetry
• Cartography
Legacy
Legacy Modernized
Modernized
Software
Software Software
Software
System
System System
System
Page: 68

63. Validation
Page: 69

64. Architecture Visualization
GMF diagram
created with ObeoDesigner
• purple: EMF Model
• green: UI
Page: 70

65. Architectural Modernization Process
Transformation Reference
Reference
Tooling Tests
Tests
Modernization
Modernization Transformation
Transformation
Strategy
Strategy & Integration
& Integration
Non-Regression
Non-Regression
Audit
Audit
• Parsing Tests
Tests
• Transformation rules
• Re-generation
Legacy
Legacy Modernized
Modernized
Software
Software Software
Software
System
System System
System
Page: 71

66. Refactoring & Migration
Model Model
Existing of the existing Transformed
of the migrated
Plug-in Plug-in Plug-in
Plug-in
Transformation Rules
What to change + How to change
• Renaming
• Changing code structure
(inheritance, attributes, methods, etc)
• Replacing method calls
• Changing instructions order
• Etc
Page: 72

67. Architectural Modernization Process
Transformation
Transformation Reference
Tooling
Tooling Tests
Modernization
Modernization Transformation
Transformation
Strategy
Strategy & Integration
& Integration
• Tests coverage analysis Non-Regression
Non-Regression
Audit
Audit Tests
Tests
Legacy
Legacy Modernized
Modernized
Software
Software Software
Software
System
System System
System
Page: 73

68. Test coverage analysis
Page: 74

69. Architectural Modernization Process
Transformation
Transformation Reference
Reference
Tooling
Tooling Tests
Tests
Modernization
Modernization Transformation
Strategy
Strategy & Integration
Non-Regression
Non-Regression
Audit
Audit Tests
Tests
• Automatic transformation
• Manual transformation
• Build
Legacy
Legacy Modernized
Modernized
Software
Software Software
Software
System
System System
System
Page: 75

70. Build
IDM++ Research Project (ANR) -> Sept 2011
Plug-ins to
build
Model of
plugins to build
Team information
(CVS, SVN, etc) constraints solver
B3 Model
(build configuration)
Model of
Update sites update sites content
(p2)
Build Strategies
Page: 76

71. Summary
Stop the Software Architecture Erosion ?
Analysis is Required
– Evaluate the architectural situation
– Take the right decisions at the right time
Modernization is Required
– Correct erosion consequences
– Go with architectural Evolutions
=> Continuous Analysis + Modernization
Page: 77

72. Thank you !
Page: 78