This is a slide deck for my ISR CMU Software Engineering PhD practicum (approved in May 2012). Full text: http://www.cs.cmu.edu/%7Eiruchkin/docs/ruchkin12-building-software-in-house.pdf
Abstract:
As domain-specific software becomes more available, businesses face a dilemma: whether to acquire commercial off-the-shelf (COTS) enterprise management systems or to build them in-house. Companies choosing to create a product internally are often rewarded with flexibility and control over their development process and its results. However, when expanding, they can outgrow their ability to support the developed software.
Working as a programmer at a medium-sized logistics company, Si-Trans, in 2010, I witnessed the long-term implications of an initial decision to build an information system in-house. While this decision was appropriate in 1997 because COTS alternatives were scarce and inapplicable, it created a favorable climate for inconsistent, ad hoc management practices within the entire company, in particular, software creation and maintenance. These practices ultimately contributed to Si-Trans' inability to see an opportunity in the early 2000s when it was feasible and advantageous to adopt a COTS-based solution. This solution would have scaled better for the company's growth and would have helped avoid an outstanding technical debt in the old system. By the end of 2011, Si-Trans finally considered the acquisition of an off-the-shelf information system, after having suffered substantial financial losses from the protracted in-house development.
4. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Practicum Context
Issue: to buy software or to build it?
Can build completely in-house.
Can buy commercial-oļ¬-the-shelf (COTS) software.
Can adopt open source software.
Can develop a hybrid solution.
Scope: non-software medium-sized business.
Common wisdom: (i) higher control and ļ¬exibility, but limited
scaling when building; (ii) higher initial investment, but lower
cost-of-ownership and higher reliability when buying COTS.
My experience: started building ā missed the point to turn to
COTS ā suļ¬ered losses ā transitioned to COTS.
3 / 23
12. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Requirements
Functional needs for a company-wide information system:
Coordination between units.
Automation of processes (esp. report generation).
Data storage, analysis, and prediction.
Qualities needed: āgood enoughā security and performance.
State of aļ¬airs in 1996:
Unstable legislation: requirements shift monthly.
Unstable market: high turnover of companies.
No suitable COTS solutions available in Russian market.
Ad hoc management: situational decisions, no long-term
commitments.
8 / 23
14. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Build decision
In 1996, a decision (implicit/explicit?) to build an in-house
system was made. A software development team of 1 person
was formed.
Interpretation
The decision was appropriate to circumstances. Even if COTS
alternatives had existed, it probably would not have been worth
investing into one because of overall instability of the
environment.
9 / 23
16. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Architecture
Architecture of the systemāthick client-based1. A single
database in Moscow; thick clients in other locations.
Interpretation
This architecture was suļ¬cient for a number of years because:
Few employees used it =ā performance needs met.
Implementation size was small =ā no stability issues.
Required functionality was trivial =ā easy to modify.
1
Thick clientāa GUI application with ingrained business logic.
10 / 23
19. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
COTS appears
In 2002ā2003, a signiļ¬cantly advanced version of COTS
product for enterprise management, 1C, was released. It
featured:
Ready-to-deploy components for personnel management,
accounting and ļ¬nance, wares management, contract
management, and report generation.
Diļ¬erent options for client-side applications: both thick
and web-based.
More scalable architecture: decoupled data storage and
client request processing.
A platform (language, API, development toolkit) for
developing new components.
11 / 23
25. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Signs of outgrowing
Si-Trans started facing a number of issues with the homegrown
system by 2006:
Source code diļ¬cult to manage: no investment in
readability, business logic growing complicated, increasing
size of source code.
Performance not suļ¬cient: number of users grows, hence
the database becomes a bottleneck.
Poor stability: bugs do not get suļ¬cient treatment as new
feature requests land on developers.
Availability is limited: not all sites have a powerful enough
internet connection to run the system.
Update delivery is messy: no control over usersā
applications.
14 / 23
26. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
My role and context
I joined as a developer in 2010. Wide range of activities:
Fixing, improving, and extending the existing information
system.
Design and implementation of a prototype for a new
system (3-tier architecture).
State of aļ¬airs in 2010:
Stable market of clients: the set of Si-Transā clients was
well-known and changed predictably.
Stable legislation: all forms to submit to government
bodies did not change much over time.
The management style of Si-Trans persisted: it was still ad
hoc, experimentation-based.
15 / 23
28. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Technical debt
By 2010, the information system has accumulated serious
technical debt. The development team has grown to 5 people,
but was stuck struggling with the outgrown system.
Interpretation
It was inconsistent management that was accountable for:
Rapid accumulation of technical debt.
An overlooked opportunity to adopt COTS.
Their management style stemmed from absolute control over
development.
16 / 23
29. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Example of inconsistent practices 1
Top management requests an immediate update that all
late transportations are checked and ļ¬agged every day.
The change is implemented as a complicated condition in
clients (not a database-level operation).
The day after management decides to abandon the idea.
Several weeks pass, and everyone forgets about the
request. The change is buried in other code.
Then, management decides to change the condition of a
transportation being late. Now, it is implemented as a
database trigger.
Result: intermittent bugs and staļ¬-hours spent on ļ¬nding
out the cause.
17 / 23
30. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Example of inconsistent practices 2
A top manager has a very speciļ¬c GUI in mind and wants
it implemented as soon as possible.
Several weeks after the GUI (and a corresponding data
model) has been implemented and deployed, it turns out
that the change is conceptually inconsistent with other
parts of the system.
E.g. it operates a concept of ācityā instead of a concept
of āoļ¬ceā.
Result: it takes a lot of time to spot all code that has
been built on the incorrect assumption. Being under
constant time pressure, programmers leave bugs behind.
18 / 23
31. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Conclusion of my experience
Interpretation
Chaotic management increased the cost of ownership and
slowed down the development progress, mainly by constantly
serving contradicting feature requests and not leaving enough
time for up-front design.
The shortsighted attitude of āwe have programmers, so they
will do whatever we wantā resulted in an immense technical
debt and nearly stalled the development.
19 / 23
35. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Outcome
In January 2012, Si-Trans top management made a decision to
transition to COTS. The development team (except the lead)
was ļ¬red. Si-Trans and 1C started an analysis of how to tailor
1C products to Si-Transā business model.
Interpretation
The decision of turning to COTS was based on the ļ¬nancial
losses on development that became obvious to management.
These losses could have been avoided by acquiring the 1C
enterprise management system as it became available. Too
much control over the inherently ļ¬exible development process
from the incompetent management costed Si-Trans a lot.
20 / 23
36. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Lessons 1/2
Inconsistent, chaotic management speeds up the
accumulation of technical debt as it makes requirements
drift. Under such circumstances, building products
in-house is less attractive in the long-term perspective.
Excessive time pressure leads to a higher rate of technical
debt accumulation because it forces sub-optimal decisions.
One particular example is well-known: ļ¬x as many bugs as
possible before implementing new features.
āDegree of experimentationā in management should
match environment and organizationās size. While
appropriate in 1996, ad hoc and ļ¬exible management in
2010 caused stagnation in development.
21 / 23
37. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Lessons 2/2
The issue of buying COTS or building software in-house
should not be treated as a one-time issue. Constant
re-evaluation of ļ¬tness for buying or building is needed.
The withdrawal of control over software development,
introduced by acquiring COTS, may act as a forcing
function to invest more into stabilizing requirements.
While usually considered negative, lack of control may
help in case of abusive management.
22 / 23
39. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
Carina Alves and Anthony Finkelstein.
Challenges in COTS decision-making: a goal-driven
requirements engineering perspective.
In Proceedings of the 14th international conference on
Software engineering and knowledge engineering, SEKE
ā02, page 789794, New York, NY, USA, 2002. ACM.
Xavier Franch and Marco Torchiano.
Towards a reference framework for COTS-based
development: a proposal.
In Proceedings of the second international workshop on
Models and processes for the evaluation of oļ¬-the-shelf
23 / 23
40. Building
Software
In-House
Ivan Ruchkin
Context
Initial decision
COTS
emerged
Outgrowth
Outcome
References
components, MPEC ā05, page 14, New York, NY, USA,
2005. ACM.
B. Craig Meyers and Patricia Oberndorf.
Managing Software Acquisition: Open Systems and COTS
Products.
Addison-Wesley Professional, 1 edition, July 2001.
Abdallah Mohamed, Guenther Ruhe, and Armin Eberlein.
COTS selection: Past, present, and future.
In Engineering of Computer-Based Systems, 2007. ECBS
ā07. 14th Annual IEEE International Conference and
Workshops on the, pages 103ā114. IEEE, March 2007.
23 / 23