Presented at Web Unleashed on September 16-17, 2015 in Toronto, Canada
More info at www.fitc.ca/webu
Declare Bankruptcy on Technical Debt!
with Kacey Coughlin
OVERVIEW
No one is paid just to deal with technical debt in an organization, and few people like doing it. Tech debt comes from a variety of places, and can pile up quickly if a developer is not careful. Tech debt often makes seemingly simple or small tasks take much longer, cost more money, and generally complicate projects and tech stacks needlessly. But if a dev knows what to watch out for, the entire team can spend less time addressing it.
Good documentation, thorough planning ahead of time, clear ownership, relentless testing, and open source solutions are all ways of reducing technical debt. If you want to get ahead of tech debt, these should be a part of your org’s daily routine.
OBJECTIVE
Help identify and reduce technical debt in your organization.
TARGET AUDIENCE
Engineers, IT professionals, developers with 2+ years industry experience.
ASSUMED AUDIENCE KNOWLEDGE
Best coding practices, version control, open source software, what a tech stack is, experience working on a development team.
FIVE THINGS AUDIENCE MEMBERS WILL LEARN
What is technical debt, and how to identify it
Where tech debt coms from
How to minimize tech debt
Best practices to not add tech debt
Best documentation practices
3. In this talk:
• What is Technical Debt?
• Where does it come from?
• Why should you care?
• How do we (try) to stop lessen it?
• Documentation?
4.
5. 1. A metaphor created to help explain issues with
non-technical people
6. 2. Refers to the eventual consequences of any
system/program.
7. 3. Cleanup work that arises
from using a less optimal
design in the short term,
because a schedule or budget
does not allow for long-
term/better design; delaying
the second release of a
product in favor of expediting
the first release.
8. When there is a change in the codebase, it often
necessitates other coordinated changes elsewhere in the
codebase, as well as documentation.
9. As a system or program is modified, its complexity
increases, as well as its entropy.
Unaddressed tech debt leads to software entropy.
10. Entropy is the measure of a system’s disorder.
Software Entropy refers to a system’s increasing
disorder or complexity.
Spoiler alert: A system’s disorder cannot be
reduced, only remain the same or increased.
11. "A little debt speeds up development, and can be
beneficial as long as the debt is paid back
promptly with a rewrite that reduces complexity
and streamlines future enhancements.”
- Ipek Ozkaya
12.
13.
14. When the WSJ’s website is offline for about an
hour due to a technical issue – we are annoyed
When planes are grounded across the country
due to a network issue at United Airlines – we are
baffled.
When the NYSE is unable to trade for about 4
hours due to a computer glitch – we start to see a
trend.
Hey guys, I’m Kacey Coughlin, I’m a web developer / designer at LinkedIn. Lets talk about technical debt! I know its not sexy, and a lot of this might be glaringly obvious to some people, but ….
…in case your one of the few that’s been too afraid to ask “Hey, WTF is tech debt?”, or you mistakenly wandered in here in a hungover stooper, OR you’re somehow still here from last night, then THIS talk is for YOU!
So in this beautiful time we have together, what will we be talking about? Well here’s a list! The basic who, what, where, when, and how. Don’t worry there’s no quiz at the end……OR IS THERE?!
What is technical debt? Well it turns out it’s a lot of things generally, and a few things specifically.
First of, it’s a metaphor created to help explain these issues to non-technical people
Referring to the eventual consequences of any program or system. Eventual because its usually not obvious at first, and that’s when most people get around to it. EVENTUALLY.
Its clean up work. Usually due to taking shortcuts, like using a hack to get the product shipped faster, and then writing the more elegant solution after launch. Essentially, you are delaying v2 in favor of expediting v1. This isn’t always the cause of tech debt, but it is a biggie.
You see, when a mommy and daddy love each other very much, they will introduce a change into the codebase. And when that happens, it will often require other changes be made elsewhere in the program. Also docs. These triggered changes, if left ignored, can grow up to be tech debt!
And so on and so forth! For every modification, the level of complexity in a program inherently grows, and with it, its entropy. Tech debt begets software entropy.
AND what is software entropy? It’s the measure of a system or program’s disorder or complexity. The thing is though, you can’t really reduce its entropy, only curb it, or increase it.
At a 2012 Agile Research Forum, Ipek Ozkaya explained it this way: A little debt speeds up development, and be beneficial as long as the debt is paid back promptly with a rewrite that reduces complexity and streamlines future enhancements.
This really sums it up nicely. If you’re going to push forward, know what you’re asking. Know what will be needed in the time after launch.
A study recently published by SEI at Carnegie Mellon, asked 1831 developers and engineers about technical debt. The results for an the question of what most contributes to technical debt, I though were very interesting. Keeping in mind, the participants on average had more than 6 years experience. Far and away, it appears most developers see architecture choices as the biggest contributor to tech debt; such as module dependencies, external dependencies, or even other team dependencies.
And diving into that a little deeper, asking the participants about specific architectural elements being among the most significant sources of tech debt. Again, bad choices in general taking the lead, but then we see inadequate testing, lack of documentation, and obsolete technology as big factors contributing to tech debt. So if we can just stop making bad choices, guys, I think we’ll all be better off!
An interesting article by Frances Lash, called “The Short Term Memory of Lessons Learned in Technical Debt”, touched on some pretty interesting things:
----- Meeting Notes (9/16/15 23:27) -----
And this brings up a good point. What are "computer glitchs" or "technical issues" really referring to? You could say they are just bugs, but usually bugs don't just happen out of no where and for no reason. Bugs come up because people were in a hurry, code wasn't reviewed, changes were made that caused other changes and no tests caught it. This could all be boiled down to technical debt.
Let’s take a look at a quick example of this. It’s not the NYSE, but it’ll do. Here is a very simple contact form. Now if I were the PM on this, I might be more concerned with cost and time, than elegance and sustainability, or even extensibility. And since this works, we’ll go with it. It just needs to work and that’s it. But NOW the designer wants to change the whole look and feel, SOOO I need new styles.
OK, so I hire a new kid to implement the new theme as quickly as possible. Lets say he is new to web dev, but said he could get it done. He just added some new classes for the new theme, said he had some trouble getting the submit button to change, so he just wrote some inline styles, no big deal, because it still works, and now I have my new theme and EVERYTHING IS AWESOME! I saved sooooooo much money and time! But WAIT, I keep getting emails with no names! I need to FORCE people to fill out the name field in order to submit this form, so I need some form validation in there somewhere…
BAM! Some form validation! But wait….
……NOW I want to move to handlebars! Lets make it a template now!
NOW I want it to be semantic! But I don’t know what that means, so I’ll just add roles to the buttons. And it works SO WHO CARES?! Ship it! SO now, I’ve fired my designer, and we’re adding Bootstrap!
Annnnnnd yea. Ok, this is still a fairly simple and easy form, but already it’s gotten fairly complex, and there’s a TON we can cut out at this point. We can make this so much more lightweight, and readable! This form is bloated, ugly, probably pretty slow. Most of what is in red can be stripped down, or even removed entirely at this point. We’re not using half those classes anymore. SO WHY does this happen?
This can and does happen for a number of reasons. Perhaps the programmer or engineer just doesn’t know. Maybe you hired someone more backend and they just don’t know how to write CSS, so now you have a lot of inline styles, and maybe Bootstrap.
Maybe there’s a gap in the process. Perhaps the business side is unaware of what tech debt is, and is pressuring the engineers to move on. Making decisions without considering future needs and implications, will almost always come back to haunt you. So the devs are writing code faster than they can review, or being forced to ship with forced hacks instead of clean, testable code.
Speaking of tests!! Did you write unit tests? I know, I know, its my least favorite part of the job, too. BUT testing beforehand ensures your code runs as you’d expect, and can uncover bugs BEFORE you ship! Think about that! This becomes doubley handy if you are developing your frontend and backend in parallel, thus making sure both sides are synced up.
----- Meeting Notes (9/16/15 22:42) -----
-Under time crunch, teams run both sides in parallel
-Both sides finish and commit code
-Get to product review before launch, find out holes in communication
-Oh you needed that to trigger an email?
-Oh you wanted tracking codes on ALL those?
-We were going to do WAHT in a cookie?
Are you using industry standards? Are you using open source libraries or rolling your own? Most of these things will have to happen eventually, so if you are not doing them at the beginning, or at least making time, then it’s going to be tech debt later. Taking advantage of open source libraries saves time, in that your not writing it all yourself, and there’s a good chance most of the bugs have already been ironed out. And if you decide to use a library later, it will most likely be much more difficult to shoehorn into your codebase, without causing a lot more debt, as we saw with the example.
AND finally, delaying the inevitable. Remember that code cleanup you said you were going to do RIGHT AFTER launch? That ticket for “spring cleaning” that you’ve been ignoring, or pushing off until a later date? Yea, you should get on that.
----- Meeting Notes (9/16/15 22:42) -----
But make sure its the right code that you are cleaning up. Sometimes, once a code cleanup is finially picked up, it might be something trivial that doesnt have as big a benefit. Make sure when you DO sit down to refactor your code, its code that will have the biggest effect in terms of paying down debt. You could worry about reducing the number of nesting levels in your Sass files, OR you could replace all your jquery animates with CSS transitions. The latter will be much more beneficial in the short term.
Though it may seem like it, keeping up with tech debt is not a losing battle. There’s a lot you can do to keep it at bay. In a lot of circumstances, YOU will have to be the advocate for it. You may have to stand up, push back, and say “LOOK! This code is messy, or old, and should be taken care of before we do anything!”
Just saying it works is not enough. I know we ALL do this. I do it. QA comes at you with something, and you can’t replicate it, so you tell them its fine in dev, or worse, it works on YOUR box so maybe they should go back and try again! They hate that, and you should be better than that. Own your code! And hold others to the same standard!
SO lets talk about code refactoring. What does that mean exactly?
Code refactoring is the effort to keep entropy to a minimum. It’s clean up. It’s you going back through the code you, or someone else, wrote, and removing anything that’s no longer needed or being used. That goes for everything. Something as simple as removing a class from a button, on up to an entire library. Are you REALLY using Jquery UI? Or can that just be dumped? Probably.
----- Meeting Notes (9/16/15 22:42) -----
And this can actually be done at any time. Personally, WHENEVER I'm fixing a bug, or adding a new feature, or I'm in existing code for some reason, and I see something, I do something about it. You know, call 311, because if you see something, say something! Even little stuff, like hoisting variables, or using CSS shorthand instead of writing 3 extra lines. It all helps, and little by little, you are making progress.
A lot of times, if there a part of the program that isn’t touched on a regular basis, that can be a good place to start. How’s that 404 page looking? Is there still a function in your codebase written by one of the founders 14 years ago?! I hope not, who would do that, that’d be weird ha ha ha….
Code refactoring should only restructure the existing code, without changing its external behavior. Improving readability of the code, reducing unneeded or unnecessary complexity, having adequate separation of concerns, this all should lead to more maintainable and testable code, AS WELL AS less tech debt.
So lets take another look at that example of ours. How could we make this cleaner? Well, there’s a lot, believe it or not.
We can remove a lot of the classes, wrapper divs, leverage placeholder text instead of labels maybe? We don’t need nearly that many IDs, or classes in general. With Handlebars, text type inputs are default, so we can remove that. Likewise, the submit type is default.
----- Meeting Notes (9/16/15 22:42) -----
Personally, I try to keep markup to a bare minimum. You rarely need IDs, and I don't but classes on things unless I really need to.
And then we put our logic in a separate javascript file…look at that! So fresh and so clean!
A lot of code can be made cleaner by NOT over thinking it, not over-engineering it. Spoiler alert: If no one can understand your code, you ARE NOT a badass. There’s a saying: Testing is twice as hard as writing the code; so if you write your code to be as clever as you are, then you’ll never be able to debug it.
So now that we kinda know what to look for, how can we track it? How can we know that we’re doing it right, and paying down the debt? Well, most people make Jira or bug tickets, or inline TODOs in the code. If you’re in an Agile/Scrum world, you will usually have these tickets in a backlog, to either be pulled in when there’s time (ie NEVER), or towards the end of an epic or year. These tickets or TODOs can be anything that comes to mind, and I will usually try to create them as I’m thinking about them. In a decorator with particularly long method names? Maybe shoot a quick ticket to the team to clean that up when they get a chance. Maybe make a whole epic that’s just cleaning up tech debt, and add tickets to that. Some things will be constants: software upgrades for examples. Some tech debt may not even be exposed until a new, more experienced person joins the team. Then you’re fucked.
And that brings us to everyone’s favorite: DOCUMENTATION!
For some reason it seems much harder to write the document that explains your code, than it is to actually WRITE THE CODE! It’s like doing the laundry or cleaning up your room, its not nearly as fun as writing the app and finally solving that weird routing issue that’s been driving you insane for the last 6 hours!...but it has to be done nonetheless. And despite the small minority of folks that do enjoy docs, they actually benefit everyone, and might save you some debt in the future! HOW, you ask? A couple key ways that I found:
1.) Planning….obviously! Take that extra couple of days, HELL take a WHOLE SPRINT, and really plan out how long a project is going to take. Now inflate those numbers, and add in another sprint for code refactoring!!!! Ideally, if you account for that time on the frontend, you will save more (and stress) on the backend. Worst case scenario, your project goes over, because of environmental issues, and you can use that extra time you added in for refactoring for getting the job done. And then refactor after.
2.) Scope creep! We ALLLLL have had, or have, a PM/client/director/whatever that has adjusted or added in, or just blown up, the product scope you were working on. You don’t need me to tell you it sucks when that happens, and I’m not saying having documentation will save you from this, BUT! If the original product spec is clearly explained, sometimes that’s enough backup for you to fend them off, and push back any new “improvements” to v2. Other times, I have had to show proof of changes and new features and constant revisions to the spec, to be able to effectively push back.
And finally….3.) Defining ownership! I cannot tell you how many times I have had to go into some code, and not be able to track down an owner. Not because they left the company, but because the code has changed hands, and no one really knows who is working on it now, and there’s no doc, or wiki for that service….. Knowing who owns what service or part of the codebase benefits you in two ways…
If you’re trying to fix a bug in someone’s code, being able to talk to the owner is obviously ideal so they can walk you through it. Ideal situation, I know, not always possible.
But more importantly, if you are that owner, you can shove off your work on someone else! Let them fix your bugs! It’s a win win!
SO to wrap all this up, I want to just point out that I realize all this may sound obvious, or not apply to everyone. I know I live and work in Silicon Valley, were most situations are far different than those at smaller companies, or companies not rooted in tech. We can’t all have a n/n-1 browser policy, I know!
But be good owners, do what you can, implement what you can. Go forth and change lives….and codebases!
I believe in you!
My name is Kacey Coughlin. I work at LinkedIn. I’m Kaceykaso on Twitter. Google me.