7 Top Reasons for Introducing Technical Debt

It's very common in software development to trade off short term gain against long-term pain. If there's a "quick way" to do get a bug fixed or a feature implemented, then more often than not, developers tend to choose it. This results, over time, in the accumulation of "technical debt", where the decisions made in the past are hindering our progress going forwards.

This is why "technical debt" is often such a controversial topic. Martin Fowler recognizes that not every decision made to introduce technical debt is necessarily a prudent one. Maybe our own naivety and inexperience leads to us "inadvertently" introducing technical debt. Or maybe we just capitulate to the business pressure to get the work done as soon as possible, long-term consequences be damned.

But I would argue, that more often than not, as developers, we believe we are making a justifiable decision. In over 20 years as a software developer I don't recall ever having a discussion saying "let's be lazy and irresponsible and get this finished as quickly as possible". Instead, there is a quite reasonable sounding rationalization every time we choose the quick way.

Here's my list of the top seven justifications for introducing technical debt.

1. "It's urgent!"

Sometimes in software development, it really is urgent. When I worked on embedded software in telecoms hardware, downtime could result in mobile phone networks losing money at an astonishing rate, and their customers potentially unable to communicate urgent messages. So if a critical bug can be fixed with three lines of slightly "hacky" code, then that's probably preferable to spending a week implementing it the "proper way".

2. "We need to be strategic"

Another common reason for rushing through some work quicker than would be ideal is the desire to be "first to market". This is one of the main scenarios Ward Cunningham envisioned when he coined the term "technical debt". Sometimes, the commercial advantage of beating your competitors to market is worth the pain of needing to rewrite large portions of your application further down the road.

3. "It's just an MVP"

Closely related is the fact that often in software development, we're creating a feature that may not end up being used long-term. We think it's what the customers want, but until we get it out there in front of them, we're not sure. So we don't want to waste vast amounts of time perfecting code that may not end up being used by anyone. When we're in this scenario, there's a lot of logic to the idea of creating a "minimum viable product" or a "proof of concept". Of course, we intend to revisit the code in the future and provide a fuller, more robust implementation if the feature proves a success. But it's not uncommon for these bare-bones initial implementations to live a lot longer than we intended.

4. "We need to be pragmatic"

Suppose you're asked to design a new feature and estimate how long it will take to implement. You produce a thorough design considering security, scalability, observability, etc, and estimate that it will take six months development time. This comes as a shock to the business. They wanted the feature, but not that much. You're asked to be "realistic" and "pragmatic" about it. You're reminded that "The perfect is the enemy of the good". And of course, it's true that there's always "more than one way to skin a cat". Maybe there's another approach to the problem that's "good enough" and can be done in half the time.

5. KISS

The "Keep it Simple, Stupid (KISS)" principle is well-known and an important reminder that often in software, the most simple solution is in fact the best one. Often by writing less code, we end up with a better, more maintainable, more performant codebase. Technical debt can be introduced not only by a careless developer overlooking important concerns, but by an over-zealous developer writing copious amounts of bloated and complex code to handle imagined problems that may never occur in real-world systems. Many developers have a tendency to "gold plate" or "over-engineer", but it requires plenty of wisdom and experience to make the right judgment calls to resist this temptation.

6. YAGNI

Another closely related reason for technical debt is the well-meaning tendency to write additional code because we think it will probably be needed in the future. At first glance, this seems to be a wise approach - surely the more we bear the future in mind when we write code, the less technical debt we will introduce. The trouble is, we are typically terrible at predicting the future. For example, I've seen complex localization frameworks added into software that was only ever delivered in English, and fragile configuration DSLs introduced for logic that never changed and would have been far simpler to hard-code. Again, striking the right balance between a strict YAGNI approach of only coding what is absolutely needed now, and strategically designing in accessibility points for features you know are coming soon, is hard to get right.

7. "It's too risky"

This one is perhaps the most common reason for introducing (or more commonly, compounding) technical debt. Say you're working on extending some existing code, and you recognize that it is in dire need of refactoring, or maybe needs an upgrade to use a newer framework version. When this happens you have a great opportunity to pay off some technical debt, improving the overall state of the existing code, before adding your new feature. The trouble is, wide-scale refactoring and upgrading to newer technology stacks are generally disruptive and risky changes. Many developers quite understandably take the safer route of maintaining the status quo, but this results in an even bigger job for whoever does finally attempt to sort out the mess in the future.

Are these valid reasons?

What do all the reasons I gave above have in common? They are all arguably the "right decision" in certain circumstances. But they may all also cause us considerable problems in the future. The challenge with technical debt is that, unlike real-world debt (where your creditors can be relied on to demand repayment), it's not at all obvious which instances "technical debt" will come back to bite you. Sometimes your sub-optimal changes sit harmlessly in source control for decades to come without getting in anyone's way. Whereas other quite seemingly reasonable and pragmatic compromises turn out to be the source of constant pain. It's only with the benefit of hindsight that it becomes clear what decision should have been made.

What can we do about this? Is it hopeless given that you can't know for sure whether these "reasons" are valid until it's too late? A few brief thoughts in conclusion.

1. Whenever you find yourself citing one of the seven reasons I gave above, try to give at least some thought to how the technical debt you introduce as a result might pose problems in the future, and what you can do to minimise those risks.

2. Cut the developers who went before you some slack when you're next about to rant about how "stupid" and shortsighted the existing code is. Most likely they were making what seemed at the time to be a sensible and pragmatic choice.

3. Accept a certain amount of "technical debt" is inevitable on any large, and ensure you incorporate sufficient time to address it and keep it from spiraling out of control. This should be a regular part of all ongoing development, not a one off technical debt "blitz".

Anyway, I know the topic of technical debt always raises some interesting (and sometimes heated) discussion, so I'd love to hear your thoughts on these reasons. This is also a great opportunity for me to say that on October 2nd I'll be speaking about technical debt at Techorama Netherlands. It's a great conference with an excellent lineup again this year, so if you're able to make it