Author: Phillip Sharpless
Managing a software engineering project can be a daunting task for even an experienced team. Two important topics of note we can take a closer look at are those of rework and technical debt. The two concepts are sometimes confused for each other but taking a look at each a little bit more should make the distinction between the two more clear. Rework is an expensive and nagging thorn in the side of any project and understanding the primary causes of it in software development can help us avoid it as much as possible. The concept of technical debt is a little bit more complicated, as it’s not strictly a bad thing, although it certainly can easily become one. Understanding what the concept represents more fully will be beneficial to any project.
Rework
A study by the IJSEA (International Journal of Software Engineering & Applications) defines rework as “the additional effort of redoing a process or activity that was incorrectly implemented in the first instance or due to changes in requirements from clients.”
By this strictest of definitions “rework” is inherently bad and should always be aggressively avoided as much as possible. It’s estimated that 30-50% of all effort in software development can be categorized as rework, a surprisingly high percentage.
Some estimates place the typical cost of rework as being 2.5 times higher than if the work had been correctly implemented in the initial development phase. You have the time spent originally doing it the incorrect way, the same amount of time spent again doing it the correct way, as well as an overhead factor of additional time needed to reconcile the original way to the corrected way.
Given the surprisingly high amount of rework and the costly nature of it, what are the best strategies to minimize it? Studies on the topic have shown that most rework stems from incomplete or poorly understood requirements and from poor communication.
Incomplete or ambiguous requirements are the leading cause of rework, which is why the requirements gathering and defining phase is so critical. The old adage “measure twice, cut once” is very applicable. Spending a little bit of extra time up front to ensure the requirements are sound will save a great deal of time later. Hallmarks of good requirements include them being clear, consistent, complete and ultimately testable.
Good communication is also key to avoid rework. If requirements do for some reason change those changes need to be communicated as early as possible to avoid rework. Looping in all relevant stakeholders for early mockups, prototypes and demonstrations is also an extremely valuable tool for ensuring the project is on the right track and course correcting as quickly as possible if not.
Technical Debt
In the realm of software development the term “technical debt” has become popular and widely used, but it can be a somewhat nebulous term different people use to describe different things, often times incorrectly.
The origin of the concept is credited to Ward Cunningham one of the most prominent pioneers of the Agile software development philosophy.
The traditional definition of the concept is that “technical debt” is accumulated when some sort of shortcut is taken during an implementation, for any reason, and at some point in time in the future the work will need to be done in the “right” way.
The reasons these shortcuts are taken can certainly be valid. A hard deadline could be approaching and getting something working before it could be the top priority. A critical new feature may need to be added quickly even in a bolt on fashion, rather than re-architecting solution to properly accommodate it, etc.
Even if one hypothetically never took any shortcuts in their own development, technical debt can still be accrued. A key library or component within your implementation may eventually be end of lifed or simply eclipsed by a newer, better tool that’s more in favor. Your current implementation may run perfectly fine for the time being, but upgrading or replacing the outdated components is a form of technical debt that accrues over time.
The term debt is useful because it communicates key aspects of the situation to a non-technical crowd. Debt accrues interest. The more of it you have and the longer you have it, the larger the ultimate cost. The debt has to be paid at some point and you can only kick the can down the road for so long.
The ultimate goal of the term was to communicate to non-technical business stakeholders that future development resources need to be allocated to refactoring and improving elements of the existing project, not just strictly towards development of new features and enhancements.
Much like its financial counterpart, taking on technical debt allows you to do something of value in the short term significantly faster than you would otherwise be able to, but at an increased cost later.
Framing technical debt in these terms shows that it is not simply bad or buggy code, it is a strategic tool that can be used when real world constraints collide with theoretically optimal solutions.
However much like real world debt, if not managed or understood properly it can ultimately lead to technical “bankruptcy”, a situation where the debt has accumulated to the point the project has become completely unmaintainable and unsalvageable in its current form, with the only realistic path forward essentially a complete system rewrite, an extremely costly situation.
So we’ve learned what technical debt is and have also concluded some degree of it is unavoidable and in some cases it can be quite useful, but if misused can lead to some very bad outcomes. So how to best manage this concept of technical debt?
A very important distinction often made is that of intentional and unintentional technical debt. Unintentional technical debt, like rework, is something to always be avoided. Unintentional debt usually caused by poor design decisions due to lack of knowledge or standards. It’s often discovered after the fact, a sub-optimal solution has been implemented and at some point in time that issue will eventually need to be addressed. It can almost be thought of as eventual rework.
Intentional technical debt on the other hand, is the purposeful making of a decision that results in your project accumulating debt and choosing to do so anyway. The pros and cons of the decision have been weighed and it’s been decided to proceed. One of the most critical things at this juncture is the documentation of what the debt fully entails so it can be prioritized and addressed later. Understanding how much “technical debt” a project has already accumulated is also a very important factor for future decision making in regards to taking on more.
Technical debt of some sort is pretty much unavoidable, as software is ultimately built in the real world of competing priorities and resource constraints. Like financial debt it’s a tool that can be managed, sometimes for clear, effective purpose but if mismanaged will cause a great deal of headache and cost.
Closing Thoughts
Software engineering has proven to be one of the more challenging forms of projects to manage. Understanding the primary causes of rework and how best to avoid it and understanding the concept of technical debt and how to properly manage it should be up front priorities for any software project. Hopefully this quick overview has provided a good start for understanding these important topics.
Sources:
