Are your products destined for the scrap heap?
Ward Cunningham, one of the authors of the Agile Manifesto , once said that problems with code are like financial debt. It’s OK to borrow against the future, as long as you pay it off.
Since Ward first used this metaphor, which he called “Technical debt” has gained momentum. While people still disagree about the exact definition of technical debt, the core concept identifies a serious problem that many delivery teams are struggling to manage.
Technical Debt has a visible and an invisible element. Businesses are aware of the visible part and monitor bugs but it’s the invisible element that kills business agility.
Am I working with an unhealthy product with technical debt? How would I know?
Let’s have a look at a few behaviours you may recognize:
- Weak definition of “done” where its not well defined and represents quality for the product
- Slowing rate of productivity and increased cycles times to add similar sized new features
- Stressful releases, as the development team goes into to crunch mode as they work through lengthily change windows, executing scripts, manually deploying and configuring components, investigating incidents, fixing defects
- Applications we are scared to touch and have aging libraries and unnecessary dependencies
- Increasing time to dealing with incident investigations and fixing defects
- Low automated testing, or no extensive test suite where we are forced to test manually which can be repetitive, slow and error prone
- Fragile and tightly coupled components due to violation of good design principles, duplicated code, tangled architecture & unnecessarily complex dependencies.
- Lack of test, build, and deployment automation, plus anything else that could be automated that you do manually today
- Long feedback loops from Continuous Integration. Build, Test, Deploy
- Slow, ineffective tools
- Long-lived branches causing merging hell and increased delivery risk through late integration
- Important technical documentation that is missing or out-of-date
- Unnecessary technical documentation that is being maintained
- Missing or not enough test environments causing delays in delivery and/or reduction in quality
With technical debt, a cumulative flow diagram showing open and closed defects over time might look like this
Defects or bugs can tell you a lot about the state of your product’s quality. Great scrum teams fix bugs as they are found and don’t let them accumulate. If the number of open or escaped defects is trending up over time it’s an indicator of lower quality and technical debt.
In the cumulative flow diagram, we can see over time that the average number of open defects (vertical) is growing over time and the average time to fix (horizontal) is increasing.
What are the impacts of Technical Debt?
Let us first have a look at the cost of change from a technical perspective. The assumption here is that the red waterfall curve is using BDUF where the design is perfected before construction. This is highly unrealistic in the high novelty complex adaptive domain of software delivery. In software, delivery design is only validated through working (releasable) software. Hence, we see that the red line loses the ability to change rapidly over time.
At the other end of the scale, we have perfect quality in purple. Using professional scrum the design emerges each sprint, the definition of done (releasable) is achieved at least once per sprint. The design is frequently refactored and improved and all sources of technical debt are kept to zero.
The blue line is what happens to quality when the sources of technical debt are not dealt with each and every sprint. With every sprint, poor design decisions impact future decisions and compound each other.
Unreasonable cost of ownership
It should come as no surprise that the blue line is the most common scenario we see today. At first, the impact is low but soon starts to build momentum. The ability to maintain, support and add new features to the product becomes more complex and more expensive over time. This leads to a higher cumulative cost of ownership.
For example, if we built feature A in month 1 and the same feature A in month 18 with a good level of quality where the development teams are maintaining a sustainable pace then the cost of developing that feature would be comparable.
With high technical debt where the development teams are fighting against the design, the costs will grow rapidly over time.
- The cost (or difficulty) of change increases, eventually to the point of unmaintainability
- The ability to respond to the needs of customers decreases, making them extremely unhappy
- The predictability of results decreases. Estimating effort and complexity becomes more difficult. This decreases transparency and can impact trust.
Who suffers in this vicious circle of deceit?
- Customers face defects, missing features, crappy service resulting in lower customer satisfaction
- 1st line support teams will create more incidents this will cascade and create increased demand on operational support teams. More incidents and defects means more development time spent investigating with quick fixes and release patching. This results in increasing operational and capital expenditure.
- Products with poor design are more complex than needed, have more dependencies, require more infrastructure, more development time, more support time to run
- The organisation, teams, and leadership get bad publicity due to defects, delays, security issues, or outages. Organisations suffer increase cost through eventually needing to rewrite products as they’re no longer fit for purpose.
- Development teams must deal with the bad work of other developers which may cause attrition and loss of talent. Can cause “broken window” syndrome. It’s in a mess already so why fix it.
- As it gets worse customers complain about slow delivery which in turn increases the pressure to take more shortcuts, which increases the technical debt. In essence its a rapidly spiralling eternal return.
Options for dealing with Technical debt
Unfortunately, by the time organisations are paying attention none of the options are good.
- Do nothing and it gets worse
- Re-place/re-write the software (expensive, high risk, doesn’t address the root cause problem). Rinse and repeat in around 2 years time
- Stop creating more technical debt and systematically invest in incremental improvement.
b-agile’s Professional Scrum courses Professional Scrum Master course discusses the concept of Done and Technical Debt to a level needed by Scrum Masters and Leadership teams to be able to educate the organisation in the impacts and how to manage it.
b-agile’s ApplyingProfessional Scrum for Software Development courses gives an experience of how to deliver quality software with Scrum and DevOps practices. Understand how modern Agile engineering practices and supportive DevOps tools improve a team’s capability to deliver working software. Gain knowledge of how to leverage modern software development tools and practices.
b-agile has Technical Coaches that have extensive experience in helping organisations reduce their technical debt and gain the mindset, culture, skills to enable business agility. If you’d like to know more or how we can help you or your teams build healthy products then check out our Technical Agility services. Or contact us to get in touch.
In our next post, we’ll look at a few ideas of how to turn the ship around