Issues with implementing Test Driven Development

Overview

An aspect of Agile is fast feedback. Test Driven Development (TDD) provides fast feedback to developers of errors in their code and is recognised as good practice by many people in the agile community. So why do so few developers practice it? This blog looks at the issues with implementing TDD in legacy code. These stem from the fact that many teams do not fully understand whom their customers are and Product Owners do not care enough about software quality.

Training

In my experience there is insufficient example material/training available on how to do TDD in real situations. This is compounded by two main factors;

1. Many developers have never been trained in TDD and do not see why they should change the way they work, which has always served them well so far; and

2. Unlike the TDD examples, most people work on legacy code. This code is not well structured, making TDD very hard to do.

To address the first point we need to create a compelling case to get developers to change how they work and then provide training in the new practices.

The case for Developers

TDD is an investment in future productivity. Putting tests in the code now saves time in debugging. The tests also helps you write code that is easier to work with as you code less defects in the first place. As the tests also encourage you to refactor, improving the software design, the code should be cleaner and better architected making it easier to work with. This makes developers more productive.

The problem with the above, from the developer’s viewpoint, is that they spend more time today in order to save time tomorrow. The reward is not fast enough.  When a developer is under pressure to deliver for the Product Owner why delay delivery today? This issue can be resolved by understanding who the customer is.

Understand the Customer’s needs

Teams have more than 1 customer. Too often only the Product Owner is considered the customer. This can be to the detriment of quality of the software asset.

As a development manager one of my responsibilities is to manage the organisations software assets for the future good of the business. This means I need to ensure the software is well designed and easy to maintain. With legacy code we often inherit poorly designed applications where the internal architecture makes it difficult to introduce change. This forces Developers to make a choice when implementing a user story. Do they improve the design of the system, by for example introducing a separation of responsibilities and create a new class, or do they deliver the story as quickly as possible. Often, only one customer is considered in delivering the story and the other’s requirements, the development manager, is ignored. Over time the code gets worse and changes take more and more time to make as the software degrades.

Build the needs of all customers into the definition of Done

As a customer I want metrics, such as Cyclomatic Complexity, built into the definition of done. We can then assess the team against improving the code over time. This is but one aspect of getting the developer to change. We need to sell them on all the benefits. From a developer’s viewpoint TDD reduces the number of defects so they get to spend more time coding than fixing errors, the tests, over time, also provide protection to the developer for future changes.

After one of my teams had been putting in unit tests for a few months they found they could start relying on the tests to find errors rather than the developer spending a lot of time worrying if their latest change may have broken something else. This is a big benefit to Developers, but it takes time in legacy code before the team start to realise the benefit. There are other benefits but we do need to sell them to developers.

One of the benefits of Cyclomatic Complexity value is also the number of tests required to test the module. The IDE should have a capability of assessing the number of tests being run and hence the test coverage by module. This gives the team feedback on where they should focus the effort, as sometimes the cost of TDD, due to the effort of large refactoring, is not worth the effort in areas of the system that rarely change.

Developer Training

Assuming we have built a case for the developers we also need to train them. Unfortunately training in TDD is not enough. We are dealing with legacy code so the TDD cycle of write a test, write the code, refactor hits a wall.

Refactoring is no easy task in legacy code. Often the internal architecture of the system is poor. We need to train developers in application architecture and software design.  SOLID principles to guide them in designing how the system should look as they refactor. This takes time and practice.

In refactoring one big issue to be overcome is when to stop.

Take an example; a developer is delivering a user story that requires a change to a calculation. The calculation is included in a large class with many responsibilities so the developer decides to implement a new class with a single responsibility but in doing so causes the system to fail, as another change is also required to enable the refactoring to work. This cause and effect chain can continue a long way with poorly designed software. To deal with this we also need to train developers in refactoring strategies, such as Mikado. These refactoring strategies help remove the smells of bed design.

Unfortunately these techniques take time to learn and reduce the team’s productivity in the short term whilst they develop new skills and ways of working. However, actively managing the quality of the code provides enormous benefits in the medium term by improving the maintainability of the code and providing very quick feedback on coding errors.

The Product Owner

Well we finally have convinced the team about the benefits and they have been trained. But what about the Product Owner? Do they understand the need for investing in the quality of the software for future maintainability/productivity or do they still only care about speed of delivery today?