A case study for BDD in improving throughput and collaboration

 

In this article, I will share a case study for behaviour-driven development (BDD) in improving throughput and collaboration.

The situation

At a well-known global investment bank, I was asked to review several product teams and to find out why the teams could not complete stories at the end of each sprint and why there was so many defects coming into the sprint.

The current measurable time spent on defects averaged at 35% of all developer time (and rising) each month. The defects were coming from UAT and Production and were typically urgent or semi-urgent and needed fixing during the sprint.

We reviewed the quality and found very low or non-existent unit and acceptance testing.

What did we do first?

I trained the teams in how to do Behaviour Driven Development (BDD). And we got the investment time we needed for the teams to build the infrastructure to support automated testing and deployment.

During the first 3 months, we learnt the science of testing. We spent significant time building frameworks and libraries to enable us to test complex data structures in the code.

When the team started, they estimated individual tests taking up 5 days of time for a 2-day story. By the end of the first 3 months, tests were being written in a few lines of code in minutes for the same type of story. However, no noticeable reduction in defect rate had been noticed.

What happened next?

In the 3-6 month period, we focused on the art of testing and included Product Owners (POs). POs wrote better acceptance tests that could be directly translated into code tests using the Gherkin language. This reduced the amount of rework within a sprint. The reason was that now POs were involved with the tests, we were more frequently building the right thing.

At the end of the 3-6 month period, we had achieved nearly zero defects on brand new greenfield code. Yet not much on the legacy code. Our time had been invested in getting POs up to speed and learning how to write good tests.

In the 6-9 months period, we saw a huge reduction in defects from the legacy code as well. By the end of the 9 months the defect rate was 4%.

This effectively saved the entire application suite and increased confidence in the development teams.

Due to the lack of unexpected issues arising from the defects, the teams were much better able to complete the feature stories on time and increase predictability too.

A happy side-effect

The side effect of all this learning and increase in quality was that a community of practice arose across the teams around testing and quality. This led to better communication across the teams on this and other matters.
Another effect was that Infrastructure were much more involved. They better understood automated testing and were able to change their model of server resource allocation. This would be used for other teams and departments.

Manual testing teams were used much less, as confidence in the test suite improved. This effectively merged test into development without any formal structure change.

The product group (all the teams) were written about in the bank’s internal magazine as Agile teams of the month. Screenshots of the build server with green ticks on the tests were included.

Agility is wholly dependent on automated testing since it gives you the real option to change direction and refactor in response to changing business needs.

I hope this case study can serve as encouragement to those considering investing in BDD testing.

Timeline added 2018:

Finally, you can read more about Behaviour Driven Development (BDD) here.