Test First approach sounds simple enough, right?

As the name suggests, "test first" means you write a test before the actual code, and keep the tests running all the time. Well, maybe not that second part explicitly. But what if I said it actually means more than that?

It did originate with that simple meaning. When we hear the word "test", most of us think of checking in the traditional sense, validating the code. That is the first source of confusion about what it means today, for technical and non-technical people alike. The way you think about testing in an agile approach is fundamentally different from the traditional one, but let's leave that for another day.

Then there are all the practices that sit under the test-first banner: TDD, BDD and ATDD. Are they all simply about writing tests first and automating them? And what are the differences, if any?

Test first has moved on from its original concept and now has a broader meaning. Over time it has grown to include:

• Test-driven development (TDD)
• Behaviour-driven development (BDD)
• A ubiquitous language for acceptance scenarios and specifications, such as Gherkin

It has become a way of focusing quality on both the product and the design, through faster feedback loops at several levels, rather than simply writing tests before code.

That is probably why, in 2011, Gojko Adzic introduced a different name to group these practices: Specification by Example, with a redefined purpose of "building the product right and building the right product". The primary outputs are living documentation and collaboration. Automated tests are the secondary output.

On first exposure to test first, TDD and BDD, people tend to think in terms of traditional testing. That sends them down the wrong rabbit hole, automating their tests with a record-and-play tool.

Context is key. I have found it useful to briefly explain how test first and its associated practices evolved, so people understand the meaning of each one, and its purpose, focus, differences and benefits.

Test-first programming (1999)

Test-first programming was introduced by Kent Beck in 1999 in Extreme Programming Explained. It says: write unit tests before programming and keep all the tests running at all times. A year later, in 2000, Beck published Test-Driven Development by Example, which explains that TDD is a software process, not just writing tests before code, with one more important aspect: refactoring. The sequence in his book is:

1. Add a test.
2. Run all the tests and see the new one fail.
3. Write the code.
4. Run the tests.
5. Refactor the code.
6. Repeat.

This is the red-green-refactor cycle. Beck and others found that writing the test first also had a tremendously positive effect on the architecture and design of the software. It became:

• Testable
• Decoupled
• Reusable
• Independently deployable
• Independently developable
• Independently reasonable

The influence on design and architecture was so significant that Beck renamed the practice from test-first programming to test-driven development.

Test-driven development 2.0 (2004)

TDD moved a step beyond Beck's original cycle with Robert C. Martin (Uncle Bob) and the three laws of test-driven development:

1. You are not allowed to write any production code unless it is to make a failing unit test pass.
2. You are not allowed to write any more of a unit test than is sufficient to fail, and compilation failures are failures.
3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.

This is sometimes called test first with baby steps. Our Introduction to test-driven development video shows TDD and baby steps in action on a real problem.

Martin went further with the third law and discovered the Transformation Priority Premise: a simpler transformation should be preferred. It helps the programmer do the simplest possible thing by favouring transformations from the top of this list over those further down:

1. ({} → nil) no code at all → code that employs nil
2. (nil → constant)
3. (constant → constant+) a simple constant to a more complex constant
4. (constant → scalar) replacing a constant with a variable or an argument
5. (statement → statements) adding more unconditional statements
6. (unconditional → if) splitting the execution path
7. (scalar → array)
8. (array → container)
9. (statement → recursion)
10. (if → while)
11. (expression → function) replacing an expression with a function or algorithm
12. (variable → assignment) replacing the value of a variable

He suggests amending the red-green-refactor cycle with three provisions:

• When passing a test, prefer higher-priority transformations.
• When posing a test, choose one that can be passed with higher-priority transformations.
• When an implementation seems to need a low-priority transformation, backtrack to see if there is a simpler test to pass.

Follow these rules correctly and you should find:

As the tests get more specific, the code gets more generic.

Behaviour-driven development (2004)

Behaviour-driven development was developed by Dan North while helping his colleagues learn TDD. He found that:

• "Behaviour" is a more useful word than "test". People's misunderstanding of TDD always came back to the word "test".
• Test method names should be a sentence, and the word "test" can be stripped out.
• A simple sentence template keeps a test method focused. It should do one thing and only that thing.
• An expressive test name helps when a test fails.

In 2003 North created the JBehave framework to replace JUnit, removing any reference to testing and replacing the testing vocabulary with a behaviour one.

In 2004 his next discovery was the concept of business value, and why we write software at all. He found the framework helped the team focus on feature behaviour, and that missing requirements are behaviour too. Around the same time Eric Evans published Domain-Driven Design, describing how to model a system using a ubiquitous language based on the business domain, so the business vocabulary permeates right into the codebase. North and his colleagues applied the same idea to the analysis process itself, which is where Given, When, Then for acceptance criteria came from.

Since then many people have built other BDD frameworks, and BDD has come to be framed as a communication and collaboration framework for developers, QA and the business. At the Agile Specification, BDD and Testing eXchange in London in November 2009, North described BDD as:

BDD is a second-generation, outside-in, pull-based, multiple-stakeholder, high automation, agile methodology. It describes a cycle of interactions with well-defined outputs, resulting in the delivery of working, tested software that matters.

In an interview with North at the GOTO conference in 2013, Liz Keogh defined it as:

It's using examples to talk through how an application behaves. And having conversations about those examples.

Since JBehave, the leading BDD framework has become Cucumber. It reads executable specifications written in plain text following a set of rules called Gherkin. It is available in most languages, you can test almost anything with it, and it works well with other tools for testing APIs, browsers and other system endpoints.

What about acceptance test-driven development?

Acceptance test-driven development, ATDD, is where the whole team collaborates to define the acceptance criteria before implementation. It drives acceptance testing rather than unit testing, through a detailed, readable description of the system's function with proper examples, and it takes an outside-in approach. Sounds familiar?

Most of the time BDD and ATDD are used interchangeably. ATDD can be implemented with the Given-When-Then format and BDD tools. And BDD frameworks can be used for TDD. Confused yet?

Personally, we used ATDD for two years before we became aware of the term BDD. When we first heard it, we could not see how it differed from what we were doing. Perhaps because we were already practising domain-driven design, collaborating, and using a test framework with a Given-When-Then option. Our acceptance tests ran end to end, but also at the component, integration and unit level.

John Ferguson Smart's BDD in Action would have cleared that up far quicker, had it existed at the time. The sections "Coding the BDD way" and "Taking BDD further" in particular. Highly recommended.

Summary

In essence, the approach has evolved to be about the conversation first, between the business and the developers, to build the right product, and an outside-in workflow to build the product right with high automation.

Understand the focus, outputs and benefits of each practice and use whichever suits your needs. Just because a practice is described as outside-in does not mean you cannot work inside-out. And do not let the tool define you. Using a BDD tool does not mean you are doing BDD.

I keep it simple. I use the term BDD when the approach is about collaboration with all stakeholders, customer, developer, tester and the rest, to build the right product and produce living documentation that everyone can read. I use TDD when the tests are about driving design and building the product right, which tends to be at the unit of work and unit test level.

Hope this helps. If you would like help bringing these practices to your team, talk to us about technical coaching.