One behavior != one assertion

The debate on the “one assertion” principle continues. Pádraic Brady and I agree on the general principle, but since he is so categorical in his second article, I needed to inspect his reasoning more closely. It’s clear that our disagreement is more than superficial. The more I think about it, the more I disagree.

The state of the art

Pádraic maintains that one assertion per test is a rule that should always be followed unless there is a specific good reason to break it. I prefer it as a guideline, as does Robert C. Martin in the book Clean Code. The reference is not intended as an appeal to authority to “prove” that I’m right. I’m just making the point that I think this reflects the current state of the art, which is not necessarily perfect, of course.

How small is a behavior?

So what is the basis for wanting to enforce this principle as a strict rule? Pádraic claims that “1 behavior == 1 assertion”.

How does that make sense? The starting point is Behavior-Driven Development (BDD). I agree with the BDD principle that one test should test one behavior—as a guideline. It’s makes tests more focused, more readable and makes for better error messages when tests fail. Above all, it forces you to think about what is a behavior and what is not. And the idea of one assertion per test drives that thinking process.

From that point of view, “1 behavior == 1 assertion” seems totally logical. Unfortunately, it misses the point. It introduces a mechanical equivalence that negates the very thinking process I was just advocating. It makes the conceptual notion of a single, atomic behavior dependent on syntax and on the implementation specifics of a test framework.

For example, let’s say we want to test that a product object represents a certain model, a Logitech Harmony One remote control.

$this->assertEquals('Logitech', $product->make);
$this->assertEquals('Harmony One', $product->model);

It’s two assertions, but we’re only testing for one thing. We just want to make sure this product is what it should be and not something else. But let’s think some more about it. There are several ways we could make this into one assertion instead of two. This one, for instance:

$this->assertTrue(
'Logitech' == $product->make  &&  'Harmony One' == $product->model);

Not very pretty, is it? I think we can agree that this is not an improvement. More importantly, it doesn’t change the nature of what we are testing. We didn’t change what we’re testing, we just changed the way we express the test. A better way to get one assertion would be to make a custom assertion method such as assertLogitechHarmonyOne(). That would be more readable, but it wouldn’t help our statistics. The test framework would still see two assertions and report this as a 1 to 2 relationship between tests and assertions. And again, what we’re testing remains the same.

It gets worse. We’re testing one thing, but are we testing one behavior? No, we’re just testing the outcome. This is another logical flaw in the “1 behavior == 1 assertion” equivalence. A behavior is not tested by assertions, it’s tested by a complete test method. What happens before the assertions also counts, obviously. You could have multiple behaviors in the test with a single outcome that could be tested by a single assertion.

But I haven’t mentioned the simplest way to get one assertion per test. Just delete all assertions but one. It’s cheating, and problably toxic the quality of your tests, but it satisfies the criterion and generates good-looking statistics. It’s the thing that’s bound to happen if you try to force this on developers.

Better not use the “one assertion” principle as a strait jacket. It works better as exercise equipment to jog your mind.