Many test methods versus custom assertions

This is something I posted to the Sitepoint PHP Application Design Forum with a little bit of added background.

The background is the idea that unit test methods, for the sake of readability, should test only one single behavior. This may mean several tests for one method under test, since one method may have several behaviors. One version of this is the notion of one assertion per test.

Web tests are not quite the same thing as unit test. On the Sitepoint forum, in a discussion on web testing, I said:

It’s also very slow to have too many separate test methods. So instead, to make it readable, I tend to use custom assertions.

Marcus Baker asked me for examples. I tried to illustrate the principle with an example that may be a something of a caricature.

If performance and speed were not an issue, I might do something like this, keeping each test method short and sweet: PHP Code:

class ArticleListTest extends WebTestCase {

    function setUp() {
        // Some code to log in and go to the article list page
        //....
    }

    function testHttpStatusOk() {
        //...
    }

    function testHasCorrectTitle() {
        //...
    }

    function testHasNoDebugInfoAccidentallyLeftBehind() {
        //...
    }
}

Unfortunately, it would be nauseatingly slow, since you have to traverse some web pages for every single test method. Instead, I can try to get similar small chunking by using custom assertions: PHP Code:

class VariousPageTests extends WebTestCase {

    function testLoginAndArticleList() {
        // Some code to log in
        //....

        $this->assertStartPage();

        // Some code to go to article list page
        //....

        $this->assertArticleListPage();
    }

    function assertArticleListPage() {
        $this->assertHttpStatusOk();
        $this->assertHasCorrectTitle();
        $this->assertHasNoDebugInfoAccidentallyLeftBehind();
    }

    function assertStartPage() {
        //
    }
}

This way, the code has names for approximately the same details as in the first example.