Agile Zone is brought to you in partnership with:

I am a programmer and architect (the kind that writes code) with a focus on testing and open source; I maintain the PHPUnit_Selenium project. I believe programming is one of the hardest and most beautiful jobs in the world. Giorgio is a DZone MVB and is not an employee of DZone and has posted 638 posts at DZone. You can read more from them at their website. View Full User Profile

Practical PHP Testing Patterns: Data-Driven Test

10.27.2010
| 7896 views |
  • submit to reddit

A Data-Driven Test stores all the data needed to run the test in an external resource, and interprets that file when launched. These data comprehend fixtures, input data and expected results of the computation.

Data-Driven Tests are used when many tests differ very slightly: variation is kept only in the data and the code is kept short and clean, without duplication. These tests maintain a good coverage with fewer total lines of code, thanks to the transition from an imperative definition of new tests to a declarative approach.

Advantages

The separation between testing procedure and testing data is the main advantage of this pattern. In fact, the highest ROI is obtained by the application of this pattern in the case the test data have great variability, and the procedure for testing a component is really always the same. For instance, a tax calculation formula implementation can be tested with lots of different parameters to make sure it works in every imaginable corner case.

Testing data can then be read or even edited by non-technical people (acceptance tests can be based on this approach) such as domain experts or customers. It's one thing to write readable tests, but I would never expect a domain expert from the customer's company to learn PHP syntax to write them; giving him a text file is simpler and more effective to gain feedback.

Implementation

Both the test data and the test code should be kept under version control and in the loop of an automated test suite. One of the goals of such systems is that a fresh checkout on a new machine can easily run an almost full the test suite to confirm that the application is in good shape, and no components are missing.

The data formats to store the data in depend on what you have available: ini, xml, csv, or whatever you want are equivalent. However, parsing it shouldn't become other code to test: it's better to choose a commodity format. For example, in a Zend Framework application, an ini file that can be parsed by Zend_Config_Ini and which will never break is ideal.

Variations

There are Data-Driven Test Framework (like Fit and FitNesse) available for the Java platform, which provide a standard for data files (HTML and tables) and support for automatic parsing and execution of adapters.

Adopting one of these frameworks, or creating one, will result in less interpreter code to write. Also, in human readable output of the test results.

The approach however has a good ROI only if the number of tests with the same procedure and different data is very high. Building more adapters is more complex than building simple test methods.

A more common solution is the Naive xUnit Test Interpreter, which consists simply in test methods with loops that read the external data. They are quick to write, but would stop on the first error encountered (assert*() methods throw exceptions).

There are even more variations, like the Test Suite Object Generator, that instances more than one time the same Test Case, with different data; of the Test Suite Object Simulator, a Test Case that acts as a multiple one, executing all its tests for each set of input data.

The last two variations are not easily supported by PHPUnit (excluding override of run() and similar hacks)

Examples

In this small example, we will use the PHPUnit @dataProvider annotation as a way of implementing a not-so naive xUnit Test Interpreter. This is an evolution of Naive xUnit Test Interpreter where the different tests count as real tests and not as failure in a single one.

There won't be really an external file for the data, but at least this PHPUnit's feature centralizes the code in only one test method, which will be executed all the times it is necessary.

Again, in the Java world there are tools such as FitNesse that interpret an external HTML file; in PHP we have something like that for BDD, which read the test actions to take from an external feature file with a custom format.

<?php
class DataDrivenTest extends PHPUnit_Framework_TestCase
{
    public static function sqrtTestValues()
    {
        return array(
            array(0, 0),
            array(1, 1),
            array(4, 2),
            array(9, 3),
            array(25, 5),
            array(10000, 100)
        );
    }

    /**
     * @dataProvider sqrtTestValues
     */
    public function testSqrtWorksAsExpected($number, $squareRoot)
    {
        $this->assertEquals($squareRoot, sqrt($number));
    }
}
Published at DZone with permission of Giorgio Sironi, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)