Agile Zone is brought to you in partnership with:

I am a programmer and architect (the kind that writes code) with a focus on testing and open source; I maintain the PHPUnit_Selenium project. I believe programming is one of the hardest and most beautiful jobs in the world. Giorgio is a DZone MVB and is not an employee of DZone and has posted 638 posts at DZone. You can read more from them at their website. View Full User Profile

Finding wiring bugs

04.02.2012
| 4269 views |
  • submit to reddit

The best way to scale a test suite to hundreds of classes and to maintain it for several years is to focus only on unit tests for specifying and checking the basic correctness of the code. Unit tests run quickly and in isolation (typically only touching memory and not filesystem or the network) and scale well with respect to end-to-end tests where a database or a browser is involved.

However, unit tests only exercise logic, and not on the wiring that connects objects together. You only have to build an object graph to run the application - but a missing object or some NULL in the wrong place can blow up everything at the first user request.

"Classic" approaches

Wiring is commonly tested with end-to-end tests that target a few use cases; this kind of errors usually blows up everything in case there is a mismatch between dependencies and created object. Even in dynamic languages like PHP, unhandled calls to missing methods or passing the wrong object to a method expecting another class is not allowed.

For example, an end-to-end test may use Selenium to open a browser, post something to a forum, and check that the new page containing the discussion is created. This kind of tests are necessary, but won't tell you where the problem is when they fail; they are also much slower to execute, but they really checks the system as a whole, even exercising the deployment process and the database configuration.

The spirit of J.B. Rainsberger's Integration tests are a scam talk is to limit these tests as much as possible, by introducing contract tests and collaboration tests at the unit level. In this approach, for each collaboration test involving a Test Double like a Mock or a Stub, a contract test is created for checking that the expectation is correct on its production implementation(s).

Maintaining collaboration and contract tests in sync is discipline-based, like many parts of TDD (if you don't write tests before adding code, nothing can save you.) Of course something must still test the whole picture, and a few end-to-end tests are necessary.

Wiring bugs

How to check that wiring is correct? Building the object graph is a controller operation: even if end-to-end tests do not cover all the code (and they shouldn't, otherwise they will be too many), most of the time construction time checks like type hints:

public function __construct(Collaborator $collaborator) { ...

ensure that the wiring is correct. You cannot pass NULL in this PHP constructor. In Java and other languages similar defensive mechanisms can be called on the injected parameters:

assert collaborator != null;

while the conformance to an interface is already ensured via static typing.

There is an interesting case not covered by these checks: omission.

For example, in this construction code:

$object->addListener(new RedListener())
       //->addListener(new GreenListener())
       ->addListener(new BlueListener());

the GreenListener class is not used and behavior is missing, but this configuration won't raise any errors unless we test that specific behavior end-to-end. And if we have to do this for any single behavior, the test suite will never finish.

Of course here the addListener() call is commented, but is usually forgotten in real world scenarios.

Reflection to the rescue

We can perform a bit of static analysis (actually semi-static analysis, if this term exist) to check the wiring.

In our case, the Listener implementation could be added to listen to particular events. Both the generation of the event and its handling are unit-tested, and the only thing that remain to check is the connection between the two places.

The test we wrote did this (in a technology-specific way, but I think it can be done in any language with good reflection support):

  1. obtain the list of classes by including files via glob() and call get_declared_classes() (more on this later).
  2. Fire up the application (or just the interesting part of the object graph.)
  3. Inspect the object graph in the connection point to check every Listener in the application is present with one instance.

Since the Listeners were all in the same data structure (an array inside the object that routes the events) it's a matter of adding a simple method to retrieve them.

Here's how the test looks like:

<?php
$files = glob(APPLICATION . 'modules/*/listener/*.php');
foreach ($files as $file) {
    require_once $file;
}
$listeners = array();
foreach (get_declared_classes() as $className) {
    if (strstr($className, 'Listener')) {
        $listeners[] = $className;
    }
}
$listenersRepository = $someFactory->buildListenersRepositoryAccordingToProductionConfiguration();
$instantiated = $listenersRepository->getListeners();
foreach ($listeners as $listenerClass) {
    // check one of $listenersRepository is an instance of $listenerClass
    $found = false;
    foreach ($instantiated as $present) {
        if ($present instanceof $listenerClass) {
            $found = true;
        }
    }
    $this->assertTrue($found, "$listenerClass is not used in the application. Have you forgot to wire it?");
}

I won't call this test "static analysis" as we are building the object graph, even if we do not use it for production functionalities but only inspect it. The test only touches memory once the object graph is build, which happens only once.

Conclusions

Reusing the construction code was easier than I thought, a sign of good design. Actually, I lied about what we check as this is a simplification: in our case Listeners may had multiple methods, so we check every single public method of them that conformed to the signature:

public function method(Event_* $event)

where Event_* is a class like Event_BrokenCar. Via the ReflectionMethod object and ReflectionParameter::getClass() it's easy to extract Event_* and check that each Listener's method is wired to listen to the event it wants. There are no subclasses to take into account.

End-to-end tests will always be necessary: but when you have a large amount of configuration, checking its sanity is way faster for you and the machine than testing its effects; not a substitute but a good complement to end-to-end testing to expand its scope.

Published at DZone with permission of Giorgio Sironi, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)