Agile Zone is brought to you in partnership with:

I am a programmer and architect (the kind that writes code) with a focus on testing and open source; I maintain the PHPUnit_Selenium project. I believe programming is one of the hardest and most beautiful jobs in the world. Giorgio is a DZone MVB and is not an employee of DZone and has posted 638 posts at DZone. You can read more from them at their website. View Full User Profile

Lower your bar in Test-Driven Development

  • submit to reddit

The Manifesto for Software Craftsmanship is trying to raise the bar of professional software development, by pointing out that the only way to go fast is to to work as well as we can possibly do. However sometimes you have to temporarily lower your bar, instead of raising it, as part of iterative development.

A Test-Driven Development story

Back in a while I was trying to get a red test to pass. As always, I use TDD for writing any kind of code that I won't throw away immediately, so I had written a test before even opening a file from the production code.

This was a unit test for an OSGi service (this particular case is not really important, but it points out that there was a bit of infrastructure to set up to get a green test.) This test (one method) would instantiate the object from my external test bundle, then inject a Stub via a setter, execute a method and check some results.

I have no chances to get it right, and I continued to get failures for two hours before resetting my working copy (not using Pomodoros at the time). I was not familiar with the Java framework, and I did not understand what was failing. And it was not a good red test, where the expected result is not equal to the actual one or a NullPointerException is thrown: the JUnit Runner was failing, and God knew why.

The solution, as I've already said, was deleting everything that was not passing, and restart from a simple test. But here's the catch: when you can't make a test green, you shouldn't go to a simpler test, but to the simplest test, and work incrementally.

My approach

  • Write an empty JUnit test which only call assertTrue(true) to avoid a warning that no test cases were found; run it, and get it to pass (my case configuring the build parameters for example; or the bundles included when the OSGi session is created.)
  • Then modify the test, and instantiate the object under test (only a line). This reminded me of exporting the Java package from the source bundle and importing it in the test ones modifying the two MANIFEST.MF (not important if you're not into OSGi, but you get the point: there is something to do because of this line added to the test).
  • Only when the test passes again, call the method and begin to check its result (then think about the collaborator to inject).  And so on, one "dangerous" line at the time (the ones were you define String object do not count.)

Developer life examples

The concept of Walking Skeleton is a method to lower your bar: get a basic system up and running to send infrastructure out of the way, so that then you can concentrate on business logic instead of class paths, TCP ports and configuration.

Integration tests (in the definition of Growing Object-Oriented Software) are also a method to lower your bar: you check the behavior of external entity your abstractions depend on, isolating them in an integration tests group. Part of this is due to the possibility the integration tests run much slower than memory-based ones, but I do also with external JARs that do not access the file system or the network. Why? I get to run the library and understand its models with repeatable exploratory testing, lowering my goals. Then I can write an abstraction on top of it (and the the whole application.)

Acceptance Test-Driven Development followed by unit-level TDD is exactly lowering your bar: write an end-to-end test, then lower your bar and only get small unit test to pass, with the goal to integrate the different pieces when they're ready (bottom-up).

Optimistic and pessimistic protocols are the same solution applied to project management. Alistair Cockburn makes the example of communication between a team and its customer, and how they revert to a pessimistic protocol if the optimistic does not work well. Of course the optimistic protocols are faster - but when they fail the option to revert to a pessimistic one is always there.

Iterative development

In Test-Driven Development we can do small steps because of the blazingly fast, automated feedback cycle.

Unit tests are also meant for quickly discovering an error: when a unit test fails, you check what is referenced from it. In turn, when a new unit test fail, you check what code is missing and you're required to write. If there are too much things you not trust - and this is the case when you're new to a framework, like I was new to Equinox and OSGi - you have to keep your tests very short and work in tiny steps, to try one thing at the time and build confidence, or you'll never understand what production code (and configuration) you need to add. If it takes less than a second to run a single unit test, how much time are you losing?

Moreover, the test suite you build with TDD finds regression and stops them, as long as when someone breaks the build. So it's impossible to make negative progress, if you commit only with a green local build and have the discipline not to change tests or comment them only to get a green bar (actually you can do that to lower your bar, but not commit your changeset until you restore the commented lines). It's time for a graph a la Kent Beck, (similar to a burndown chart, but reversed):

On the x-coordinate we have time, on the y-coordinate progress measured by story points (completed, not remaining). You can get to peak periods where you make every test you write pass without problems, but also to periods with minor progresses or no apparent progress at all, while you're refactoring some code. But this line is an integral of a non-negative function - absence of regression is something in TDD we are starting to take for granted.

So don't be afraid to be slow, because every step you make will force your progress to converge (asymptotically) to the completed application. Asymptotically means that theoretically it can take an infinite time to get to the finish line, but in practice you'll probably jump over it when your velocity stabilizes. Also, if the finish line is the problem is scope creep, not the development methodology.

Feel free to lower your bar and take small steps, for making steady progress is more important than giant leaps.

Published at DZone with permission of Giorgio Sironi, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)


Mikael Couzic replied on Wed, 2010/06/23 - 10:15am

I'm sorry, but I'm pretty confident you misunderstood the message of the Manifesto for Software Craftsmanship. Let me explain.

"Raising the bar" means improving the quality of the software we craft. It doesn't mean going into long iterations at the end of we which we deliver perfect software. In fact, the Manifesto for Software Craftsmanship was written as an extension to the Agile Manifesto, whose authors are for many of them advocates of TDD. 

I feel like what you mean by "lowering the bar" is similar to what the Agile Manifesto refers as "Simplicity". It is in no way in opposition with "raising the bar" as defined in Software Craftmanship.

In that context, "lowering the bar" would mean making crappy and unreadable code, tangled dependencies, Big Bang development style.

Giorgio Sironi replied on Thu, 2010/06/24 - 5:35am in response to: Mikael Couzic

I know the motif of the movement, the opening was more a pun on the motto than a critic. I "ordered" yesterday a Green Wrist Band from Object Mentor. :) Moreover it's also a pun on the expression red bar / green bar.

I fel that "lowering the bar" was an appropriate title because it's exactly the situation of having written a test which is too difficult to satisfy, like a bar too difficult to pass in high jump, in the athletics metaphor. Athletes never lower the bar, but we can. So its only a title, I didn't mean to offend the Software Craftsmanship movement, which I really admire.

Mikael Couzic replied on Fri, 2010/06/25 - 10:16am in response to: Giorgio Sironi

I "ordered" yesterday a Green Wrist Band from Object Mentor

Awesome ! I want one too ! ^^

I agree, the title is appropriate, I just wanted to prevent any possible confusion about your article being in opposition with Software Craftmanship. On the contrary, lowering the bar temporarily, in the sense of making smaller steps, using TDD, is definitely an agile way to go (and by extension, a craftsman's way).

Giorgio Sironi replied on Sat, 2010/06/26 - 6:32am in response to: Mikael Couzic

Feedback is always welcome.

For the Green Wrist Band instead:

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.