Agile Zone is brought to you in partnership with:

I am a programmer and architect (the kind that writes code) with a focus on testing and open source; I maintain the PHPUnit_Selenium project. I believe programming is one of the hardest and most beautiful jobs in the world. Giorgio is a DZone MVB and is not an employee of DZone and has posted 638 posts at DZone. You can read more from them at their website. View Full User Profile

TDD for multithreaded applications

01.03.2012
| 10223 views |
  • submit to reddit

This article describes some practices for test-driving multithreaded and distributed applications written in Java. The example I worked on and we will use is a peer-to-peer application composed of many Nodes (clients) and of a few Supernodes (servers).

The ultimate goal it to build an application composed of all these entities, but the first tests target a Supernode serving one or more Nodes.

The walking skeleton

TDD is mostly iterative, but needs a starting point. The simplest story we can think of is that of a Node connecting to a Supernode.

Client and servers usually run in their own threads (in the case of the server, multiple ones), but initially the Node object can just be a POJO and run in the test's thread because we do not need to manage multiple Nodes yet.

The Supernode object instead is a Thread (or a Runnable) and so we already face a simplified version of the synchronization problem: how to make sure the Supernode is ready to answer to connections once we have started its thread?

The JUnit test is the following:

    @Test
    public void aNodeCanConnectToASupernode() throws Exception {
        Supernode supernode = new Supernode(8888);
        supernode.start();
        supernode.ensureStartupIsFinished();
        Node n = new Node();
        n.connect("127.0.0.1", 8888);
        assertEquals(1, supernode.getNodes());
    }

supernode.start() runs the new thread, while the call to ensureStartupIsFinished() will have to block until the other thread is ready. Then, we create a Node object and tell it to connect; after it has finished this operation, we count how many nodes have connected to the Supernode.

To satisfy this test, the Supernode can be a single-threaded server:

public class Supernode extends Thread {

    private int port;
    private boolean startupCompleted;
    private int nodes = 0;

    public Supernode(int port) {
        this.port = port;
        this.startupCompleted = false;
    }

    public void run() {
        ServerSocket sock;
        try {
            sock = new ServerSocket(this.port);
            // ...networking setup...

        synchronized (this) {
            this.startupCompleted = true;
            notify();
        }
          while (true) {
            // ...accepting new connections on sock and other stuff
        }
    }

    public int getNodes() {
        return nodes;
    }

    synchronized public void ensureStartupIsFinished() throws InterruptedException {
        while (!this.startupCompleted) {
            wait();
        }
    }

}

What's in this first example?

  • Thread objects are manageable as POJOs from a single JVM: as long as we write them with this API it will be simple to instantiate and terminate them, and to add primitives for synchronization.
  • The startupCompleted field, which is an example of this synchronization behavior added to the production code. Adding production code just for end-to-end testing purposes is not uncommon.

The test thread blocks inside ensureStartupIsFinished() until it is woken up via notification. Even then, startupCompleted must be true or it will wait more. This is Plain Old Java Synchronization: note the synchronized blocks around this.wait() and this.notify(). The problem with frameworks and containers is you have to hope they provide the synchronization facilities to test your code once it's inside them: have you ever tried to wait for Tomcat to start?

There are some noticeable missing parts in this code:

  • the threads for each node. The current test does not require them as only one Node is connecting for now.
  • Thread.sleep() calls: at least for the happy paths I have covered until now, I never need to introduce them and considered them a smell.
  • Configuration files: if we had to read configuration, the tests would take really long to write and would refer continuously to external resources. This is the case when testing with external tools which are not embeddable (Tomcat requiring configuration files while Jetty allowing configuration to be passed in Java test code). You can always add file-based configuration later, but for now it will slow us down.

Evolution

By adding one test at the time with a larger scope, we can try to evolve the code and add the difficult networking, multithreading part one bit at a time.

After some iterations, the test becomes:

public class FileSharingNetworkTest {
    Supernode supernode;
    
    @Before
    public void setUp() throws Exception {
        supernode = new Supernode(8888);
        supernode.start();
        supernode.ensureStartupIsFinished();
    }
    
    @After
    public void tearDown() throws Exception {
        supernode.ensureStop();
    }
    
    @Test
    public void aNodeCanConnectToASupernode() throws Exception {
        Node n = newNode(Arrays.asList("1.txt", "2.txt"));
        n.ensureConnectionIsFinished();
        assertEquals(1, supernode.getNodes());
        assertEquals(2, supernode.getDocuments());
    }
    
    @Test
    public void multipleNodesCanConnectToASupernodeSimultaneously() throws Exception {
        Node n1 = newNode();
        Node n2 = newNode();
        n1.ensureConnectionIsFinished();
        n2.ensureConnectionIsFinished();
        assertEquals(2, supernode.getNodes());
    }
    
    private Node newNode() {
        Node n = new Node("127.0.0.1", 8888);
        n.start();
        n.setDocumentList(Arrays.asList("1.txt", "2.txt"));
        return n;
    }
    
    private Node newNode(List<String> documentList) {
        Node n = new Node("127.0.0.1", 8888);
        n.start();
        n.setDocumentList(documentList);
        return n;
    }
}

The server-side code doesn't have multiple threads yet. What is the test case that will call for them? You have to find it and write it. This workflow will ensure that there is a test that targets this case. In my case, it was the first test requiring interaction between the two clients, where one had to see the documents listed by the other after both had connected.

Even if you know where you will end up, you can test-drive the implementation: the advantage is that you understand better a standard design and ensure its test coverage. After a few more tests, I have reached a multithreaded server with a main thread and chidren for managing the connections; and Node objects implemented as independent threads.

Conclusions

When working with TDD at a system scale that includes asynchronous behavior, we should strive for a test suite that is:

  • fast; even with multiple threads to wait for, a single end to end test should take less than a second to complete.
  • Comprehensive; TDD makes us only write tested code instead of copying down snippets from the web.
  • Robust: totally deterministic, as every run will either pass or fail, even when repeated dozens of times. There should be no sleeping calls for all the happy paths; there should be synchronization and stopping facilities built into the system.
  • Featuring unit tests: along with the end to end tests we should write unit tests for the objects we need to extract (and that will be single threaded). It was easy for me to get caught up into covering more and more cases with a full scale test, but unit tests are better at pointing out where a bug resides.

We also have to keep in mind how to design our objects and interfaces:

  • not starting with N threads but with at most 1 more than the test's one (the server or a remote peer).
  • Evolving them: adding a few lines of verbose Java networking code each time. My example has evolved to N client threads, a server main thread and N server children threads talking with each client. I will now have to evolve it to a network of supernodes, being this about a file sharing network; to introduce secure channels and certificates. The difficult part is to constantly refactor to support new stories without having to rework the whole system for a single one.
  • Not only extracting methods (an automated operation), but also to extract interfaces and most importantly objects; targeting the longest and complex classes and chopping them down into basic responsibilities.
Published at DZone with permission of Giorgio Sironi, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Tom Gardner replied on Tue, 2012/01/03 - 6:19am

If you have a "totally deterministic" set of test, how do you suggest checking for race conditions and priority inversions and avoidance of livelock/deadlock?

Once you have provided an answer for "simple" J2SE programs, please extend your answer to include programs that depend on "complex" frameworks such as JEE servers (or JAIN servers for that matter).

Finally, please extend your answer to include consideration of partial failure within the system of JVMs and physical servers and databases and network.

After those tests have completed, and not before, you will have confidence that your multithreaded application does not have any latent multithreading problems.

(Hint: doing that even the first problem for "just" livelock will mean that you have made your fortune, both academically and financially)

Giorgio Sironi replied on Tue, 2012/01/03 - 12:28pm in response to: Tom Gardner

We don't have to enter the realm of distributed computing to find this issue: as Dijkstra said in 1969 there is no testing procedure that can prove the total absence of bugs (even in the implementation of the + operator). The problem this approach tries to solve (see http://martinfowler.com/articles/nonDeterminism.html) is that often slow and brittle tests invalidate the whole suite; the scenarios that we test should complete as soon as possible.

Tom Gardner replied on Tue, 2012/01/03 - 6:54pm

Precisely - except that I think you'll find that the concepts predate Dijkstra!

It would have been courteous to your audience if you had given them an early indication of what problems were solved by your technique, and just as importantly, what problems remained or weren't addressed.

Johan Haleby replied on Wed, 2012/01/04 - 10:00am

Hi,
You may also want to checkout the Awaitility framework that I've founded to help you test multithreaded apps.
Regards,
/Johan

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.