Test Driven Synchronization Policies with assertSynchronized

As you're surely aware, JConch 1.1 was released yesterday... it includes a tool I've used over the last two years at work to test drive synchronization policies in Java code: assertSynchronized. Even if you're not practicing TDD, assertSynchronized has proven to be a great aide in replicating concurrency defects and verifying that fixes work correctly. And, it's great to have test coverage on synchronization policies during refactoring time. I, and several team members, have genuinely found this assert useful, and this post walks you through how it works.

Here's the idea behind assertSynchronized:

1. Wrap code snippets that should be thread safe in Callable objects


2. Create another Callable that returns all your code snippets as a Collection


3. then assertSynchronized runs your code snippets a whole bunch of times across several threads and makes sure there are no exceptions

This is not deterministic but is effective. My main concern is quickly replicating concurrency defects, which this assertion enables.

For clarity, let's name and describe a few things before seeing a code sample. Each (hopefully) synchronized code snippet is a "task". The Callable that produces the task list is a "task factory". assertSynchronized makes 1000 passes at the task factory. In each pass it gets the task list and queues up as many tasks as it can on multiple threads. When the queue is full, it releases the tasks and hopes for concurrency related exceptions. assertSynchronized moves on to the next iteration when all the tasks from the task factory are complete. Any errors are accumulated and reported at the end with an AssertionError taking the form of "java.lang.AssertionError: An exception was raised running the synchronization test. The test failed x out of y times. Last known error:" followed by a stack trace. There are zero dependencies on a testing framework, so it works in JUnit or TestNG.

So in practice, what does this look like? Answer: an orgy of generics and anonymous class cruft. Consider the case of ArrayList, whose add(T) methods are not synchronized. Calling ArrayList#add across multiple threads eventually breaks on my machine. The Groovy version later is much terser syntactically, but the Java version,although workable, might be a little shocking:

Assert.assertSynchronized(
  new Callable<List<Callable<Void>>>() {
  
    public List<Callable<Void>> call() throws Exception {
      final List<Object> unsafeObject = new ArrayList<Object>();
      final Callable<Void> unsafeInvocation = new Callable<Void>() {
      
        public Void call() throws Exception {
          for (int x = 0; x < 1000; x++) {
            unsafeObject.add(new Object());
          }
          return null;
        }
      };
      return Arrays.asList(
        unsafeInvocation,
        unsafeInvocation
      );
    }
  }
);

Starting from the bottom up... this attempts to add 2000 objects to an ArrayList on two different threads: 1000 on each thread. And assertSynchronized will do this all 1000 times by default. This fails roughly 40 out of 1000 times on my machine. The key to understanding assertSynchronized is to know that the task factory will be invoked 1000 times, so shared state needs to be instantiated within the task factory and not within the individual task.

A good IDE makes all these anonymous classes easy to write, but it's hard to deny that the Groovy version from the JConch examples is more elegant:

Assert.assertSynchronized(
  {
    def target = new Vector()   //fail if ArrayList 
    return [
      { (0..1000).each { target.add(new Object()) }  }  as Callable, 
      { (0..1000).each { target.add(new Object()) }  }  as Callable, 
    ]
  } as Callable
)

So the Twin Cities OTUG group started a concurrency SIG. I added assertSynchronized to JConch as preparation for this, hopefully it helps you out. Consider joining in the discussion here. I'd love to flesh out the JConch concurrent unit testing support more, so email any suggestions to me or leave a comment. I'd love to see what other people are doing for concurrent testing!