Hijacking Goto Labels: Peek Inside the Spock Framework

The Spock testing and specification framework was released a few months ago, along with some wiki pages and surprisingly good documentation showing its usage.

The homepage greets you with a somewhat cryptic HelloSpock example:

def "can you figure out what I'm up to?"() {

  expect:
        name.size() == size

    where:
        name << ["Kirk", "Spock", "Scotty"]
        size << [4, 5, 6]
}

Beyond the question of whether or not this is an innovation in testing, don't you wonder how the heck this works? Is that even a valid method name? Are those goto labels in there? Whaaaaat?

Ignoring momentarily what makes it work, the 'expect:' block is going to be executed 3 times in this example: once with (Kirk, 4), once with (Spock, 5), and once with (Scotty, 6). You sort of have to read the 'where:' block horizontally to quickly see how the name and size variables relate to each other. Also, the variables name and size are somehow defined in the 'where:' block and then safely referenced from the 'expect:' block. So you're not only reading horizontally but also a little bottom to top. Maybe it'll help to rotate your monitor.

Oftentimes, javap is a good starting point to understand how things work. Here is the javap output (cleanup up a little) of the HelloSpock example:

$> javap HelloSpock

Compiled from "HelloSpock.groovy"
public class HelloSpock extends java.lang.Object implements groovy.lang.GroovyObject{
    ...
    public void can you figure out what I'm up to?();
    public void setup();
    public java.lang.Object __feature0(java.lang.Object, java.lang.Object);
    public java.lang.Object __feature0prov0();
    public java.lang.Object __feature0prov1();
    public java.lang.Object __feature0proc(java.lang.Object, java.lang.Object);
    ...
}

I, for one, had no idea that question marks and spaces were legal method name characters in Java class files. But 'can you figure out what I'm up to?' in the class file proves it must be! So the specification method from the example is compiled to a method, and the class picks up a void setup() method... this isn't too surprising. There are a few other new mystery methods all starting with "__feature0". We'll get to those later. A bigger mystery is how the 'can you figure out what I'm up to?' method works at all. Check out how it looks in the AST Browser:

The first thing you'll notice from the AST Browser is that I'm using Windows. Yeah, I'm lame. Get over it. The second thing you should notice is that there is hardly any code in the method! Where did it all go? All that's there is a single static method call. In Groovy code, the method body would be written as:

void "can you figure out what I'm up to?"() {
    org.spockframework.runtime.SpockRuntime.featureMethodCalled()
}

If you have trouble translating the AST into Groovy code then take a second look at the AST Browser and imagine how it might work.

Obviously, the code being executed is no longer in the method body where we first defined it. So where is it? The answer is those "__feature" methods. Check out what the debugger teaches you when the breakpoint is set within the 'expect:' block:

Remember the javap output, and how the __feature0 method took two parameters of type object? Well the 'expect:' breakpoint stops in "__feature0" (as witnessed by the current frame) and there are two variables in scope: name (Kirk) and size (4). As you'd expect, __feature0 is invoked 3 times during this test run, once for each name/size pair. We can verify that name and size are method parameters of __feature0 by looking at the AST for the MethodNode:

The two green parameter nodes show that name and size are passed as parameters to the method. If you experiment with the 'where:' block and add a new variable declaration, then you'll see the arity of the method increased by one to __feature0(Object, Object, Object). Kinda makes you wonder what the maximum number of method parameters is? I little expirement proves 255 is the answer, and if you hit this error then PLEASE FOR GOD'S SAKE REFACTOR YOUR TEST.

Hopefully, how Spock works is beginning to take shape. In the HelloSpock example, Spock is synthesizing 4 methods. The 'expect:' block is transformed into a runnable assertion method named __feature0 and the 'where:' blocks are accumulated and iterated, passing the test data into the assertion method one at a time. Using the debugger, putting a breakpoint in the 'where:' block reveals some interesting information. The name << ["Kirk", "Spock", "Scotty"] line of code is executed in the method __feature0prov0 and the size << [4, 5, 6] line is executed in __feature0prov1, despite being adjacent in the source file! Time to resort back to the AST Browser: The synthesized _feature0provN methods are simple one line methods returning the right-hand-side of the 'where:' block variable declarations. Hmmm. The last piece of the puzzle is to find out what the __feature0proc(java.lang.Object, java.lang.Object) method does. There isn't much point in looking at the AST... it seems to be an intermediary that binds the size and name variables into something usable. You can discover that the method parameters are named p0 and p1, and then use a little creative logging to print out the values at runtime:

where:
    ignored = println ("where: \n p0=$p0\n p1=$p1")
    name << ["Kirk", "Spock", "Scotty"]
    size << [4, 5, 6]

If you're paying attention, you'll realize that the name<< is the first line to execute, size<< the second, println the third, and the the 'expect:' block last. Executing top to bottom? That's a pretty good trick. As for the logging, perhaps there is a way better way to do this... but you can't just put a println in the 'where:' block because Spock will try to bind it as a variable. Luckily, there aren't really void methods in Groovy; methods just return null instead. So setting ignored equal to a println result is totally valid, as long as you're willing to put up with a null 'ignored' variable in scope for your 'expect:' block. Anyway (p0, p1) is (Kirk, 4) as you'd expect, along with the other combinations later.

So that's it. That's how the goto hijacking of Spock specifications works. The method called "can you figure out what I'm up to?" is almost totally discarded and replaced within the test runner. Each 'where:' block variable is transformed into a separate method call named __feature0__provN, and the 'expect:' block is transformed into a method that takes the input generated and runs the assertions. The __feature0proc method worries about moving data from the 'where:' blocks into a format the 'expect:' block can understand.

That is pretty cool. I wonder if it actually helps you write better tests.