Showing posts with label mocks. Show all posts
Showing posts with label mocks. Show all posts

Wednesday, July 15, 2009

Patterns of Enterprise Cruft: Generic Object Factory

Hmmm, what to do about unit testing a production component that clearly needs to call an object's dependency laden constructor...

My last post detailed the subclass and override approach, which is a simple, non-intrusive mechanism to allow you to mock out constructor calls. An alternative that a lot of people reach for is to create some sort of generic object factory that provides a layer of indirection across all constructor calls. This sounds like a decent enough idea. Let's see how far we can take it before things go wrong.

As context, imagine a system of legacy business objects with persistence baked into the interfaces. This means that persistence and domain objects are not separate and there is no entity manager... a pretty common legacy scenario in my experience.

An example document-style service to work with User objects would just move data from DTO objects into the database:

public class UserService {

private DataSource datasource;

public void create(UserDTO input) {
User user = new User(datasource);
user.setFirstName(input.getFirstName());
user.setLastName(input.getLastName());
user.save();
}

// read, update, and delete methods omitted
}

A naive indirection to provide would be to extract the constructor call to "new User(datasource)" into a separate UserFactory object, which could then be mocked out for testing. The problem is that the 1-to-1 correspondence of domain objects to factory objects doubles the number of classes in your system. This abundance of classes is fine if your entire domain consists of about five objects. But if that's the case you don't really have an enterprise, do you? You should just refactor the objects to not couple the domain and persistence and stop reading these blog posts. But if your domain is 250 objects, creating 250 factories starts to look like a hassle.

No, clearly we need a generic factory that can furnish your services with any type of domain object that is requested. Something like a Registry object from Fowler's PEAA, but just to dole out instances of new objects. If the service were refactored with this factory in mind, then testing is as simple as mocking out factory in the example below:
public class UserService {

private ObjectFactory factory;

public UserService(ObjectFactory factory) {
this.factory = factory;
}

public void create(UserDTO input) {
User user = factory.make(User.class);
user.setFirstName(input.getFirstName());
user.setLastName(input.getLastName());
user.save();
}
}

You can use a mock object framework like Mockito to help you in this endeavor. Mockito is nicer than some other frameworks because it allows you to verify individual methods being invoked rather than all interactions with a particular object. This lets your test methods follow William Wake's arrange-act-assert sequence and makes your tests more understandable:
    import static org.mockito.Mockito.*;

// arrange
User user = mock(User.class);
ObjectFactory factory = mock(ObjectFactory.class);
when(factory.make(User.class)).thenReturn(user);

// act
UserService service = new UserService(factory);
service.create(new UserDTO("fname", "lname"));

// assert
verify(user).setFirstName("fname");
verify(user).setLastName("lname");
verify(user).save();

Easy. And the best part of all this is that ObjectFactory is fun to write. You get to use generics and reflection! Wheee! It's as simple as internally holding a map of interface types to implementation types and reflectively calling a standard constructor when one of the objects is requested:
public class ObjectFactory {

private Map<Class, Class> classToType = new HashMap<Class, Class>() {{
put(User.class, User.class);
}};

private DataSource datasource;

public <T> T make(Class<T> clazz) {
try {
Class type = classToType.get(clazz);
if (type == null) throw new RuntimeException("Type not found: " + clazz);
Constructor constructor = type.getConstructor(DataSource.class);
if (constructor == null) throw new RuntimeException("Constructor not found: " + clazz);
return clazz.cast(constructor.newInstance(datasource));
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}

I've officially seen 3 variations of this implementation, but they're all pretty similar. As long as you have a standard constructor then this approach works. If by working you mean you can get test coverage using mock objects and verifying side effects.

What happens when your objects don't share a common constructor and dependencies? Spring to the rescue! You can define a specific Spring factory for your domain objects and then the ObjectFactory can retrieve new prototyped-scoped beans by type whenever one is requested. The implementation is a little simpler in Java but hiding behind it is the complexity of the XML configuration:
public class SpringObjectFactory {

private XmlBeanFactory factory = new XmlBeanFactory(
new ClassPathResource("objectFactory.sping.xml")
);

public <T> T make(Class<T> clazz) {
Map<String,T> beanMap = factory.getBeansOfType(clazz);
if (beanMap.isEmpty()) throw new RuntimeException("No beans found of type: " + clazz);
if (beanMap.size() > 1) throw new RuntimeException("Too many beans found: " + beanMap.keySet());
return beanMap.values().iterator().next();
}
}

This is kinda nice. So why is it a Pattern for Enterprise Cruft?

What was the root problem again? Oh yeah, the persistence layer was baked into the domain layer, they were not separate.

Does a generic object factory solve this problem? Clearly not. It only solves how to test a system written like this.

Does a generic object factory drive us closer to solving this problem? By most accounts, the solution is removing persistence from the domain objects and placing it in some sort of entity manager. An entity manager would handle saving the domain objects but not creating them. In the future, it would be the entity manager that would be mocked out not the domain object constructor itself. So no, a generic object factory is not going in the right direction. All of out unit tests rely on mocking the domain object instantiation logic, not the save logic! In the future, the instantiation of the objects isn't going to change, but the saving will. When it comes time to create a DAO layer we'll have hundreds of fragile mock-sensitive unit tests that are worthless and need to be rewritten. A better solution is to leave the constructors alone and introduce a generic save mechanism, not a generic construction mechanism. That would be a design that is a step close to a DAO layer.

Is a generic object factory a good short term solution to getting higher test coverage? If your problem is low test coverage, then this approach can fix that. But what was the root problem again? Persistence, not testability. So consider the alternatives. Subclass and override is simpler to the production code at the cost of having extra subclasses in the test tree. However, you can get around this by testing with partial mocks, and the technique at least does not move you away from a more appropriate solution.

A bigger risk of introducing a generic object factory is death by a thousand clever ideas. It's not your object factory that is so complex. It's that there is probably another one from the team down the hall. And a third tucked inside another module somewhere. And pretty soon you have a 100 different points of cleverness to deal with, and all those clever spots aren't really solving your problem, they're just getting you to (hopefully) hit a delivery date and be proud of your coverage. As far as I can tell, a whole bunch of "Enterprise Complexity" lies in accumulated clever solutions that solve problems one layer away from the root cause. So, I'll ask one last time: What was the root problem? Beware solutions that don't address the answer to this question.

And welcome to the Enterprise!

Saturday, April 5, 2008

Testing Java from Groovy? 2 Misconceptions

The Groovy language has a great story around unit testing... the built in MockFor and StubFor objects demo so well, dynamic typing lets you pull some clever run time tricks, and the Java interoperability means you can test your Java code from Groovy. From time to time I also hear that Groovy unit tests on Java code are an easy way to get Groovy into your organization. Given that unit testing is easier in Groovy (it is), and Groovy interops great with Java (it does), it should be a no-brainer to convince the authorities that dropping in groovy-all.jar is a good thing, right?

Well, when it came time for me to sell this idea to my work, my argument fell flat. The problem was that my two biggest reasons for advocating Groovy testing turned out to be false. Let's look at each in turn.

Groovy lets you mock out static method calls
Legacy codebases are often plagued by static dependencies in the form of static method calls. But in Groovy you're allowed to mock these out! Awesome, you can actually intercept a static method call to MyService.makeDatabaseCall() and provide a mock response. This is huge for me. Consider an example of a static service (written and compiled in Java!) providing file sizes for String based filenames:

import java.io.File;

public class JavaService {
static long getFileSize(String path) {
File file = new File(path);
return file.length();
}
}
And let's say the class you want to test makes a static call to this method...
class GroovyClient {
def filename;

public long getFileSize() {
return JavaService.getFileSize(filename);
}
}
We have a Groovy class we want to test making a static call to a Java object. Untestable in Java, easily mocked in Groovy:
def client = new GroovyClient(filename: "shortFile.txt")   

def mock = new MockFor(JavaService)
mock.demand.getFileSize("shortFile.txt") { -1 }
mock.use {
assert -1 == client.getFileSize()
}
This is huge, it means static dependencies no longer results in untestable code! Eh, not quite. This works because the object making the static call is written in Groovy. If the object making the call is written in Java (as my production code is) then the static call can't be intercepted. So if you make the "GroovyClient" above into a "JavaClient" the test will fail. Grrr.

This is no secret; nobody told me this would work. In fact, someone probably told me it would not work and I just wasn't listening. That's how misconceptions arise, I suppose.

So... it turns out that mocking out statics isn't a good argument for testing in Groovy, but it is a good argument for writing production code in Groovy (admittedly a harder sell).

Groovy lets you mock out the new operator
If you stop to think about it, you'll realize I'm just repeating myself here. But just in case you don't want to think...

Remember how our JavaService performed a "new File()" operation? Well, you can mock that out too! This is a great way to get inside your objects being tested and break some of those static dependencies:
def mock = new MockFor(File)         

mock.demand.length() { longFileSize }
mock.use {
assert longFileSize == GroovyService.getFileSize("shortFile.txt")
}
Sweet, Groovy lets you mock out new calls. Holla! And it doesn't work if the new is being performed by Java. What is new, anyway? In some languages new is just a class method, the same way that any of your other static methods are class methods. In Java, it's a little more special than that, but you can still think of it as a static method call. All the reasons you can't mock out a static call in Java apply to trying to mock out new.

Again, Groovy is a great choice for writing unit tests, but works best when the production code is also Groovy.

So what's it good for?
Despite discussing two reasons not to write unit tests for Java code in Groovy, I still do. These two posts hit the nail on the head: complex object creation is much simpler in Groovy than Java. In Groovy, my setUp code is shorter and simpler. Properties on the constructor makes object creation easy. Using a map to implement an interface makes stubs easy. Closures, collect() and inject() make working with large test data sets trivial. Builders simplify object graph creation. When testing Java from Groovy, it's all about how nice your setUp code becomes. That's the most convincing argument to me!

And what about you? Why do you test Java from Groovy? Or why not?

(And the big unanswered question of the post is why doesn't this work. I could look at the documentation and search the mailing list to find an answer, but it's Saturday morning, the weather is beautiful, and I'm going out to run some parkour. Yes, that's me in the blue track suit). Enjoy Spring, I say.

Sunday, March 23, 2008

Mockito - The New Mock Framework on the Block

xUnit Patterns by Gerard Meszaros brought the idea of Test Spies to the front of my mind last Fall. I'll summarize my previous post by saying there are stubs and mocks, which most people are aware of, but there are also Test Spies. They lie between stubs and mocks in terms of complexity, and are characterized by having many features of mocks without the record/playback baggage that comes with the most popular mock frameworks. In my humble opinion, my previous post on the topic is worth reading.

When you're writing your own hand-rolled mock objects, test spies are an easy, logical extension beyond a simple stub. If your system under test is calling a void method, foo( ), and you want to make sure that method is called, then you write a wasFooCalled( ) method on the mock and verify it in your test. Simple and effective.

Let's consider the near-canonical example of an Order object (a product ID and a quantity) and a Warehouse object which can fill orders.

public interface Warehouse {
boolean remove(String id, Integer quantity);
}

public class Order {

private final String id;
private final Integer quantity;

public Order(String ID, Integer quantity) {
this.id = ID;
this.quantity = quantity;
}

public boolean fill(Warehouse warehouse) {
return warehouse.remove(id, quantity);
}
}

The Order.fill( ) method collaborates with the Warehouse object, and our test should verify that interaction. A hand rolled mock of Warehouse might look like this:

class WarehouseMock implements Warehouse {
boolean wasCalled = false;

public boolean remove(String id, Integer quantity) {
wasCalled = true;
return true;
}

boolean removeWasCalled() {
return wasCalled;
}
}

And the Order unit test might look like this:

WarehouseMock warehouse = new WarehouseMock();

Order order = new Order("TALISKER", 50);
order.fill(warehouse);

Assert.assertTrue(
"Warehouse must have remove() called",
warehouse.removeWasCalled("TALISKER", 50));

Enter Mockito, a new test spy framework that takes the drudgery out of hand rolling your own "wasXcalled" methods. Here is the same test with Mockito:

Warehouse warehouse = Mockito.mock(Warehouse.class);
Order order = new Order("TALISKER", 50);
order.fill(warehouse);

Mockito.verify(warehouse).remove("TALISKER", 50);

Those familiar with record/playback frameworks like EasyMock will notice the lack of a set expectations step or a replay step. This is still a very simple test. You mock out warehouse, you execute some methods on the class under test, and then you verify that the mock was used correctly. Compare this to the EasyMock version:

Warehouse warehouse = EasyMock.createMock(Warehouse.class);
EasyMock.expect(warehouse.remove("TALISKER", 50))
.andReturn(true)
.once();
EasyMock.replay(warehouse);

Order order = new Order("TALISKER", 50);
order.fill(warehouse);

EasyMock.verify(warehouse);

Mockito really cleans up the unit test by not requiring expectations. Personally, I much prefer the Mockito API to the EasyMock API for that reason. Now, there are still times when you need to set expectations. Mockito works great if you're making sure a void method gets called (testing indirect output), but often you need to supply indirect input to your class via the mock, and in those cases you really do want to set some expectations. Since Mockito is a fork of EasyMock, you still have the option to do this if you choose. Feel free to set expectations when you want, you just aren't forced to in all cases.

Mockito has a couple other nice features too, like being able to mock classes, a good attempt at producing better error messages, and some annotation magic for mock instantiation. As the author, Szczepan Faber says, "Hats down before EasyMock folks for their ideas on beautiful and refactorable mocking syntax." It's a great framework that has served me well for years, but I for one am switching to Mockito for the time being. Great work, everyone involved!

Sunday, October 21, 2007

Mocks and Stubs aren't Spies

At this point, we all know the difference between mocks and stubs... right? Well, perhaps not. That's OK, I'll try to explain it. And if I do a poor job you can always go read the article. (I've tried to have these samples follow Fowler's samples so that the two articles can be read together easily).

So in unit tests, when your system under test requires a collaborator you provide it with a dummy implementation that does nothing, like so.
public void testFillingOrder() {
Warehouse dummyWarehouse = new Warehouse() {
public void remove(String product, int quantity) {
//overridden so database query is NOT executed
}
}
};
Order order = new Order("TALISKER", 50);
assertTrue("order must be filled", order.fill(dummyWarehouse);
}
It isn't simple, but it isn't very complex either. Basically, the system under test requires that a warehouse be present, but beyond that it doesn't really care that the warehouse does anything. Test code gets more complex when your system under test requires reading certain values from a collaborator. You then have to provide it with a stub implementation that returns that value, like so (notice how the remove() method returns a value).
public void testFillingOrder() {
Warehouse warehouseStub = new Warehouse() {
public boolean remove(String product, int quantity) {
//overridden so database query is NOT executed
return true;
}
}
};
Order order = new Order("TALISKER", 50);
assertTrue("order must be filled", order.fill(dummyWarehouse);
}
This example makes the difference look pretty simple, but in real unit tests stubs are a lot more complex than dummy objects because you usually need a way to modify the return value on the warehouse. It's still fairly easy to follow though, and probably doesn't represent a maintenance burden. It starts to get really complex when your system under test requires that certain methods on a collaborator get called (possibly in a certain order). Then you need to use a mock that can record how it is used and be verified later on.
public void testFillingRemovesInventory() {
//setup - data
Order order = new Order("TALISKER", 50);
Warehouse warehouseMock = createMock(Warehouse.class);

//setup - expectations (really a part of verify)
expect(warehouseMock.remove("TALISKER", 50)).once();

//exercise
order.fill(warehouseMock);

//verify
warehouseMock.verify();
}
I tried to find a way to have the sound of a needle scratching a record play as you read that last code sample, but I couldn't figure out how to do it. Imagine one now. Why? For starters, this code is a lot more complex than a stub. As I wrote previously, recording, verifying, expecting, and playing back objects is not a language commonly used in testing. So there is that mental hurdle to jump. But do you know which line of code this test will most commonly fail on? The line that says verify(). I hope you like looking at stack traces. Or debugging. But until you have fairly extensive experience with a mock object framework, you're probably not going to know exactly what went wrong by simply looking at the error message. (Everyone I know has grappled with the problem of a mock object error message that reads something like "Failed: Expected 'result' received 'result'". For the uninitiated, this is a sign that your failure is being masked by a toString() method. Intuitive, huh?).

But my biggest issue with the mock test method is that it fails to follow the most common test structure of setup, exercise, and verify. Following this structure in your tests leads to tests as documentation and communication of intent. But mock objects do their verify just after setup, in a setup, verify, and exercise pattern. Despite the fact that the verify() method occurs after exercising the tests, to understand what the method is supposed to do you need to look several lines up in your test method. Breaking out of this pattern by using mocks has lead me to write some really hard to maintain and fragile unit tests.

Wouldn't it be easier if we didn't have to set expectations on mocks? If we could just assert that something happened at the end of the test? Like this:
public void testFillingRemovesInventory() {
//setup - data
Order order = new Order("TALISKER", 50);
Warehouse warehouseSpy = createSpy(Warehouse.class);

//exercise
order.fill(warehouseSpy);

//verify
assertTrue(warehouseSpy.removeWasCalledOnce("TALISKER", 50));
}

Using a Test Spy, as in this example, is a much simpler way to test how collaborators were used than creating a record/playback style mock. Moving the expectations to the end of the test method more clearly reveals the intent of what should occur. The test is also devoid of any record/playback language, which is accidental complexity and should be removed. A final complaint about mocks is that they lead to very fragile tests, in which a change in implementation leads to hard to diagnose failures in the unit tests. Extensive use of mocks has led me to write over specified and fragile unit tests.


Before moving on, I'd like to clarify and define some terms in use here, which I originally discovered in Gerard Meszaros' xUnit Patterns book.

  • A Dummy Object is a placeholder object passed to the system under test but never used.
  • A Test Stub provides the system under test with indirect input
  • A Test Spy provides a way to verify that the system under test performed the correct indirect output
  • A Mock Object provides the system under test with both indirect input and a way to verify indirect output

The still missing piece is a Test Spy framework that provides calling semantics like the spy.removeWasCalledOnce() method. This could be done in a dynamically typed language like Groovy using its metaprogramming facilities, or it could be done in Java using a more reflective style API. This would be hugely beneficial in my unit tests efforts. It would declare intent and document the system better, and unburden the unit tests from the problems associated with mocks.

In closing, when writing unit tests that require collaborators, consider that Mocks aren't a one size fits all solution. As always, figure out what you need first and then find the best tool for the job. And you can let this handy chart guide your decisions:



Watch this spot for more news on a Groovy Test Spy framework!

Monday, October 15, 2007

Is it really a domain specific language?

We have now reached the point where the term Domain Specific Language is so overused that it has become meaningless. And if not, then we are quickly approaching it. In my mind, this Fall's No Fluff conference will be remembered as "the one about DSL". But we need to slow down... I feel like standing up and shouting, "Not everything is a DSL!" Excel functions? Yes, great example. EasyMock's humane interface? Eh, perhaps. Ordering hashbrowns at Waffle House? umm... maybe this one has questionable value.

It seems that several terms are being confused, and we should be specific about different ideas. A domain specific language is a vocabulary for discussing a specific problem you have. Excel functions are a darn good vocabulary for working with spreadsheet data. They may not be easy to learn, but they are an extremely powerful language to solve the problem of expressing complex formulas in spreadsheets. A fluent interface (or humane interface, or literate interface) is not the same thing as a DSL! A fluent interface is an API design that allows users to work with concepts in a way that closely matches most Western sentence structure, such as:

receipt = order.2.lattes.andPay($10)
Compare this API to the traditional Java style API:
Order order = new Order();
order.setQuantity(2);
order.setItem(latte);
Receipt receipt = order.pay(new Currency(10, DOLLARS));
Which API is easier to read, the first or second? Yes, fluent interfaces are easier to read (you answered "first", right?). Hopefully your native language follows the same subject-then-predicate order of English, or else the code example may confuse you, but that's an aside. I do think fluent interfaces are a good idea: they're easy to read and easy to write, despite sometimes being harder to debug because of their multi-step nature.

But just because something has a fluent interface doesn't mean it is a DSL. EasyMock has a fluent interface, but it is not a DSL. Consider the following unit test code:

MyObject mock = new EasyMock.createMock(MyObject.class);
EasyMock.expect(mock.foo()).andReturn(10).once();
new SystemUnderTest().perform(mock);
EasyMock.verify(mock);
This code, if you're at all familiar, creates a mock object and tells it to return 10 when the foo() method is called, and only do it once. The system under test is then exercised and the mock is verified to have been called correctly. This is a fluent interface. The methods are chained together on the 2nd line and it is sort of easy to read.

Let's revisit my definition of a DSL: "a vocabulary for discussing a specific problem you have". Is EasyMock a DSL for unit testing? EasyMock's fluent interface provides a nice way to set and verify expectations on mock objects. But setting expectations was not my problem, unit testing collaborating objects was my problem. And the two are different! EasyMock uses a record/playback metaphor to create mocks, and its interface is a DSL for record and playback. Using EasyMock's API in my project didn't solve the problem of writing better unit tests, it created a problem of how to work with record/playback frameworks. Since this fluent interface does not provide a vocabulary for unit testing, it is therefore not a DSL.

So a very important part of a DSL is that it solves the problem you have, not just any problem. Let's consider the famous Waffle House ordering "DSL" of "scattered, covered, and smothered" (these are three different ways to order hashbrowns, by the way). Is this a DSL? What problem does it solve? If you wait tables at Waffle House then it is a terser than English vocabulary used to communicate with the cooks, and therefore a DSL. But if you are a customer at Waffle House, what possible problem does it solve? Can you not say the words "with cheese"? I say ordering at Waffle House is harder than it needs to be because we are forced to adopt the language of the cooks (the implementers). Have you ever ordered a Philly Cheese Steak in Philadelphia? If you don't say it right then you don't get your food. It's nerve wracking for the tourists! Imagine this one... imagine a restaurant where the cooks are Mexican. Can you imagine that? I've just described a ton of places you eat, trust me. Wait if you had to order in Spanish because that is the DSL of the cooks? Well, that's what you're doing at Waffle House. And that is what I'm doing with EasyMock every time I set an expectation.

My preference is to define a DSL in relationship to a person and their particular problem. A language might be a DSL for one person and not another. If you're an API designer, simply saying that you're going to wrap your implementation in a fluent interface is not the same thing as creating a DSL, and you might be exposing the wrong level of abstraction to your users. Creating mini-languages cannot be our goal. We need to first think hard about which problem we need to solve. And providing a nice, fluent interface at just the right level of abstraction can then be a beautiful thing.

Monday, September 24, 2007

Overriding final Java classes in unit tests using Groovy

As with a lot of legacy codebases, the code I work with on a daily basis is sometimes extremely hard to unit test. There are times where the scope and constraints of your changes don't allow you to refactor the code until it is testable. In fact, if there is low test coverage in the first place, refactoring the code is an extremely dangerous endeavor.

There are some known techniques for testing software that wasn't designed to be tested. In Java, you can place the test classes in the same package as the production classes and then call package and protected methods on the system under test without breaking the compiler's visibility rules. Or you can use reflection to examine and invoke methods on a system under test that are not visible, such as private methods. One method I like to use is to create a test specific subclass by subclassing the system under test and gaining visibility to state and behavior not available through its public interface. None of these strategies are pretty, and they all break some object oriented principles somewhere. Life is full of compromises. I may be willing to make some that you are not.

My problem is that I'm not allowed to subclass my production classes because they are all marked final (sealed in C#). Final methods may not be subclassed. This is done because of a rigid (blind?) adherence to Joshua Bloch's Effective Java rule of "Design for inheritance or prohibit it". Let's not consider right now whether this is actually a good thing. Instead, let's look at how you can break this rule by using Groovy to create a test specific subclass of a final class.

Start with a simple production class that is final... no subclass can be made:


public final class SystemUnderTest {
public String foo() {
return "Executing within SystemUnderTest...";
}
}

The following Groovy test case shows how to "subclass" this final class to alter the behavior of the final SystemUnderTest object. A GroovyTestCase is a lot like a JUnit TestCase, and I'll explain what is happening below the example.

import groovy.mock.interceptor.*

class OverridingFinalTest extends GroovyTestCase {
void test_Overriding_Final() {
// Mock out our class under test
def mock = new MockFor(SystemUnderTest)

// Demand that the getString method gets called
mock.demand.foo {
return "Executing within Groovy..."
}

// Use our mock for this block
mock.use {
def mocked = new SystemUnderTest()
assert mocked.foo() == "Executing within Groovy..."
}

def real = new SystemUnderTest()
assert real.foo() == "Executing within SystemUnderTest..."
}
}


The "new MockFor" statement is creating a mock object for the SystemUnderTest class. The mock.demand.foo block is redefining the behavior for the foo() method for that object. The mock.use block tells Groovy to use the mock object as a replacement for the actual SystemUnderTest within the use block. Any call to SystemUnderTest within the mock.use block will be filtered through our custom definition of the mock. So creating a new SystemUnderTest in the use block is actually creating an instance of our mock object, which delegates to an instance of the SystemUnderTest class... in this way our redefined foo() will execute the Groovy code when called, but any other method on the SystemUnderTest will be dispatched to the real SystemUnderTest.

As the test shows, invoking new SystemUnderTest outside the use block will result in not using the mock... and foo() method calls are dispatched to the real object, not filtered through our mock.

My real intent when starting this project was to create an object factory in Groovy that could return to me subclasses of final classes. But the Groovy mock never implements or extends the class it is mocking, it is just a proxy for that class. In the bytecode, the MockFor is not a SystemUnderTest object, so any attempt to return that object back to Java and use it as a SystemUnderTest will fail. What does this mean? You still can't mock out final classes in Java test cases, but it is entirely possible in Groovy test cases. Now if we could just get the architects to allow us to check in Groovy test cases...

Thank you to the Groovy Users of Minnesota and Jesse for helping me with the example.

Tuesday, February 13, 2007

EasyMock - Pros, Cons, and Best Practices

Last fall the EasyMock jar file was dropped into my code base so that we could start writing JUnit tests with it. EasyMock documentation is online if you want to know more.

The good news is that mocking out interfaces is very, very easy. I like the “Humane Interface” that it offers; always returning this from methods so that method chaining is easy.

EasyMock.expect(MyObject.message()).andReturn(true).once();

This makes writing EasyMock code much easier. I find EasyMock a great tool for mocking out interfaces where the System Under Test's (SUT) success/failure criteria is not dependent on the method calls to the collaborator (indirect output). If your mock objects are having get* methods invoked and you’re using JUnit asserts afterwards to assert proper state within the SUT, then you’re in EasyMock’s sweet spot. Also, don’t be afraid to write ArgumentMatchers. The first one is tough to write, but they are all basically the same after that. EasyMock has definitely sped up the writing of many of my unit tests.

I do have a list of complaints about EasyMock, though.

Only Works on Interfaces - EasyMock does not mock out classes, only interfaces. This has lead to an explosion of interfaces in my code base. The interfaces only exist as a means to using EasyMock. So test details are dictating what the production code looks like, and not in a good way. I call the proliferation of classes named *Impl the Interface/Impl Anti-Pattern. Four things can be done about this:

  1. Live with it and call it “test driven design”
  2. Upgrade to JUnit 4.0 and upgrade to the version of EasyMock that mocks classes
  3. Use Groovy script to mock classes
  4. Make collaborating classes non-final and write a test specific subclass

“Humane Interface” - The method chaining style interface is easy to write, but I find it difficult to read. When a test other than the one I’m working on fails, it’s often very difficult to determine what exactly is going on. I end up having to examine the production code and the test expectation code to diagnose the issue. Hand-rolled mock objects are much easier to diagnose when something breaks. Also, setting expectations for void methods is done differently than non-void methods. This results in code that mixes a humane interface with a classic API interface… which means my brain must process both styles at once, which is more difficult than either one alone. This problem is especially nasty after refactoring expectation code to reduce duplication. For the life of me, I cannot follow expectation code that has been refactored into shared methods. I now allow small amounts of duplicate expectation code to dwell in my unit tests. I need to see all the expectations at once, in one method, or I have no hope of understanding it.

Abstract Test Cases - Managing EasyMock within abstract test cases has proven to be very difficult. Managing replay and record states leads to a confusing mess. At this point I’ve given up mixing EasyMock and abstract TestCase objects. When something breaks it simply takes too long to diagnose. An alternative is to create custom assertion methods that can be reused. Beyond that, I've given up on Abstract TestCase objects anyway, on the grounds of preferring composition of inheritance.

Don’t Replace Asserts with Verify - The easiest methods to understand and test are methods that perform some sort of work. You run the method and then use asserts to make sure everything worked. In contrast, EasyMock makes it easy to test delegation, which is when some object other than the SUT is doing work. Delegation means the method’s purpose is to produce a side-effect, not actually perform work. Side-effect code is sometimes needed, but often more difficult to understand and debug. In fact, some languages don’t even allow it! If you’re test code contains assert methods then you have a good test. If you’re code doesn’t contain asserts, and instead contains a long list of EasyMock.verify() calls, then you’re relying on side effects. This is a unit-test bad smell, especially if there are several objects than need to be verified. Verifying several objects at the end of a unit test is like saying, “My test method needs to do several things: x, y, and z.” The charter and responsibility of the method is no longer clear. This is a candidate for refactoring.

All or Nothing Testing - With hand-rolled mock objects it is easy to target just a critical section of a method and ignore other parts. Maybe all I really care about testing is the first 4 lines of code and not the last 4. Once you use an EasyMock object you no longer get this flexibility. EasyMock objects are verified in entirety… there is no way to make it a strict mock for one method call and nice for all the others. You can’t verify like that either. I end up littering the test code with expectation-setting calls that I truly don’t care about. EasyMock could fix this by having NiceMocks try not to return null. For instance, if a NiceMock string returning method is called, then just have the NiceMock return “” instead of null. Use reflection to invoke some sort of default constructor on the return type if available. This would make NiceMocks a lot more useful.

Create/Replay Code Duplication - Since NiceMocks aren’t really that nice to work with (see aove item), I see the same 3 lines to create and replay an object littered across many test cases. At some point the cost of repeating this code outweighs what it would take to just hand-roll a nicely behaved mock object. My lesson learned is not to forget how to write your own mocks. EasyMock is not a replacement for all mock objects, and should be used only when it would reduce the amount of code written.

Error Messages - Is there a free framework that includes readable error messages? Every user who starts out gets confused by the message ‘Expected “Object X” but received “Object X”‘. It’s a lesson in object equality and toString for each new user. The error message relies on the toString() method while the comparison uses reference comparison. Be Warned.

While the list of complaints is longer than the list of accolades, EasyMock has made unit testing a lot easier. After using it a few months, I’ve reflected on the experience and create a list of Best Practices.

Easy Mock Best Practices

  1. Learn to use EasyMock and ArgumentMatchers. The learning curve is steep but over quickly.
  2. Upgrade to the newest version so you can mock out classes. Beware of creating interfaces simply for testing.
  3. Beware the Abstract TestCase. What makes sense to you at the time will confuse almost everyone else!
  4. EasyMock.verify() is a bad smell. Ask yourself if you really need it.
  5. Don’t throw out hand-rolled mock objects. They have their place.

Thoughts anyone?