Mockito - Pros, Cons, and Best Practices

Mockito - Pros, Cons, and Best Practices

It's been almost 4 years since I wrote a blog post called "EasyMock - Pros, Cons, and Best Practices, and a lot has happened since. You don't hear about EasyMock much any more, and Mockito seems to have replaced it in mindshare. And for good reason: it is better.

A Good Humane Interface for Stubbing
Just like EasyMock, Mockito allows you to chain method calls together to produce less imperative looking code. Here's how you can make a Stub for the canonical Warehouse object:

Warehouse mock = Mockito.mock(Warehouse.class);
Mockito.when(mock.hasInventory(TALISKER, 50)).
        thenReturn(true);

I know, I like a crazy formatting. Regardless, giving your System Under Test (SUT) indirect input couldn't be easier. There is no big advantage over EasyMock for stubbing behavior and passing a stub off to the SUT. Giving indirect input with mocks and then using standard JUnit asserts afterwards is simple with both tools, and both support the standard Hamcrest matchers.

Class (not just Interface) Mocks
Mockito allows you to mock out classes as well as interfaces. I know the EasyMock ClassExtensions allowed you to do this as well, but it is a little nicer to have it all in one package with Mockito.

Supports Test Spies, not just Mocks
There is a difference between spies and mocks. Stubs allow you to give indirect input to a test (the values are read but never written), Spies allow you to gather indirect output from a test (the mock is written to and verified, but does not give the test input), and Mocks are both (your object gives indirect input to your test through Stubbing and gathers indirect output through spying). The difference is illustrated between two code examples. In EasyMock, you only have mocks. You must set all input and output expectations before running the test, then verify afterwards.

// arrange
Warehouse mock = EasyMock.createMock(Warehouse.class); 
EasyMock.expect(
    mock.hasInventory(TALISKER, 50)).
    andReturn(true).once();
EasyMock.expect(
    mock.remove(TALISKER, 50)).
    andReturn(true).once();
EasyMock.replay(mock); 

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
EasyMock.verify(mock); 

That's a lot of code, and not all of it is needed. The arrange section is setting up a stub (the warehouse has inventory) and setting up a mock expectation (the remove method will be called later). The assertion in all this is actually the little verify() method at the end. The main point of this test is that remove() was called, but that information is buried in a nest of expectations. Mockito improves on this by throwing out both the record/playback mode and a generic verify() method. It is shorter and clearer this way:

// arrange
Warehouse mock = Mockito.mock(Warehouse.class);
Mockito.when(mock.hasInventory(TALISKER, 50)).
        thenReturn(true);

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
Mockito.verify(warehouse).remove(TALISKER, 50);

The verify step with Mockito is spying on the results of the test, not recording and verifying. Less code and a clearer picture of what really is expected.
Update: There is a separate Spy API you can use in Mockito as well: http://mockito.googlecode.com/svn/branches/1.8.3/javadoc/org/mockito/Mockito.html#13

Better Void Method Handling
Mockito handles void methods better than EasyMock. The fluent API works fine with a void method, but in EasyMock there were some special methods you had to write. First, the Mockito code is fairly simple to read:

// arrange
Warehouse mock = Mockito.mock(Warehouse.class);

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
Mockito.verify(warehouse).remove(TALISKER, 50);

Here is the same in EasyMock. Not as good:

// arrange
Warehouse mock = EasyMock.createMock(Warehouse.class); 
mock.remove(TALISKER, 50); 
EasyMock.expectLastMethodCall().once(); 
EasyMock.replay(mock); 

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
EasyMock.verify(mock); 

Mock Object Organization Patterns
Both Mockito and EasyMock suffer from difficult maintenance. What I said in my original EasyMock post holds true for Mockito:
The method chaining style interface is easy to write, but I find it difficult to read. When a test other than the one I'm working on fails, it's often very difficult to determine what exactly is going on. I end up having to examine the production code and the test expectation code to diagnose the issue. Hand-rolled mock objects are much easier to diagnose when something breaks... This problem is especially nasty after refactoring expectation code to reduce duplication. For the life of me, I cannot follow expectation code that has been refactored into shared methods.
Now, four years later, I have a solution that works well for me. With a little care you can make your mocks reusable, maintainable, and readable. This approach was battle tested over many months in an Enterprise Environment(tm). Create a private static method the first time you need a mock. Any important data needs to be passed in as a parameter. Using constants or "magic" fields hides important information and obfuscates tests. For example:

User user = createMockUser("userID", "name"); 
...
assertEquals("userID", result.id()); 
assertEquals("name", result.name(); 

Everything important is visible and in the test, nothing important is hidden. You need to completely hide the replay state behind this factory method if you're still on EasyMock. The Mock framework in use is an implementation detail and try not to let it leak.
Next, as your dependencies grow, be sure to always pass them in as factory method parameters. If you need a User and a Role object, then don't create one method that creates both mocks. One method instantiates one object, otherwise it is a parameter and compose your mock objects in the test method:

User user = createMockUser(
    "userID", 
    "name", 
    createMockRole("role1"), 
    createMockRole("role2")
); 

When each object type has a factory method, then it makes it much easier to compose the different types of objects together. Reuse. But you can only reuse the methods when they are simple and with few dependencies, otherwise they become too specific and difficult to understand.
The first time you need to reuse one of these methods, then move the method to a utility class called "*Mocker", like UserMocker or RoleMocker. Follow a naming convention so that they are always easy to find. If you remembered to make the private factory methods static then moving them should be very simple. Your client code ends up looking like this, but you can use static imports to fix that:

User user = UserMocker.createMockUser(
    "userID", 
    "name", 
    RoleMocker.createMockRole("role1"), 
    RoleMocker.createMockRole("role2")
); 

User overloaded methods liberally. Don't create one giant method with every possible parameter in the parameter list. There are good reasons to avoid overloading in production, but this is test. Use overloading so that the test methods only display data relevant to that test and nothing more. Using Varargs can also help keep a clean test.
Lastly, don't use constants. Constants hide the important information out of sight, at the top of the file where you can't see it or in a Mocker class. It's OK to use constants within the test case, but don't define constants in the Mockers, it just hides relevant information and makes the test harder to read later.

Avoid Abstract Test Cases
Managing mock objects within abstract test cases has been very difficult for me, especially when managing replay and record states. I've given up mixing mock objects and abstract TestCase objects. When something breaks it simply takes too long to diagnose. An alternative is to create custom assertion methods that can be reused. Beyond that, I've given up on Abstract TestCase objects anyway, on the grounds of preferring composition of inheritance.

Don't Replace Asserts with Verify
My original comments about EasyMock are still relevant for Mockito: The easiest methods to understand and test are methods that perform some sort of work. You run the method and then use asserts to make sure everything worked. In contrast, mock objects make it easy to test delegation, which is when some object other than the SUT is doing work. Delegation means the method's purpose is to produce a side-effect, not actually perform work. Side-effect code is sometimes needed, but often more difficult to understand and debug. In fact, some languages don't even allow it! If you're test code contains assert methods then you have a good test. If you're code doesn't contain asserts, and instead contains a long list of verify() calls, then you're relying on side effects. This is a unit-test bad smell, especially if there are several objects than need to be verified. Verifying several objects at the end of a unit test is like saying, "My test method needs to do several things: x, y, and z." The charter and responsibility of the method is no longer clear. This is a candidate for refactoring.

No More All or Nothing Testing
Mockito's verify() methods are much more flexible than EasyMock's. You can verify that only one or two methods on the mock were called, while EasyMock had just one coarse verify() method. With EasyMock I ended up littering the code with meaningless expectations, but not so in Mockito. This alone is reason enough to switch.

Failure: Expected X received X
For the most part, Mockito error messages are better than EasyMock's. However, you still sometimes see a failure that reads "Failure. Got X Expected X." Basically, this means that your toString() methods produce the same results but equals() does not. Every user who starts out gets confused by this message at some point. Be Warned.

Don't Stop Handrolling Mocks
Don't throw out hand-rolled mock objects. They have their place. Subclass and Override is a very useful technique for creating a testing seam, use it.

Learn to Write an ArgumentMatcher
Learn to write an ArgumentMatcher. There is a learning curve but it's over quickly. This post is long enough, so I won't give an example.

That's it. See you again in 4 years when the next framework comes out!