Goals of Test Automation

I'm slowing working my way through xUnit Test Patterns by Gerard Meszaros. I have mixed feelings about the book in general, but I came to appreciate the material more this afternoon while talking with co-workers.

We have some issues with our unit tests; specifically they are hard to understand and maintain. There are a number of contributing factors, most of which revolve around an abstract subclass of TestCase that both runs its own tests and also contains template methods to exercise other code. Everyone agrees on the problem but each person has their own preferred approach to fixing it.

I was reminded of the Project Goals and Test Goals set forth in xUnit Patterns, a snippet of which follows:

Each preferred approach on the team addressed a different project goal, but the team had never openly discussed the unit test goals. We were stuck with a bunch of good ideas, yet no way to judge them against each other because we had never defined our judgment criteria. Hopefully, reviewing these goals as a team will provide a better backdrop for decisions on how to refactor the tests.

The xUnit Patterns book has its flaws, and early on I felt disappointed with it. However, as time passes I find myself more frequently using terminology from it and using it as a reference. I wasn't immediately impressed with it, but it might end up being a really great book.

Update: Using the project goals to shape the discussion worked wonderfully. We started by reviewing the goals at the beginning and then came back to them every time the discussion got too off track. We ended up ranking our goals as 1) Safety Net, 2) Easy to Write/Maintain, and 3) Defect Localization. Over time I see #2 being a contentious issue because it can be interpreted as either DRY (do not repeat yourself) or an acceptable level of duplication in test code. In the end I don't quite see a solution beyond allowing the team as a majority to decide and then accepting that some team members will dislike some of the unit tests. Not much of a solution, that.