behind the times

The Best Groovy Inspections You’re Not Using

2012-01-18T07:51:00.002-06:00

Many, many tools perform static analysis on code. There are all sorts of automated ways to look through your code and tell you if there are likely errors or not. FindBugs, PMD, and CheckStyle are some of the big names from the Java world. On the Groovy side we have two real options: IntelliJ IDEA and CodeNarc.

This post (over on the Canoo Blog) highlights my favorite static analysis rules for Groovy in IDEA that are not enabled by default. Groovy in IDEA 11 has well over 100 rules, but less than half of them are activated when you install the product. To turn these rules on you’ll need to go into Settings (Ctrl+Alt+S) and enable them under Inspections.

So surf over to the main post and check it out. If you want to leave a comment, then do it on the DZone page. I've been turning comments off on the main posts.

If you like this then you might check out some of my other IDEA related posts: The 10 Best Inspections You’re Not Using (in Java), or the IDEA archive on my blog and the Canoo blog.

And remember if you need Groovy and Grails help, then give me a call or drop me an email at hamlet.darcy@canoo.com

Android Testing in IntelliJ IDEA

2012-01-12T07:35:00.001-06:00

Google’s Android site has some fairly detailed instructions for testing Android applications… from Eclipse. They were nice enough to supply a “Testing from Other IDEs” page, but that is nothing more than instructions on using Ant and the command line. Well, if you are using IntelliJ IDEA then you already believe the IDE is going to be a better tool than Ant for this. It’s easy to set up a test project in IDEA and get your tests running. Here are some simple instructions.

As usual, the full post is over on the Canoo website. Wanna be cool about it? Upvote it at DZone or Reddit.

Thanks for paying attention.

IntelliJ IDEA 11 for the Groovy Developer

2011-12-20T03:57:00.002-06:00

IntelliJ IDEA 11 was released a few weeks ago, and it contains quite a few new features for the Groovy developer. Everything listed in my blog post is in the free and open source Community Edition of IntelliJ IDEA. There are plenty of new Grails features as well, but I wanted to separate out the Ultimate Edition features into a different post.

To read the full post, surf on over to the Canoo Blog: http://www.canoo.com/blog/2011/12/20/intellij-idea-11-for-the-groovy-developer/

And, as usual, you can vote for this thing over at DZone.

Thanks everyone.

The Art of Groovy Command Expressions in DSLs

2011-12-08T03:09:00.001-06:00

Domain Specific Languages (DSLs) are often littered with the accidental complexity of the host language. Have you ever seen a supposedly “friendly” language expression like “ride(minutes(10)).on(bus).towards(Basel)”. The newest version of Groovy contains a language feature that aims to eliminate the noise of all those extra periods and parenthesis, so that your DSL looks more like “ride 10.minutes on bus towards Basel”. This article shows you step-by-step how to use Groovy Command Expressions and plain old metaprogramming to write just this DSL, and also offers advice on when, and when not, to use this new language feature.

The full article is available over on the Canoo Blog: http://www.canoo.com/blog/2011/12/08/the-art-of-groovy-command-expressions-in-dsls/

And of course you can upvote in all the usual places.

Until next time...

Global Day of Code Retreat Comes to Switzerland

2011-11-24T13:51:00.001-06:00

Next Saturday, 3rd December, Canoo is sponsoring the Swiss installment of the Global Day of Code Retreat. It's being held in Lugano, which is easily reachable from Switzerland or Northern Italy. The event is free, lunch is provided, and you'll get a chance to practice your programming skills while taking part in a world-wide event that criss-crosses the globe. Sound fun? It will be. Here's what you need to know:A code retreat is a day long event where programmers get to practice and hone their craft. The format was created three years ago and has been used and improved regularly since then. During the day we'll use Conway's Game of Life to practice our design, testing, and pair programming skills. The event has a predefined format and will be facilitated by Canooie Hamlet D'Arcy. This is quite different from our Hackergartens, so you may want to read up on the format so you know what to expect. Michael Hunger has an excellent synopsis of Code Retreat up on the InfoQ website.The originator of the idea is Corey Haines and he will be facilitating the first code retreat of the day in Australia and then flying to Hawaii to also facilitate the last retreat of the day. There is most likely a code retreat in your area, just in case you can't make it to Lugano. Check out the map to see where you can go.There are several sponsors for the Code Retreat in Lugano: Canoo, Ex Machina, and JetBrains to name a few. If you show up then expect to leave with some goodies as well as the free lunch. And please, if you plan on coming then be sure to register on the website so we can provide enough food and coffee.See you Saturday!

Android and IDEA for the Eclipse Refugee

2011-10-21T06:41:00.000-05:00

For the full article jump to: http://www.canoo.com/blog

Earlier this month I switched from writing my Android projects in Eclipse to writing them in IntelliJ IDEA. Overall the experience has been great, and I much prefer using IDEA to Eclipse for Android development. And now that IntelliJ IDEA 11 EAP (Early Access) has a visual layout window, there is almost no reason for me to write my Android apps in anything else. I wrote this post to help other users along their way when converting between the IDEs.

The full article with all the screenshot glory is over on the Canoo Blog

If you want to be a bro and slip me an upvote, then please do: Hackernews, Reddit, or DZone

Should Static Type-Checked Compilation Come to Groovy?

2011-08-01T23:21:00.009-05:00

Please note: As always, I speak for myself alone and not as a representative for any project or company. These are my opinions and reading of an ever-changing situation.

Update: This was a pretty stupid thing to post, because everything is in flux and these are just some thoughts.

Why Should Groovy have a Static Mode?

There were some internal discussions about "Grumpy mode" among some of the Groovy committers. I had a lot of reservations about introducing static-typing or static-mode into Groovy. When I thought about the idea I come up with many unanswerable questions. Is it a good idea or not? Only time will tell. Here are some of the questions that I wish I had answers to.

What exactly do users want?

Do people want a static language with a dynamic backdoor, such as the way C#/.NET has a dynamic keyword or the way you can use Groovy++ by default with the .gpp file extension and then sprinkle in @Dynamic annotations? This is one approach, where the core of your language is statically typed but you have an option to escape into dynamic-land. Or do people want a dynamic language with a static backdoor, which Groovy will be (as far as I know) one of the few examples of. This use case is the same as adding an @Typed keyword to your Groovy code to enable static compilation, or adding an @Grumpy keyword. Or do people just want better performance from Groovy, end of story?

If you want a statically typed language without the verbosity of Java then use Scala. Yes, Scala is complex. But Groovy is just as complex. Does anyone really know Groovy without reading a few books? Yes you can get things done, but understanding metaprogramming, understanding method dispatch, and understanding all the closure options (delegates, delegate-first, trampoline) is just as complex. I am excited about Kotlin and think it looks like a great statically-typed language, but at this point it is nothing more than a slidedeck and a wiki for anyone that isn't on the IntelliJ IDEA development team. It's not available to anyone.

I honestly don't think a Grumpy mode will boost my productivity much. Will it aide in language adoption? I say probably not. In my opinion there are three large barriers to Groovy adoption in the mainstream: performance, tooling, and bugs. Not being statically typed is a connoisseurs criticism, and connoisseurs generally chase after things that are interesting rather than good (it's been said about me as well). Groovy is a language that appeals to people that just want to get something done; it's about productivity not about programming language theory. Adding static typing might please some high-end connoisseurs if it is done in a good way, but I have trouble believing it will please many real users. Search the web for "Groovy sucks". You won't find pages about dynamic type systems, you'll find performance benchmarks. In my personal experience, people I talk to about Groovy++ are asking about performance. Sure, traits are clever. Sure, the type-checking is an interesting idea. But the interest is in performance. Alex's blog posts are mostly about x-thousand transactions per second, or beating Scala performance. I think the number 1 priority of Groovy++ is performance, and type checking is a 2nd nice to have benefit.

And what about people that want a static language with a dynamic backdoor (for builder objects perhaps)? Other languages have this. C# has a "dynamic" variable, right? Boo has a dynamic variable as well, and even lowly Flex has dynamic XML support. This is useful and valuable. I would love to see an @Groovy annotation in Java, where you escape from Java into Groovy, rather than an @Grumpy annotation for Groovy. The benefit of @Groovy for Java is that tool support is the best in the world (without a doubt) and the language has a specification, is well known, and is well tested. Of course, an @Groovy annotation would only ever work with core Java language grammar changes or very ugly hacks.

Lastly, do users just want static compile-time typing for tooling support to catch silly errors as early as possible? Well yes they want that. Who wouldn't? But who is writing Groovy code without testing it? And we already do a great job of recommending suggestions when methods fail ("hashode not found. Did you mean hashcode()"). And the tools exist to do this. IntelliJ IDEA is very reliable in this regard and GroovyLint does the same thing on a limited scale. These features exist today but since they aren't in Eclipse by default then I suspect they are underused. Catching silly errors is entirely possible without the burden of static typing. In fact, some might say that the majority of CodeNarc errors fall into the silly category!

Can Groovy Afford the Complexity?

The Java world talks about a "complexity budget" where a language can only absorb so much complexity before developers leave for something simpler. The term "complexity budget" is poorly defined and often used as a scare tactic in hyperbolic arguments, but I often fear that Groovy could become the next C++ and overrun this arbitrary "complexity budget". C is a good and simple language. Java is a good and simple language. Neither C nor Java are good for all tasks, but they are both successful and capable of having system written in them. C++ started building on C and had every feature known to man added. Nowadays the criticism of C++ is that only 20% is safe to use, but no two C++ programmers can agree on which 20%. Groovy is a multi-paradigm language. You can script, you can OO, and you can FP. Already I am advising teams to find their standard. If you like FP then embrace curry, trampoline, and functions. If you don't then steer clear of those features. Adding a static mode is going to make Groovy more complex. The same code will most likely behave differently between Grumpy mode and not Gumpy. This is one more thing to think about when programming, even if we work to minimize the differences.

Looking at Groovy++, they have done some very cool things. But sometimes the basic features of Groovy++ and Groovy differ, and this cannot be allowed to happen because it is too complex. Already Groovy developers need to know Java and the JVM. I don't know how you could us Groovy outside of Grails without knowing Java. Introducing a new mode that is subtly different could be too much complexity.

Time and Budget Trade-offs

Adding static-mode seems to me like a big undertaking in terms of developer hours. Groovy++ is a good proof of concept for the idea, but the implementation won't be merged into core-groovy. The approach used by Groovy++ is to subclass ASTNodes with AST Transformations and then write out different bytecode for the node, fighting with groovyc to make sure the correct node is invoked. It is better to implement this feature as a part of the core compiler than as an AST transformation, and that is the route the team are currently taking.

It is wonderful that VMWare supports Groovy financially by giving several people jobs. But remember there are finite resources for the project. Any time spent on static-mode means there is less time spent on other features. There aren't enough hours in the day to do everything. So developing a high-performing and well-behaving @Grumpy mode means there are less resources to solve the performance problems of plain old Groovy. I already asserted that performance was a big barrier to wider adoption. Optimized ints are good for benchmarks and came out in Groovy 1.8, but I reckon most method calls are on user defined objects. How many ints are used in a Grails app? Maybe there is a way to do everything at once, but I'm a little sad to see performance improvements not come faster to the language.

My Wishlist

Here is my wishlist of what @Grumpy would do for me and how the feature should roll out.

Some social wishes...

* GEP - The GEP process for Groovy enhancements is supposed to help us formalize features and provide feedback, but it's underused. Please, let's start a GEP as soon as possible and move this big change into the public debate.

* Dev-List Discussion - The dev-list exists to give developers a forum to communicate with each other. There will always be internal conference calls and IM Chats as well, but please lets move more discussion back into the public domain.

* Research - Are there any other dynamic languages with Static backdoor that we learn from them? I don't know of one, but would love to see more examples of what works and what doesn't

Some type inference and static typing wishes...

* Closure Return Type Type-Inference - Closures should have their return type inferred. This is non trivial, and you can have closures that return the values of closure.apply(). At some point type inference breaks. Where will Groovy's fail? How far will the type inferrer go? And how will you specify a return type on a closure when it does fail?

* Closure Arity - The type checker should check the number of parameters a closure has. You should not be able to pass a one method closure to something like Collection.inject() that requires two parameters. All closures need to specify at their type declaration how many parameters it takes.

* Closure Parameter Types - The parameter types of a closure should be type checked. You shouldn't be able to pass a closure that takes a String argument to something that requires a closure that takes an Event argument.

* Closure casting to interfaces - When you cast a Closure into a Single-Method-Interface then the arity, return type, and parameter types should all be checked and verified.

* Interfaces Implemented as a Map - When you implement an interface using the Map notation [ methodName: { /* body */ } ] then the closure arity, parameter types, and return type should be checked, and the compiler should make sure all interface methods are implemented.

* curry/ncurry type issues- Functions like curry, lCurry, rCurry, and nCurry should work in a correct, statically-typed way. For example, if you curry of a 2-arg closure then you're returned a 1-arg closure. Curry a 4-arg closure and you're returned a 3-arg closure. This is a fun edge case that can't be implemented with just overloaded methods!

* Calling groovy from Java - The semantics for calling Grumpy Groovy from Java must remain sane. I'm interested to see some examples of what the APIs look like once all the new types are added. For instance, what is the Closure class hierarchy? Will there be Closure0, Closure1, Closure2 types to model the arity of closures?

* A Grumpy GDK - All of the libraries that ship with Groovy should be as Grumpy as possible. That way a Grumpy developer never has a problem just because a Groovy GDK method was not Grumpy as well. For instance, you should be able to easily make a Grumpy subclass of GroovyTestCase, so any dynamic method on GroovyTestCase should be updated to be as static as possible. Only really use dynamic features when they are necessary.

* Generic by Default - The type inference mechanism should create the correct generics for methods, similar to how F#/Ocaml creates generic methods by default. For instance, the closure: def wrap = { println "logging..."; it() } should have a generic type signature that returns the same type that the 'it' parameter returns.

* AST Transforms Can Read Types - Transforms should be easily able to read the type information out of the AST, and it should be there as early as possible in the compilation process. So the type inference should happen early. The types don't need to be checked early, but they need to be inferred early. Also, when the type of an object is unknown then an AST transformation should be able to read this. Currently, an unknown type is Object to Groovy, but there should be a difference between Object and Unknown.

* AST Transforms Can Write Types - Transforms should be able to write type information. Transforms should be able to participate fully in this process if they are to remain powerful and flexible.

* Type Verification Happens at the End - Type checking and verification should occur at the end of all the compiler phases. This enables AST transforms to actively participate in the type inferring and type specifying process. If type verification runs and fails too early then transforms that could fix the problem won't have a chance to run

So that is my wishlist, perhaps it is a connoisseur's wishlist. What is it that you want to see in Grumpy mode? What do you not want to see?

You can leave feedback here are on the Groovy dev mailing list. We'll see where this goes!

Java 7 Small Language Changes Screencast

2011-07-14T14:46:00.003-05:00

This screencast demonstrates the small language changes that are part of Open JDK 7, which is available from the Open JDK website. It demonstrates multi-catch, try with resources, strings in switch statements, underscores in literals, and the diamond operator.

If you have any issues watching the video below, then you may have better luck viewing it on the JetBrains.tv site.

I've made a lot of screencasts and blog posts over the years. If you like this, then there are many ways to see the other stuff I've done:

My main blog: http://hamletdarcy.blogspot.com
My other JetBrains.tv posts: http://tv.jetbrains.net/tags/hamlet
IDEA related posts on my blog: http://hamletdarcy.blogspot.com/search/label/IDEA
My screencasts on YouTube: http://www.youtube.com/user/HamletDRC
IDEA related Posts on my work blog: http://www.canoo.com/blog/tag/idea/
Or follow me on Twitter: @HamletDRC

The screencast was created with Ubuntu 10.04, PiTiVi, Audicity, gtk-RecordMyDesktop, IntelliJ IDEA, and LibreOffice. OS from top to bottom.

Thanks for watching, leave a comment, or upvote it at your favorite link site!

Mock Objects with Spock Screencast

2011-07-12T18:50:00.002-05:00

This screencast demonstrates how to use Spock testing specifications and Groovy for mocking and stubbing behavior in unit tests. It covers creating the mock object syntax, setting expectations, verifying and spying on results, and argument matchers.

If you have any issues with video playback, then trying viewing it from the JBrains.tv website

Here are some useful links to read for this webcast:

Spock Framework - http://code.google.com/p/spock/
Spock Basics Screencast: http://tv.jetbrains.net/videocontent/ffff
Mocks and Stubs aren't Spies: http://hamletdarcy.blogspot.com/2007/10/mocks-and-stubs-arent-spies.html
Mocking with Spocks: http://www.canoo.com/blog/2010/04/20/spock-and-test-spies-a-logical-choice/

I've made a lot of screencasts and blog posts over the years. If you like this, then there are many ways to see the other stuff I've done:

My main blog: http://hamletdarcy.blogspot.com
My other JetBrains.tv posts: http://tv.jetbrains.net/tags/hamlet
IDEA related posts on my blog: http://hamletdarcy.blogspot.com/search/label/IDEA
My screencasts on YouTube: http://www.youtube.com/user/HamletDRC
IDEA related Posts on my work blog: http://www.canoo.com/blog/tag/idea/
Or follow me on Twitter: @HamletDRC

Phew, that's a lot of self-promotion :)

The screencast was created with Ubuntu 10.04, PiTiVi, Audicity, gtk-RecordMyDesktop, IntelliJ IDEA, and LibreOffice. OS from top to bottom.

Thanks for watching, and leave a comment!

Grails Podcast Interview with Hamlet D'Arcy

2011-07-11T15:11:00.001-05:00

Last week I sat down with the gang at the Grails Podcast and talked shop for about 45 minutes. We talked about a lot of different topics such as Groovy, Lean software, Spock, Groovy in Action, and of course Hackergarten. Check out the full audio and shownotes over at Grails Podcast Episode 125.

IntelliJ IDEA 10.5 for the Groovy and Grails Developer

2011-05-23T09:00:00.002-05:00

The formal release of IntelliJ IDEA 10.5 came out this month, and the new Groovy features are all part of the free and open source Community Edition, and the Grails features are part of the Ultimate Edition. IDEA X (or 10 to you non-Romans) was a larger release of the product, and I already blogged about IDEA X for Groovy and IDEA X for Grails. There’s still plenty of nice features in 10.5 though. The prices for IDEA recently dropped between $100 and $50, and anyone purchasing IDEA since last November gets 10.5 as a free upgrade.

You can read the full article detailing my overview of the features over on the Canoo blog.

There is also a whole bunch of other IDEA related content on my own blog and on the Canoo blog. And if you're feeling promotional, you can always upvote at DZone. Enjoy.

IntelliJ IDEA Keyboard Stickers

2011-05-06T06:25:00.001-05:00

They finally arrived: IntelliJ IDEA keyboard stickers!

Inspired by the Vi Keyboard Stickers, I've been working with JetBrains over the last couple weeks to create keyboard stickers to remind users (and myself) of the most common IDE shortcuts. Here are the stickers applied to my keyboard:

Do you see the subliminal advertising? No really, my business card always sits by my monitor like that.

Anyway, the keys and modifiers are color coded. Blue is control, red underline is Shift. So then the F9 key has the word "Compile" with a blue background and a red underline, it means that Ctrl+Shift+F9 is the shortcut for Compile.

I had to trim a few of the stickers for my Swiss keyboard (the Shift key is much smaller for instance), but otherwise the stickers adhere well and don't come up. I hand wrote all the Shift-values for the numbers, so that I can remember that Shift-3 is the * character (again, good for non-US keyboards). Also, there are one or two stickers that may not apply to non-US keyboards (I can't type Shift+[ for instance).

Remember to be extra careful when applying the stickers. Overlapping on the upper edge tends to catch your fingers, so either center them exactly or err on the side of bottom edge overlaps. And one last errata: the sticker for space is labeled "Backspace". This was my fault, but you can take a pen and scratch out the work "back". Viola, fixed.

Would you like a copy of the stickers for yourself? I have a giant stack of them and so does JetBrains. Here are some ideas on getting them from me, based on location:

Near Switzerland? Come to a Hackergarten in Basel or just DM me (@HamletDRC)
Denmark? I'll be at GR8 Conf May 17th to 19th
Poland? I'll be at GeeCON in Krakow next week
In the US? I'll soon be in New York and Salt Lake City for No Fluff, Denver for ÜberConf, and Minneapolis for GR8 in the US
Spain? I'll be at Greach in Madrid in November
Anywhere else in Europe?

talks and training

Java, Groovy, or Agile talks listed here

* I am no longer legally able to visit Barneveld in the Netherlands.

Getting Started with Spock and Groovy

2011-04-15T10:58:00.002-05:00

This screencast demonstrates how to create Spock testing specifications. It covers creating basic when/then blocks, given/when/then blocks, expect/where blocks, and data tables. It explains how to create Spock file templates and Spock Live Templates for IDEA.

If there are any playback issues then you might try watching it from the JetBrains.tv site.

Here are some useful links to read for this webcast:

Spock Framework - http://code.google.com/p/spock/
IDEA File Templates - http://www.jetbrains.com/idea/webhelp/file-templates.html
IDEA Live Templates - http://www.jetbrains.com/idea/webhelp/live-templates.html

To work with Spock, I use one file template and three live templates.

My file template creates a Spock Specification with the correct java package and javadoc:

The text of the template is:

#if (${PACKAGE_NAME} != "")package ${PACKAGE_NAME};#end

import spock.lang.Specification

#parse("File Header.java")
class ${NAME} extends Specification {

}

My first Live Template is the when-then template:

The text of the template is:

def "$NAME$"() {
  when:
  $END$

  then:
  true
}

The second template is the given/when/then template:

The text of the template is:

def "$NAME$"() {
  given:
  $END$

  when:
  // TODO: add when

  then:
  true
}

The last template is the expect/where template:

The text of the template is:

def "$NAME$"() {
  expect:
  $END$

  where:
  // TODO add where block
}

I've made a lot of screencasts and blog posts over the years. If you like this, then there are many ways to see the other stuff I've done:

My main blog: http://hamletdarcy.blogspot.com
My other JetBrains.tv posts: http://tv.jetbrains.net/tags/hamlet
IDEA related posts on my blog: http://hamletdarcy.blogspot.com/search/label/IDEA
My screencasts on YouTube: http://www.youtube.com/user/HamletDRC
IDEA related Posts on my work blog: http://www.canoo.com/blog/tag/idea/
Or follow me on Twitter: @HamletDRC

Phew, that's a lot of self-promotion :)

Thanks for watching, and leave a comment!

New in IntelliJ IDEA 10.5: Less Clutter

2011-04-01T01:50:00.004-05:00

Interview with Antonio Goncalves about the past, present, and future of Java EE

2011-03-31T04:45:00.003-05:00

I got a chance to sit down and talk to Java Champion Antonio Goncalves about the past, present, and future of Java EE. I've been working for the last six months in a heavy EE/SOA stack, and it's been interesting to see the advantages and disadvantages. I definitely come from the other side of the world where specifications aren't seen as an inherent sign of quality, and frameworks not sanctioned by Sun/Oracle are not to be feared. It was fun to get his opinions about this stuff.

The full interview is on the JetBrains Zone at DZone. We're both JetBrains Academy Members and we're slowly interviewing each other.

P.S. This is the first post I've ever made that mentioned Java Enterprise Edition. I suspect the next time EE is mentioned will be in another few years :)

IntelliJ IDEA Static Analysis: Custom Rules with Structural Search & Replace

2011-03-29T04:13:00.000-05:00

Well, well, I made another screencast. This time I'm taking on IntelliJ IDEA code inspections, and writing your own static code analysis rule (and quick fix!) using Structural Search & Replace. Not bad for under 5 minutes.

If you have any trouble viewing the video then perhaps you should watch it directly on the JetBrains site. And if you're feeling generous, then clicky clicky to upvote at DZone.

Check out these pages for more information on IDEA inspections:

JetBrains' Code Inspection page
IDEA's Webhelp for inspections

Read these pages to learn more about Structural Search and Replace:

JetBrains' Structural Search and Replace Documentation
IDEA's Webhelp for Structural Search and Replace

I've made a lot of screencasts and blog posts over the years. If you like this, then there are many ways to see the other stuff I've done:

My main blog: http://hamletdarcy.blogspot.com
My other JetBrains.tv posts: http://tv.jetbrains.net/tags/hamlet
IDEA related posts on my blog: http://hamletdarcy.blogspot.com/search/label/IDEA
My screencasts on YouTube: http://www.youtube.com/user/HamletDRC
IDEA related Posts on my work blog: http://www.canoo.com/blog/tag/idea/
Or follow me on Twitter: @HamletDRC

Phew, that's a lot of self-promotion :)

Thanks for watching, and leave a comment!

Waste!

2011-03-08T07:39:00.002-06:00

In our industry no one recommends increasing waste, there are no waste evangelists, and no pro-waste lobby. So why is there so much of it? As we scramble frantically through the day to please our customers we are often not even aware of the pointless trail of half done work and unwanted features we leave in our wake. This article helps you understand what waste is, where it comes from, and what you can do about it by mapping the seven wastes of Lean Manufacturing into the software development field. If you want both better software and happier customers then quit wasting time and read this article.

This article originally appeared in the October issue of No Fluff Just Stuff magazine. The full article is available on the Canoo blog. As always, you can upvote at DZone.

Thanks for bearing with the teaser/redirect thing I keep having to do.

Griffon MVC Groups and Event System

2011-03-04T02:12:00.002-06:00

Griffon has a strong MVC focus, and functionality is built using MVC groups. This is not a screen or a form, an MVC group is a piece of functionality, and a single form is made of MVC groups composed together. The way MVC groups talk to each other is through an event bus, rather than being coupled directly to each other.

I created a screencast that demonstrates these concepts in just under 7 minutes. In the video, I create an app, create several MVC groups, and then wire them together with the event bus.

If the video doesn't play correctly, you may want to launch it from the JetBrains.tv site. Also, if you want to be nice then you can upvote this at DZone.

Boom. Winning.

If you like this, you can check out my other screencasts on YouTube (http://www.youtube.com/user/HamletDRC) or JetBrains.tv (http://tv.jetbrains.net/tags/hamlet)

If you want written documentation on these topics, then please read:
* The Griffon RefCard - http://refcardz.dzone.com/refcardz/getting-started-griffon
* The Griffon User Guide - http://dist.codehaus.org/griffon/guide/index.html
* The Griffon Main Site - http://griffon.codehaus.org/

CDI Event Bus Basics Screencast

2011-03-02T08:41:00.003-06:00

Another day another screencast. This one's about the Event Bus that is part of CDI. Contexts and Dependency Injection is part of Java EE but available in SE using the Weld framework. This screencast shows you how to use CDI events and the event system, which is just one part of the larger CDI feature set. The screencast briefly explains the benefits of an event system and then shows how easy it is to use it from CDI.

If it doesn't play correctly, you may want to launch it from the JetBrains.tv site.

You could do worse with 5 minutes of your time.

If you like this, you can check out my other screencasts on YouTube (http://www.youtube.com/user/HamletDRC) or JetBrains.tv (http://tv.jetbrains.net/tags/hamlet)

Lean Groovy 7 - Optimize the Whole

2011-03-02T04:38:00.003-06:00

The successful software project is dependent on more than just the proper functioning of each part of the life-cycle: coding, testing, analysis, and deployment. Success hinges on how well all of these pieces fit together. Optimizing the Whole in Lean refers to making sure you always measure up. Improving your business is too often an exercise in sub-optimization: you may spend all your effort on improving the unit testing experience without realizing that it’s your interactions with operations that are derailing the project, or you may perfect your team’s delivery process while the rest of the company continues behaving wastefully...

Yep, it happened again. I put a two sentence teaser here only to redirect you to the full article over on the Canoo web site. That's just how I roll. I've got tiger blood from a different terrestrial plane. Or something.

Anyway, check out the full article and vote at DZone!

Screencast: GWT Event Bus Basics

2011-02-25T08:40:00.005-06:00

Wheee! I made a screencast. Happy Friday to you too.

This screencast is about the GWT Event Bus. It explains how an Event Bus can be used to loosely couple controllers or views in MVC applications, and includes live coding of a GWT using the SimpleEventBus client. The source is all available on GitHub. And as always, up votes are appreciated.

If it doesn't play correctly, you may want to launch it from the JetBrains.tv site.

"This screencast explains why MVC applications benefit from an event bus, and it demonstrates how to create, wire, and respond to events in Google Web Toolkit (GWT). Not bad for just six and a half minutes." Too much fun.

Effective Groovy with CodeNarc 0.12

2011-01-25T14:11:00.002-06:00

The new release of everyone’s favorite static anaylsis engine for Groovy is here. Time to download CodeNarc 0.12 while it’s hot. And in case this is your first time, CodeNarc analyzes your Groovy code and looks for potential problems and errors, creating a nice report or flagging the problem in your IDE. It’s like FindBugs, Lint, or PMD, but for Groovy.

So what’s new in 0.12? Performance improvements, 35 new analysis rules, and a new logo. It’s supposed to be an armadillo, but no one can tell. We’re calling it the CodeNarc Grue, as in “You are in a maze of twisty little groovy scripts, all alike.” Anyway, you can read the full rundown over on the Canoo blog. My blog post describes the rules and reasons why you might want to use them.

Read the Canoo Blog Post here.

And as usual, vote at DZone or Retweet to spread the word.

IDEA X for the Grails Developer

2010-12-23T00:08:00.002-06:00

The new version of IntelliJ IDEA was released last week. I already blogged what IDEA X means for Groovy developers, and all of those features are obviously available to any Community Edition user (that’s the free one). This post looks at all the new Grails features available in the Ultimate Edition (the one you pay for). These features are a little harder for me to try out personally as my current project is EJB3 and not Grails (someone save me please!). I’ve tried quite a few of them personally, but feedback is always welcome in case I miss or exaggerate something.

As usual, you can read the full article over on the Canoo blog. Feeling magnanimous? Upvote at DZone.

Happy holidays everyone. Six years ago this week I decided to take a few vacation days at my in-law's house and finally get serious about learning Java. I wrote a little Swing app called "TimeTool" whose source code I would be thoroughly embarrassed by today. What a long, strange trip it's been.

Beginner Bash: Must-Know Bash Commands

2010-12-22T00:48:00.002-06:00

I switched full time to Ubuntu Linux last year and haven’t looked back. In this year I’ve learned to love the Bash shell (which includes the Terminal in Mac and Cygwin on Windows). At this point, I can finally say I’m faster in Bash then I was in Windows Explorer, Commander, Nautilus, or the Windows command prompt; and I prided myself on being a guy with a lot of .bat files. My goal now is to write some tips I learned in the last year. Before explaining the more advanced stuff I want to introduce some of the most basic basics.

So here it is: "Stuff I wish someone had explained clearly to me a year ago". Almost all these apply to Cygwin in Windows as well as Mac/Ubuntu Terminal.

As is usual these days, the full article is available on the Canoo blog. And if you want to upvote then head on over to DZone or clicky clicky on up arrow at the bottom of the post.

IntelliJ IDEA X for Groovy Developers

2010-12-20T04:07:00.001-06:00

The new version of IntelliJ IDEA was released last week. Here’s what it means for Groovy developers. I’ve tried out most of these new features, but there are one or two that I haven’t yet seen in action. Also, there’s a whole slew of Grails features that I’ll cover in a later blog post.

Without further ado, read about the features over on the Canoo Blog. As always, if you want to contribute to the hype cycle then vote this up at DZone.

Mockito – Screencasts for Stubbing, Verifying, and Argument Matching

2010-12-14T23:40:00.000-06:00

We're rolling out Mockito and trying to raise our testability at work and I'm set to give a presentation/training session tomorrow to a few new teams. In case you don't know, Mockito is a mock object framework for Java. It's competitors are EasyMock, JMock, and others if you're more familiar with those. If you haven't seen it then you may want to check out my old post "Mockito - Pros, Cons, and Best Practices". To prepare for my presentation I decided to record myself practicing my material and post it on youtube. Enjoy!

The first screencast is about creating mock objects and stubbing behavior. These are the absolute basics of mocking.

The second screencast is about verifying behavior, or verifying side effects, using Mockito. This is a little more advanced but still an essential API in working with Mockito. YouTube reports this video as 13 minutes long, but don't worry it is only 4:45. Some quirk of YouTube.

The final screencast is about argument matchers, which add flexibility to your stub and verify phases.

I hope you enjoyed these. They could be a little more practiced, but I'm happy enough with the quality. I recorded these on Ubuntu 10.4 using recordMyDesktop and mencode to convert from .ogv to .avi.

Slimmed Down Software – A Lean, Groovy Approach Part 6 – Respect People

2010-12-07T14:36:00.002-06:00

Respecting people means creating an environment where the human element drives innovation and value. Creative problem solving and spontaneity are our most valuable assets, and great companies arrange their development process to let every single employee be able to exercise these traits...

The entire article, as usual, is available over to the Canoo Blog. And you can of course vote on DZone.

This article originally appeared in the September 2010 edition of GroovyMag, the Groovy and Grails magazine. Part 7 is currently available for download from the magazine’s site, and more will come each month. Previous articles in this series are on the Canoo website: Part 1: Eliminate Waste, Part 2: Build Quality In, Part 3: Create Knowledge, Part 4: Defer Commitment, and Part 5: Deliver Fast. Lastly, if you like this, you may want to check out some of my older blog posts from my personal site under the ‘craft’ category. Enjoy!

Remote Script Execution with GroovySocketServer

2010-12-02T05:02:00.001-06:00

File under: extremely dangerous but definitely fun. Here's some directions on how to execute any arbitrary Groovy script on a remote server. We're going to write a one line Groovy script that, when run, will listen to a socket and execute any incoming text as if it were Groovy script and return the result to the client. We'll do it from the command line and programatically. The client is actually more difficult to write and weighs in at a whooping 5 lines. Let's get started.

The whole article is available on the Canoo Blog. For those inclined there is certainly a Dzone Link to upvote. And none of the URLs have dirty words in them this time!

CodeNarc and GroovyServ Screencasts

2010-11-29T12:40:00.002-06:00

At the last Hackergarten we came together and created two screencasts, both about projects in the Groovy ecosystem.

The first is about CodeNarc, a static analysis tool for Groovy similar to FindBugs or PMD for Java. The screencast can be upvoted at DZone or watched below:

The second is about GroovyServ, an application that increases the startup time of Groovy scripts. It is in German, but is easy to follow along even if you don't speak the language. You can vote for it at DZone here and watch it below:

Be sure to follow the mailing list for future Hackergarten events... it would be great to have some more people come out and join us next time.

Groovy CodeNarc 0.11 Unofficial Guide

2010-11-15T12:04:00.002-06:00

Groovy CodeNarc is a static analysis tool for the Groovy language and does for Groovy what PMD and FindBugs does for Java: it analyses Groovy code for defects, mistakes, and bad practices. The new version 0.11 is a huge release for us. We’ve added over 50 new rules to the product (bringing the total to 130+ rules) and a few really great features. This is THE new look CodeNarc, and it’s ready to use today, try it out in the CodeNarc Web Console or download the package directly from the CodeNarc site. (Ant, Maven, grails, gradle, and griffon plugins all exist as well).

To announce the release I wrote up the Groovy CodeNarc 0.11 Unofficial Guide. You can read it over on the Canoo site and please, if you like, vote over at DZone.

There and Back Again - A Developer's Recursion Tale

2010-11-08T14:39:00.003-06:00

Recursive functions are so 2009. All the cool kids are now scrambling to convert those legacy recursive functions into stack based iteration. Why would you ever need this, you ask? Maybe you’ve heard of my friend “StackOverflowError”, he likes to hang out on the JVM. Or maybe you’re a browser kinda guy and you’ve seen your alert messages say “Stack Overflow”. Or worst case (like me), you’re supporting IE6 and you finally figured out that the “Browser shows a blank screen” defect you can’t reproduce is IE’s way of overflowing (oh yes it really is). Well, stack based iteration is a way to leave your nice recursive function the way it is, yet still avoid the overflow errors. You just have to keep track of the function stack frames yourself by adding a little boilerplate around your method (and even that can be cleaned up fairly easily).

So here is the tale of taking the road now commonly traveled. Learning to program imperatively, writing some beautiful recursive functions, and then ripping it out again because of platform limitations. There and Back Again – A Developer’s Tale of Recursion...

Read the full article over on the Canoo blog. And if you want to be kind and vote, then head over to DZone first.

Advanced Mockito: Capturing State with Answer and Captors

2010-11-05T12:25:00.002-05:00

Hmmm… I really need to mock out java.sql.ResultSet, I know I’ll just subclass it with a hand-rolled mock. Oh-no, the default implementation without method bodies is 650 lines of code long. WTF? Did you know there are 197 public methods on the java.sql.ResultSet interface? This is an absurdly large interface, possibly the largest in the JDK (if not drop a comment, I’d love to know what the largest is). The workaround is to throw one together in Mockito (or your mock framework of choice). My implementation turned out to be 21 lines of code long. Not bad at all. To do it I creatively used ArgumentCaptors, Answer objects, and anonymous inner classes to capture state... (read on for the full article).

The full article is posted over on the Canoo Blog. And don't forget to vote at dzone.

Interview with Dierk Koenig – Author of Groovy in Action

2010-11-05T01:56:00.002-05:00

Early this week I pestered Dierk Koenig into letting me ask him a few questions about Groovy past and present and the upcoming release of Groovy in Action 2nd Edition. If you aren’t aware already, Dierk is the author of Groovy in Action, a Canoo Fellow, and a Groovy, Grails and GPars committer.

You can read the complete interview over at the Canoo blog. And if you want to be a kind bro then you can vote at DZone. Do people even say "kind bro" anymore? Anyway... happy Friday.

Slimmed Down Software- A Lean, Groovy Approach Part 5 - Deliver Fast

2010-10-28T14:56:00.002-05:00

"It is clear and logical that developing features in small increments means you deliver sooner, and there is value in that. Delivering a partial system on an earlier calendar date means the customer starts accruing a return on investment before project completion, and this can even result in the system paying for itself before the project is even finished. But will delivering smaller increments really increase your output in the long term? The answer from queuing theory research is a resounding "yes"...

Part 5 of Slimmed Down Software is now available on the Canoo Blog. To read the entire article please surf on over there. And if you want to be kind then upvote at DZone.

This article originally appeared in the August 2010 edition of GroovyMag, the Groovy and Grails magazine. Parts 6 and 7 are currently available for download from the magazine’s site, and more will come each month. Previous articles in this series are on the Canoo website: Part 1: Eliminate Waste, Part 2: Build Quality In, Part 3: Create Knowledge, and Part 4: Defer Commitment. Lastly, if you like this, you may want to check out some of my older blog posts from my personal site under the ‘craft’ category. Enjoy!

Speaking at Devoxx

2010-10-27T12:50:00.002-05:00

Good news for me, I'll be giving a 15-minute quickie at Devoxx on the topic of Groovy Code Generation. The talk is normally a little longer, but most talks can be improved by making them shorter. I'm sure the 15 minutes will be great fun. Check out the slides for the longer version here.

If you can't make Devoxx, then be sure to catch fellow Canooie Dierk König at W-JAX during the same time. His slots are all in German, but the code samples will surely be in Groovy.

For those interested here is the abstract for my talk. If you run or help organize a JUG and want a speaker then please contact me. I'm excited to travel around in 2011, see more of the world, and meet great people.

Code Generation on the JVM: Writing Code that Writes Code
"The Pragmatic Programmer" admonished us all to "write code that writes code": use code generators to increase productivity and avoid duplication. Today's language communities have clearly caught on, as more and more frameworks generate code at compile time: AST Transforms, Project Lombok, Spring Roo, and more.

This session reviews these approaches including examples of how and why we'd want to do this. Come see the newest Groovy language tools, look in-depth at production deployed AST Transforms, and view libraries based on these techniques.

Audience: developers searching for cutting edge solutions to increasing team velocity.

IntelliJ IDEA Shortcut Wallpaper

2010-10-11T15:57:00.005-05:00

The fastest developers use the keyboard almost exclusively. To help you learn the IntelliJ IDEA shortcuts, I created a desktop wallpaper that lists the most common ones for Linux, Mac and Windows users. Can't remember the command? Just pop up the desktop and check it out. Bored while waiting for a compile? Ditto.

There are a few resolutions:

IntelliJ IDEA Linux/Windows 1440x900
IntelliJ IDEA Macintosh 1440x900
IntelliJ IDEA Linux/Windows 1680x1050
IntelliJ IDEA Macintosh 1680x1050
IntelliJ IDEA Linux/Windows 1920x1200
IntelliJ IDEA Macintosh 1920x1200

Not on IDEA? You can download the Eclipse Desktop wallpaper from us, or the vim quick reference from our friend Ted Naleid.

doce ut discas - Teach, that you yourself may learn.

Eclipse Keyboard Shortcut Wallpaper

2010-10-11T15:53:00.013-05:00

The fastest developers use the keyboard almost exclusively. To help you (and me) learn the Eclipse shortcuts, I created a desktop wallpaper that lists the most common ones for Linux, Macintosh and Windows users. Can't remember the command? Just pop up the desktop and check it out. Bored while waiting for a compile? Ditto.

There are a few resolutions:

Eclipse Linux/Windows 1280x1024
Eclipse Linux/Windows 1440x900
Eclipse Linux/Windows 1680x1050
Eclipse Linux/Windows 1920x1080

Not on Eclipse? You can download the IntelliJ IDEA wallpaper from us, or the vim quick reference from our friend Ted Naleid.

doce ut discas - Teach, that you yourself may learn.

Mockito - Pros, Cons, and Best Practices

2010-09-29T15:04:00.003-05:00

Mockito - Pros, Cons, and Best Practices

It's been almost 4 years since I wrote a blog post called "EasyMock - Pros, Cons, and Best Practices, and a lot has happened since. You don't hear about EasyMock much any more, and Mockito seems to have replaced it in mindshare. And for good reason: it is better.

A Good Humane Interface for Stubbing
Just like EasyMock, Mockito allows you to chain method calls together to produce less imperative looking code. Here's how you can make a Stub for the canonical Warehouse object:


Warehouse mock = Mockito.mock(Warehouse.class);
Mockito.when(mock.hasInventory(TALISKER, 50)).
        thenReturn(true);

I know, I like a crazy formatting. Regardless, giving your System Under Test (SUT) indirect input couldn't be easier. There is no big advantage over EasyMock for stubbing behavior and passing a stub off to the SUT. Giving indirect input with mocks and then using standard JUnit asserts afterwards is simple with both tools, and both support the standard Hamcrest matchers.

Class (not just Interface) Mocks
Mockito allows you to mock out classes as well as interfaces. I know the EasyMock ClassExtensions allowed you to do this as well, but it is a little nicer to have it all in one package with Mockito.

Supports Test Spies, not just Mocks
There is a difference between spies and mocks. Stubs allow you to give indirect input to a test (the values are read but never written), Spies allow you to gather indirect output from a test (the mock is written to and verified, but does not give the test input), and Mocks are both (your object gives indirect input to your test through Stubbing and gathers indirect output through spying). The difference is illustrated between two code examples. In EasyMock, you only have mocks. You must set all input and output expectations before running the test, then verify afterwards.


// arrange
Warehouse mock = EasyMock.createMock(Warehouse.class); 
EasyMock.expect(
    mock.hasInventory(TALISKER, 50)).
    andReturn(true).once();
EasyMock.expect(
    mock.remove(TALISKER, 50)).
    andReturn(true).once();
EasyMock.replay(mock); 

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
EasyMock.verify(mock);

That's a lot of code, and not all of it is needed. The arrange section is setting up a stub (the warehouse has inventory) and setting up a mock expectation (the remove method will be called later). The assertion in all this is actually the little verify() method at the end. The main point of this test is that remove() was called, but that information is buried in a nest of expectations. Mockito improves on this by throwing out both the record/playback mode and a generic verify() method. It is shorter and clearer this way:


// arrange
Warehouse mock = Mockito.mock(Warehouse.class);
Mockito.when(mock.hasInventory(TALISKER, 50)).
        thenReturn(true);

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
Mockito.verify(warehouse).remove(TALISKER, 50);

The verify step with Mockito is spying on the results of the test, not recording and verifying. Less code and a clearer picture of what really is expected.
Update: There is a separate Spy API you can use in Mockito as well: http://mockito.googlecode.com/svn/branches/1.8.3/javadoc/org/mockito/Mockito.html#13

Better Void Method Handling
Mockito handles void methods better than EasyMock. The fluent API works fine with a void method, but in EasyMock there were some special methods you had to write. First, the Mockito code is fairly simple to read:


// arrange
Warehouse mock = Mockito.mock(Warehouse.class);

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
Mockito.verify(warehouse).remove(TALISKER, 50);

Here is the same in EasyMock. Not as good:


// arrange
Warehouse mock = EasyMock.createMock(Warehouse.class); 
mock.remove(TALISKER, 50); 
EasyMock.expectLastMethodCall().once(); 
EasyMock.replay(mock); 

//act
Order order = new Order(TALISKER, 50);
order.fill(warehouse); 

// assert
EasyMock.verify(mock);

Mock Object Organization Patterns
Both Mockito and EasyMock suffer from difficult maintenance. What I said in my original EasyMock post holds true for Mockito:
The method chaining style interface is easy to write, but I find it difficult to read. When a test other than the one I'm working on fails, it's often very difficult to determine what exactly is going on. I end up having to examine the production code and the test expectation code to diagnose the issue. Hand-rolled mock objects are much easier to diagnose when something breaks... This problem is especially nasty after refactoring expectation code to reduce duplication. For the life of me, I cannot follow expectation code that has been refactored into shared methods.
Now, four years later, I have a solution that works well for me. With a little care you can make your mocks reusable, maintainable, and readable. This approach was battle tested over many months in an Enterprise Environment(tm). Create a private static method the first time you need a mock. Any important data needs to be passed in as a parameter. Using constants or "magic" fields hides important information and obfuscates tests. For example:


User user = createMockUser("userID", "name"); 
...
assertEquals("userID", result.id()); 
assertEquals("name", result.name();

Everything important is visible and in the test, nothing important is hidden. You need to completely hide the replay state behind this factory method if you're still on EasyMock. The Mock framework in use is an implementation detail and try not to let it leak.
Next, as your dependencies grow, be sure to always pass them in as factory method parameters. If you need a User and a Role object, then don't create one method that creates both mocks. One method instantiates one object, otherwise it is a parameter and compose your mock objects in the test method:


User user = createMockUser(
    "userID", 
    "name", 
    createMockRole("role1"), 
    createMockRole("role2")
);

When each object type has a factory method, then it makes it much easier to compose the different types of objects together. Reuse. But you can only reuse the methods when they are simple and with few dependencies, otherwise they become too specific and difficult to understand.
The first time you need to reuse one of these methods, then move the method to a utility class called "*Mocker", like UserMocker or RoleMocker. Follow a naming convention so that they are always easy to find. If you remembered to make the private factory methods static then moving them should be very simple. Your client code ends up looking like this, but you can use static imports to fix that:


User user = UserMocker.createMockUser(
    "userID", 
    "name", 
    RoleMocker.createMockRole("role1"), 
    RoleMocker.createMockRole("role2")
);

User overloaded methods liberally. Don't create one giant method with every possible parameter in the parameter list. There are good reasons to avoid overloading in production, but this is test. Use overloading so that the test methods only display data relevant to that test and nothing more. Using Varargs can also help keep a clean test.
Lastly, don't use constants. Constants hide the important information out of sight, at the top of the file where you can't see it or in a Mocker class. It's OK to use constants within the test case, but don't define constants in the Mockers, it just hides relevant information and makes the test harder to read later.

Avoid Abstract Test Cases
Managing mock objects within abstract test cases has been very difficult for me, especially when managing replay and record states. I've given up mixing mock objects and abstract TestCase objects. When something breaks it simply takes too long to diagnose. An alternative is to create custom assertion methods that can be reused. Beyond that, I've given up on Abstract TestCase objects anyway, on the grounds of preferring composition of inheritance.

Don't Replace Asserts with Verify
My original comments about EasyMock are still relevant for Mockito: The easiest methods to understand and test are methods that perform some sort of work. You run the method and then use asserts to make sure everything worked. In contrast, mock objects make it easy to test delegation, which is when some object other than the SUT is doing work. Delegation means the method's purpose is to produce a side-effect, not actually perform work. Side-effect code is sometimes needed, but often more difficult to understand and debug. In fact, some languages don't even allow it! If you're test code contains assert methods then you have a good test. If you're code doesn't contain asserts, and instead contains a long list of verify() calls, then you're relying on side effects. This is a unit-test bad smell, especially if there are several objects than need to be verified. Verifying several objects at the end of a unit test is like saying, "My test method needs to do several things: x, y, and z." The charter and responsibility of the method is no longer clear. This is a candidate for refactoring.

No More All or Nothing Testing
Mockito's verify() methods are much more flexible than EasyMock's. You can verify that only one or two methods on the mock were called, while EasyMock had just one coarse verify() method. With EasyMock I ended up littering the code with meaningless expectations, but not so in Mockito. This alone is reason enough to switch.

Failure: Expected X received X
For the most part, Mockito error messages are better than EasyMock's. However, you still sometimes see a failure that reads "Failure. Got X Expected X." Basically, this means that your toString() methods produce the same results but equals() does not. Every user who starts out gets confused by this message at some point. Be Warned.

Don't Stop Handrolling Mocks
Don't throw out hand-rolled mock objects. They have their place. Subclass and Override is a very useful technique for creating a testing seam, use it.

Learn to Write an ArgumentMatcher
Learn to write an ArgumentMatcher. There is a learning curve but it's over quickly. This post is long enough, so I won't give an example.

That's it. See you again in 4 years when the next framework comes out!

Clearer Unit Tests: Assert, Guard, and Throw

2010-09-27T16:11:00.002-05:00

I hate the book “xUnit Test Patterns” for being over 800 pages long. I am convinced a good editor can turn any 800 page book into a 700 pager with the exact same content… that includes you Leo Tolstoy! The problem is that xUnit Patterns is still the definitive reference to test concepts and implementation. There is no better book on testing available, but the sheer weight and size the book means few have actually read it. Well I have, and I highlighted the damn thing just to prove it. In this post you’ll find 3 techniques to improve test clarity that, in my opinion, are underutilized in the development world today.

Surf over to http://canoo.com/blog to read the rest of the article, or go straight to the Dzone link if you're inclined to vote it up. Thanks!

Beautiful Java Enums

2010-09-24T13:20:00.002-05:00

File under “It’s never too late to blog about Java 5 features” and “you wrote 1000 words on what?”.

Way back before Java 5 I fell in love with the Type Safe Enum pattern described in Martin Fowler’s Refactoring. You know the one, where an instance of a class has members of itself as public static fields. It was so clever (not always a compliment) and I applied it everywhere I could. Fowler’s example dealt with blood types... to read the full article surf over to http://canoo.com/blog/2010/09/24/beautiful-java-enums/

And don't forget to spread a little DZone love...

@Log – Groovy’s New and Extensible Logging Conveniences

2010-09-20T14:37:00.001-05:00

The latest version 1.8 snapshot of Groovy includes a cool new logging feature called @Log. The key of @Log and the other logging annotations is that the parameters sent to logging methods are not evaluated unless that specific log level is enabled. For instance, you can call Logger.debug with a long running closure as a parameter, and if the debug log level is not enabled then the parameter is never evaluated.

My full article on the Canoo blog shows the details, all the frameworks supported, and how to use your own logging framework with the feature: @Log – Groovy’s New and Extensible Logging Conveniences

More Pecha Kucha Lessons Learned

2010-09-19T14:51:00.003-05:00

Last weekend was Canoo’s annual Continuous Improvement workshop, where we retreat to our secret Alp fortress to discuss the coming year and where we want to go as a company. My favorite memory is getting licked by a cow in a breakout session. Only in Switzerland. My second favorite was the Pecha Kucha contest.

A Pecha Kucha is a 20 slide presentation where each slide is presented for only 20 seconds. That’s 6 minutes and 20 seconds total. It’s a lot different than a normal presentation, and I’ve already blogged my “9 Beginner Mistakes” a few years ago. After watching 6 other people give Pecha Kuchas, I was struck by a couple common themes in what was working and what wasn’t. Check out my Canoo.com blog post "More Pecha Kucha Lessons Learned" to read more about Pecha Kuchas and what I think it takes to deliver a great one.

Slimmed Down Software – A Lean, Groovy Approach Part 4 – Defer Commitment

2010-09-13T14:40:00.002-05:00

Just published a new blog post over on Canoo.com. In my humble opinion, this is the best one of the series. Less code, more jokes, and more ideas than normal. Check it out here:

Part 4 of Lean Groovy - Deferred Commitment

(or be kind and vote at DZone.)

This article originally appeared in the July 2010 edition of GroovyMag, the Groovy and Grails magazine. Parts 5 and 6 are currently available for download from the magazine’s site, and more will come each month. Previous articles in this series are on the Canoo website: Part 1: Eliminate Waste, Part 2: Build Quality In, and Part 3: Create Knowledge. Lastly, if you like this, you may want to check out some of my older blog posts from my personal site under the “craft” category. Enjoy!

Slimmed Down Software – A Lean, Groovy Approach Part 3 – Create Knowledge

2010-08-12T06:55:00.004-05:00

Alright, another episode of Lean Groovy has entered the public domain.

This month we take a look at the Spock framework with "Slimmed Down Software – A Lean, Groovy Approach Part 3 – Create Knowledge".

Part 4 and 5 are available today from GroovyMag. More to come shortly.

If you like it vote at DZone: http://is.gd/eenuZ

Project Lombok - Java Without the Boilerplate

2010-07-27T01:55:00.002-05:00

I've been playing around with Project Lombok and Canoo's RIA Suite (nee ULC) recently, getting ready for my next project. Lombok is a pure Java library that does compile time byte code generation to remove/generate a whole bunch of Java boilerplate for for you... getters, setters, toStrings, etc.

You can read about Lombok, and how to use it within RIA Suite, on the Canoo Blog. And, of course, you may vote for it on DZone.

Enjoy!

Ubuntu Kung-Fu – 10 Best Tricks (and some even work on Macs)

2010-07-20T06:41:00.003-05:00

I switched to Ubuntu at home about a year ago and at work on January 1st. At this point I don't think I could ever go back to anything else (including my wife's sleek looking Macintosh).

Over my summer break I bought the book Ubuntu Kung-Fu. It was a fun and valuable read. I wrote up a quick note about my 10 favorite tricks over on the Canoo blog. You can read it there or head over to DZone to show your love with an upvote.

Griffon Refcard Released, Lead Interviewed

2010-07-19T03:16:00.004-05:00

Big Griffon news today.

First, the Griffon Refcard is available from DZone. It is a concise 6-page reference to getting started and working with Griffon. If the only thing you know about Griffon is that it is a Grails and Groovy based framework for building desktop applications, then this RefCard will fill you in with tons more details. And if you’re an old Griffon hack, then you’ll probably still pick up some new tips and tricks. I wrote it, and always like to hear feedback.

Plus, DZone published an interview with Canooie Andres Almiray, the Griffon project lead. You can read the interview here, or click the RefCard below to download the card.

Anyone interested in Griffon in the Swiss area should come out to Hackergarten on the 30th of July. Several experienced Griffon developers will be present including Andres. And there is free pizza!

Code Generation on the JVM: Video and Slides

2010-07-14T08:04:00.001-05:00

Two weeks ago I spoke at the Chech Java User Group (CZJUG) on the topic of “Code Generation on the JVM”. Some of the technologies covered are Lombok, Groovy, GContracts, Spock, AST Transformations, Spring Roo, and other fun stuff.

Links to the slides and videos are available on the Canoo Blog: http://canoo.com/blog/2010/07/14/code-generation-on-the-jvm-video-and-slides/

Slimmed Down Software – A Lean, Groovy Approach Part 2 – Build Quality In

2010-07-05T09:07:00.002-05:00

The 2nd part of "Slimmed Down Software - A Lean, Groovy Approach" is posted over on the Canoo Blog.

This was originally printed in GroovyMag, and parts 3 and 4 are available now from the GroovyMag website.

If you like it then be sure to vote at DZone.

IT Certification Alternatives: Cost Benefit Analysis

2010-06-09T02:24:00.003-05:00

I wrote a blog post about IT Certifications over on the Canoo Blog.

IT Certification Alternatives: Cost Benefit Analysis

I think there are a lot of good opportunities outside of the traditional MCSP/SCJP track that are lower risk, higher value, and lower cost.

As always, votes at DZone or Reddit appreciated.

Slimmed Down Software – A Lean, Groovy Approach Part 1 – Eliminate Waste

2010-06-07T01:18:00.002-05:00

GroovyMag is currently running a 7 part series I wrote called "Slimmed Down Software - A Lean, Groovy Approach". Part 1 started in the April 2010 edition, and now that it is June the copyright has reverted back to me and I get to self publish it on the web! If you like this then consider buying the May and June issues.

The full article is published on the Canoo Blog. As usual, up-votes at DZone are welcome.

Here's the summary:

The Groovy Programming Language advertises itself as an “agile and dynamic language for the JVM”, but what does this mean exactly? This series of articles explains Lean Software Development, and shows how your choice of programming language can make your entire process remain nimble and adaptive. Each month will cover one of the seven Lean Software Development principles and explain how Groovy and the associated ecosystem help eliminate waste, defer commitment, and build quality into your product.

GR8 Conference Europe Wrap Up

2010-05-24T10:44:00.001-05:00

Last week was the GR8 Conference for Groovy, Grails, Griffon, etc. in Copenhagen Denmark.

Besides having an awesome Hackergarten event, there was a traditional conference as well. A few of us at work were lucky (or hard working) enough to go. We wrote our experiences up on the Canoo blog.

Post is here: http://is.gd/cn2PW

And DZone Upvotes always appreciated: http://is.gd/cn2YG

Gilad Bracha and the Java Post Mortem

2010-05-07T12:40:00.002-05:00

At Jax.de this week I got to see two sessions with Gilad Bracha and sit on a panel with him (never would have imagined that).

I wrote my thoughts about the day up on the Canoo blog in a piece called "Gilad Bracha and the Java Post Mortem"

Upvotes always appreciated!

Spock Framework Mock Objects: Better Groovy Based Testing

2010-04-21T08:23:00.002-05:00

I wrote an article on the Canoo blog called "Spock and Test Spies: A Logical Choice".

If you are interested in alternative (and better) mock object frameworks in Groovy then check it out. Spock is a heavy user of Groovy AST Transformations, so those interested in transformations may also want to take a peek.

And if you have come here simply to vote the article up on DZone, then click away!

Groovy Interview and GR8 Conference Website

2010-04-15T00:48:00.002-05:00

Søren over at the GR8 Conference European Headquarters was kind enough to ask me a few questions about Groovy. I find the itnerview an entertaining read, but then I'm maybe biased just a little.

Check it out here: http://www.gr8conf.org/blog/2010/04/15/70

Groovy 1.7.2 - Three Features Worth the Upgrade

2010-04-08T14:50:00.002-05:00

Groovy 1.7.2 was released 22 hours ago, so by now you have surely spent hours playing with the new bits and combing through the release notes looking for new nuggets of productivity.

Or maybe not.

For all those busier than me, here are the top 3 reasons to upgrade to the point release:

Convert GPathResults to XML - JIRA 1178

XmlSlurper is a great way to parse, manipulate, and generally tease XML. The problem is that once you start drilling into XML using the dynamic syntax, then you are working with GPathResults objects and not plain text or Strings. It was quite hard to pull out snippets of XML as text. No more! Now you can use XmlSlurper to pull out pieces of XML and then easily manipulate them as text by converting a GPathResult into a String.

Here is the classic XML structure from the XmlSlurper documentation:

def CAR_RECORDS = '''
   <records>
     <car name='HSV Maloo' make='Holden' year='2006' >
       <country>Australia</country>
       <record type='speed'>Production Pickup Truck with speed of 271kph</record>
     </car>
     <car name='P50' make='Peel' year='1962'>
       <country>Isle of Man</country>
       <record type='size'>Smallest Street-Legal Car at 99cm wide and 59 kg in weight</record>
     </car>
     <car name='Royale' make='Bugatti' year='1931'>
       <country>France</country>
       <record type='price'>Most Valuable Car at $15 million</record>
     </car>
   </records>
 '''

Hopefully Blogger hasn't totally destroyed the XML tags. Yet one more reason to hate on XML. So anyway, XmlSlurper makes it trivial to pull out the second car element from the example XML:

def records = new XmlSlurper().parseText(CAR_RECORDS)
def secondCar = records.car[1]
assert secondCar.getClass() == NodeChild
assert secondCar.toString() == 'Isle of ManSmallest Street-Legal Car at 99cm wide and 59 kg in weight'

There are a few problems with this example, and it isn't that the second car element has an index of 1. The secondCar variable is of type NodeChild. This is great if you want to perform more dynamic matching against NodeChild, but not so great if you want to get the full XML of the child and all descendants. The toString() is pretty meaningless, it's just a concatenation of all the child node values. In 1.7.2, StreamingMarkupBuilder lets you quickly get the child XML as a String. The assertion and code sample is messy, but the feature isn't:

def xmlString = new StreamingMarkupBuilder().bindNode(secondCar).toString()
assert xmlString == "<car name='P50' year='1962' make='Peel'><country>Isle of Man</country><record type='size'>Smallest Street-Legal Car at 99cm wide and 59 kg in weight</record></car>"

Nice. I understand you could somehow get the XML in previous Groovy versions, but I never figured it out. I'm lucky if I can just get something to pretty print.

Easier Map and Property Sorting - JIRA 2597

It is easier than ever to sort Maps and Properties objects. The first step is to create an unsorted Map:

def map = [c: 1, b: 2, a: 3]

In previous Groovy releases there were two options to sort a Map. One sort method took a closure that would be invoked like a Comparator, and the other way is just to wrap the Map in a TreeMap. Here are some assertions showing how they worked:

assert map.sort { Entry it -> it.key }*.key == ['a', 'b', 'c']
assert new TreeMap(map)*.key == ['a', 'b', 'c']

Not bad, but now the Map object has a no-arg sort that does default sorting and an API that lets you pass a real Comparator:

assert map.sort()*.key == ['a', 'b', 'c']
assert map.sort({ a, b -> a <=> b } as Comparator)*.key == ['a', 'b', 'c']

And remember, these methods return a new Map, they do not mutate the original. No Maps are harmed during the sorting of Maps.

ncurry and rcurry - JIRA 4144

The curry method has been around Groovy for a long time. It lets you bind variables into closures from left to right:

def multiply = { a, b -> a * b }
def doubler = multiply.curry(2)
assert doubler(4) == 8

In the above example, multiply has a type signature of "Object -> Object -> Object" (two Object parameters and an Object return type). When you curry multiply into doubler you get a type signature of "Object -> Object" (one Object parameter and an Object return type). Parameters are bound from left to right. Curry always bound the left most parameter in the parameter list. Until now.

rcurry will bind parameters from right to left. So a halver can now be made from a divider. Whee!

def divider = { a, b -> a / b }
def halver = divider.rcurry(2)
assert 5 == halver(10)

ncurry is the API sibling of curry and rcurry. It lets you bind parameters based on the integer index of the parameter in the signature. So you can bind parameter 2, 3, 4, or whatever. Check it out:

def operation = { int x, Closure f, int y -> f(x, y) }
def divider = operation.ncurry(1) { a, b -> a / b }
assert 5 == divider(10, 2)

Before you go too crazy with ncurry and rcurry, be warned that I think there is a bug when you try to mix them. Using ncurry and rcurry alone seems to work great, but rcurrying an ncurried closure does not currently work. I have a hunch that this does not effect a whole lot of people. I'll post a comment when I know what is wrong or when it is fixed.

Now go upgrade to 1.7.2!

Slimmed Down Software – A Lean, Groovy Approach in GroovyMag

2010-04-07T01:51:00.004-05:00

The newest issue of GroovyMag is hot off the presses, and it contains the first of seven articles on Lean Software and Groovy.

The title is “Slimmed Down Software – A Lean, Groovy Approach Part 1 – Eliminate Waste”, and from the article:

The Groovy Programming Language advertises itself as an “agile and dynamic language for the JVM”, but what does this mean exactly? This series of articles explains Lean Software Development, and shows how your choice of programming language can make your entire process remain nimble and adaptive. Each month will cover one of the seven Lean Software Development principles and explain how Groovy and the associated ecosystem help eliminate waste, defer commitment, and build quality into your product.

Enjoy!

Gradle Plugin Conventions: Groovy Magic Explained

2010-03-31T04:44:00.002-05:00

Hackergarten last Friday was a big success. About 10 of us (not all Canooies!) stayed up late and created an Announce plugin for Gradle. In the 0.9 release you will be able to announce build events to Twitter, Snarl, and the Ubuntu notification service (sorry, no Growl yet, it is coming). It was fun, educational, and energizing, and you should sign up for the Google Group and come to the next event.

While writing the plugin, we were briefly delayed by a technical detail. The user guide explains how to pass simple parameters to a plugin, but does not explain how to nest parameters within a closure. For instance, in the announce plugin you specify the Twitter username and password within an announce block:

announce {
  username 'your-username'
  password 'your-password'
}

It turns out to be easy (just not documented, but we will fix that soon). Here is the full gradle build that shows you how to do it, with a detailed explanation of how it works below. As an example, we'll extend the Gradle documentation greeter custom plugin. This whole thing is runnable from the command line with "gradle hello":

apply plugin: GreetingPlugin

greet {
  message = 'Hi from Gradle'
}

class GreetingPlugin implements Plugin<Project> {
  def void apply(Project project) {

      project.convention.plugins.greeting = new GreetingPluginConvention()
      project.task('hello') << {
          println project.convention.plugins.greeting.message
      }
  }
}

class GreetingPluginConvention {
  String message

  def greet(Closure closure) {
      closure.delegate = this
      closure()
  }
}

Starting at the top... there is a property declaration called "message" within a "greet" block. This is a much nicer configuration option than just using build-global variables, and your plugin should do something like this. It is important to understand what this block actually is and how it works. In Groovy, parentheses on method calls are optional and curly braces declare a closure. So this greet block is really an invocation of a method with type "Object greet(Closure)". To read and capture this message variable, you must somewhere declare and wire in this "Object greet(Closure)" method within your plugin.

Moving on to the Plugin declaration. Plugin configuration is captured in something Gradle calls a "Convention" object. It is just a plain old Groovy object, and there is no Gradle magic... all the magic happending is standard Groovy. Let's examine the apply() method closely:

def void apply(Project project) {

  project.convention.plugins.greeting = new GreetingPluginConvention()
  project.task('hello') &lt;&lt; {
      println project.convention.plugins.greeting.message
  }
}

During the configuration phase (before any targets are run) you will be given a Project object, and that Project object has a Convention object that holds onto a Map of all the project conventions. Three pieces of Groovy magic: Getters can be accessed with property notation, Map entries can be accessed with dot notation, and Map.put can be called with the = sign. So this line:

project.convention.plugins.greeting = new GreetingPluginConvention()

Is the same as the Java equivalent:

project.getConvention().getPlugins().put("greeting", new GreetingPluginConvention());

During the build, Gradle is going to execute the build script. Any time an unknown property is assigned or an undeclared method is invoked, Gradle will look at this plugins Map and use the respondTo method to check if any of the Plugin conventions accepts that variable. If it does, then your convention object gets invoked and that variable is set. Gradle takes care of translating an unknown method call of "Object greet(Closure)" into a call to your convention object. All that you need to do is make sure the greet(Closure) method is defined properly:

class GreetingPluginConvention {
  String message

  def greet(Closure closure) {
      closure.delegate = this
      closure()
  }
}

Now it is up to you to dispatch the parameters declared within the closure into fields on your convention object. If you just try to execute the closure then you will receive a nice PropertyMissingException because the "message" field is not declared anywhere. You get around this by defining a "message" field on your object and then setting "this" to the closure's delegate. Groovy method dispatch is flexible (complex?). Closures have a delegate object that will act as a sort of "this" reference when the closure executes. So the message field was out of scope when the closure was created, but adding a delegate makes the message field from the convention object in scope. When the closure assigns message a value, it will be the message on your Convention object. Check out my old blog post Fun with Closures for a more in depth explanation.

Now your convention object has the message String, hooray! You can do something interesting with it, like printing it out to the console:

println project.convention.plugins.greeting.message

Or maybe you had something more interesting in mind for your cool plugin. Let us hope.

By the way, you can just as easily declare a settings variable. This is an equivilent implementation of apply:

def settings = new GreetingPluginConvention()
project.convention.plugins.greeting = settings
project.task('hello') << {
  println settings.message
}

And now there is just one last piece of Groovy magic left unexplained:

apply plugin: GreetingPlugin

This too is plain old Groovy. Method parenthesis are optional. Method parameters can be passed in as name value pairs. And Class literals can be referenced without the .class extension. The apply plugin statement is really just a plain old method call in disguise, and it is an invocation of apply(Map). The apply statement can be written in Java as:

Map map = new HashMap();
map.put("plugin", GreetingPlugin.class);
apply(map);

Groovy magic is good. Thanks for taking some time to understand it. Hopefully this example makes it to the Gradle user guide in the next few days.

Unrolling Spock: Advanced @Unroll Usages in 0.4

2010-03-23T15:54:00.004-05:00

Some of the Spock Framework 0.4 features are starting to see the light of day, with the Data Tables being explained last week in a nice blog post from Peter Niederwieser. One of the new features that I had not seen before is the new advanced @Unroll usage. Mixed with Data Tables, it produces some very cool results, and it can still be used with 0.3 style specs as well. Here's the juice:

JUnit Integration and @Unroll
Spock is built on JUnit, and has always had good IDE support without any effort from you as a user. For the most part, the IDEs just think Spock is another unit test. Here's the a Spock spec for the new Data Tables feature and how it shows up in an IDE.

import spock.lang.*

class TableTest extends Specification {

  def "maximum of two numbers"() {
     expect:
     Math.max(a, b) == c

     where:
     a | b | c
     3 | 7 | 7
     5 | 4 | 5
     9 | 9 | 9
  }
}

The assertion will be run 3 times: once for each row in the data table. And JUnit faithfully reports the method name correctly, even when the method names has a space in it:

The problem with data driven tests and xUnit is poor error location. When a test fails you will receive an error stating which method is the culprit... but what if the method runs an assertion across 50 or 60 pieces of data? The cause of a failure is almost never clear with data driven tests. At it's worst you have to step through several iterations of code waiting for an exception. Good tests have a clear point of failure, but good tests also do not repeat themselves with boilerplate. This is exactly why Spock has the @Unroll annotation. As a test author you get to write one concise unit test, and JUnit does the work of reporting results that help you isolate failures. Consider the same test method with the @Unroll annotation and the accompanying IDE output.

@Unroll
def "maximum of two numbers"() {
   expect:
   Math.max(a, b) == c

   where:
   a | b | c
   3 | 7 | 7
   5 | 4 | 5
   9 | 9 | 9
}

When executed, JUnit sees three test methods instead of one: one for each row in the data table:

The end result for you as a test writer is accurate failure resolution. You can pinpoint exactly which row failed. This feature is available in Spock 0.3 and you can use it today. What is new in 0.4 is the ability to change the test name dynamically. Here is a full @Unroll annotation that changes the method name:

@Unroll("maximum of #a and #b is #c")
def "maximum of two numbers"() {
   expect:
   Math.max(a, b) == c

   where:
   a | b | c
   3 | 7 | 7
   5 | 4 | 5
   9 | 9 | 9
}

Notice the #variable syntax in the annotation parameter. The # produces a sort of GString-like variable substitution that lets you bind columns from your data table into your test name. The annotation parameter references #a, #b, and #c, which aligns with the data table definition of a | b | c. Check out the IDE output:

Previously, the test name was just the iteration number within the test. The new @Unroll parameter allows you to make the test name much more meaningful. Your tests will improve because failures become more descriptive. Unrolled failure messages before simply had the iteration name embedded in them, while now they can have meaningful data that you prescribe.

My favorite part of playing with the new @Unroll was to see the default value of the parameter within the Spock source code:

java.lang.String value() default "#featureName[#iterationCount]";

Talk about eating your own dog food... the default value is a test name template, just like you could have written in your own test. Makes you wonder what other variables are in scope, huh?

Spock snapshot builds for 0.4 are available at: http://m2repo.spockframework.org. Get it before the link breaks.

10 Lessons Learned from Usability Testing

2010-03-19T03:11:00.002-05:00

This morning I published an article on my lessons learned from 7 years of experimenting with usability testing. It is called "10 Lessons Learned from Usability Testing". Creative, huh? The post is on the Canoo blog instead of here... most modern web browsers will show you the article if you click the link above. I say most because I can't trust IE 6 to do anything right.

If you like it then please do the up-vote thing at DZone.

Happy Friday everyone.

Hackergarten in Basel Next Friday

2010-03-18T16:18:00.002-05:00

There is a new group starting up, and if you're anywhere near Basel Switzerland next Friday (March 26th) then you definitely want to stop by and participate.

Hackergarten is a group for people who define themselves by what they make, not what they consume. We've decided to meet regularly to create things as a group and give our work back to the community. Our goal is to create something that others can use; whether it be working software, improved documentation, or better educational materials. Our intent is to end each meeting with a patch or similar contribution submitted to an open and public project. Membership is open to anyone willing to contribution their time.

Our first event this Friday is a "Gradle Jam" at the Canoo offices in Basel. We are planning to add an "announce" plugin to Gradle that integrates with Twitter, Mongrel, and other messaging services. I expect a lot of pair programming, so please do not worry if you have no Gradle, Groovy, or Java experience.

The best way to get started is to join the mailing list and lurk around or follow the Twitter account @Hackergarten.

I expect this to be a lot of fun, a little work, and ultimately rewarding. Please do stop by!

Upcoming Gigs: Cologne, Frankfurt, Copenhagen, and Prague

2010-03-06T14:12:00.002-06:00

The Groovy roadshow is on tour this Spring! My benevolent benefactors are letting me out of the office to visit a few user groups and conferences:

March 8 - Cologne JUG - Groovy AST Transformations

May 3rd - 7th - Jax.de - Code Generation on the JVM

May 19th to 20th - GR8 Conference - Groovy AST Transformations

June 28th - Czech JUG - Code Generation on the JVM

Sadly, I am not going to be at GR8 North America... but if you are in the US then that is the event of the Spring, in my opinion.

I'm crazy excited to visit places I have never been to and meet other programmers. If you have any desire to see any of my talks then drop me an email. I think these are my three best sessions:

Code Generation on the JVM

We're seeing more and more JVM frameworks designed to generate code at compile time: AST Transforms, Project Lombok, Spring Roo, Clojure Macros, and more. This session reviews these approaches, including examples of how and why we'd want to do this. We'll see the newest Groovy language tools, write our own AST Transform, and look at some amazing libraries based on these techniques.

Slimmed Down Software: A Lean, Groovy Approach

The Groovy Programming Language advertises itself as an "agile and dynamic Language for the JVM", but what does this mean exactly? This sessions explain Lean Software Development, and shows how your choice of programming language can your entire process remain nimble and adaptive. Come learn about the principles of Lean, and see how Groovy and the associated ecosystem help eliminate waste, defer commitment, and build quality into your product.

Legacy Code, Groovy, and You

Thinking about writing Groovy unit tests for your legacy Java code? This session is an honest discussion about what Groovy will gain you and what it won't. Come learn the engineering practices and tools that you can use to battle tight coupling, monolithic projects, and tangled dependencies, and then decide for yourself whether Groovy is the answer for your project. Plan on returning to work with a vision of what your team can do to write better software.

Groovy ANTLR Plugins for Better DSLs

2010-02-19T09:03:00.002-06:00

DSLs, ANTLR, and Groovy in one blog post? Oh yes, this should be good: a trifecta of interesting keywords.

There is a horrible scourge upon Groovy based Domain Specific Languages, and no I'm not talking about curly braces. That awful syntax blocking our users from natural language based productivity is that insidious amalgamation of about 5 pixels called "the comma". If only we could rid ourselves and our users of this terrible burden!

Some are trying: GEP-3 is the Groovy Enhancement Proposal called "Command Expression based DSL" that will allow some commas to be optional. While interesting, there has not been much public activity on it recently. One of the requirements of dropping commas from methods invocations is that "the evaluation must be easily explainable". I think most people agree that we have more work to do before the proposal is easily explained.

So where does that leave us? Does Groovy force these spurious commas on unsuspecting programmers? Hardly. The compiler architecture of Groovy is fairly open, and with a little creativity you can make the commas optional in your DSL. As long as you have access to the CompilerConfiguration then you have options, whether it be an AST Transformation or an ANTLR Plugin. And remember, if you are using GroovyShell then you have access to it.

As an example, consider the easyb syntax for behaviors. What I would like to see is the comma dropped. Mmmm... it looks so much better without the commas:

given "some data" {
    println '... setting expectations'
}
when "a method is called" {
    println '... calling some method'
}
then "some condition should exist" {
    println '... making an assertion'
}

At some point in the compilation process, this source code will be represented as a text stream. What we're going to do is intercept that text stream, and provide some simple rewrite rules to add the comma in where it should be. An ANTLR plugin can intercept this text, add commas into it, and then pass it only to the Groovy compiler: Groovy is none the wiser. There is some boilerplate wiring together to do, but the bulk of the work is defining the rewrite rule (also known as a production). For the simple case, we can use a regular expression to add the comma in:

String addCommas(text) {
    def pattern = ~/(.*)(given|when|then) "([^"\\]*(\\.[^"\\]*)*)" \{(.*)/
    def replacement = /$1$2 "$3", {$4/
    (text =~ pattern).replaceAll(replacement)
}

Does a regular expression scale well to larger problems? No really. At some point you will need an alternative (maybe even at this point!) For instance, this Regex matches nested quotes, but the quotes must be double quotes, and Groovy's single quotes and multiline strings are not supported. Oh well, it is just an example.

The "wiring together boilerplate" consists of subclassing AntlrParserPlugin so that you can write the text and subclassing ParserPluginFactory so you can wire in your AntlrParserPlugin subclass. The ParserPluginFactory can then be passed directly to the CompilerConfiguration which is passed to GrovoyShell. That makes no sense to me even as I write it, so it is probably best to go look at the full source code listing in Groovy Web Console.

For those of you using browsers not supporting anchor tags, here is the code inline:


class SourceModifierParserPlugin extends AntlrParserPlugin {
    Reduction parseCST(SourceUnit sourceUnit, Reader reader) throws CompilationFailedException {
        def text = addCommas(reader.text)
        StringReader stringReader = new StringReader(text)
        super.parseCST(sourceUnit, stringReader)
    }
}

def parserPluginFactory = new ParserPluginFactory() {
    ParserPlugin createParserPlugin() {
        new SourceModifierParserPlugin()
    }
}

def conf = new CompilerConfiguration(pluginFactory: parserPluginFactory)
def binding = ... 
def shell = new GroovyShell(binding, conf)

And once you have a GroovyShell you can evaluate the world! Including the pseudo-easyb script from the beginning of the post. It runs with no problems... missing commas and all.

ANTLR plugins have been around Groovy for a long time, and this example is based off of Guillaume Laforge's famous Groovy Web Console Script #3. There is nothing particularly hard about writing an ANTLR plugin, but there might be something difficult with maintaining it. DSLs come with a host of issues including versioning. If you create an external DSL then you've published a language. It's fun at first but not so much later.

And there you have it. No more of those despicable commas! Now we just need to do something about those dispicable DSLs.

Testing Asynchronous Code with GPars Dataflows

2010-02-19T01:59:00.002-06:00

In my last post I showed how to use JConch 1.2 to unit test asynchronous code. It contains a locking/barrier mechanism that allows you to gracefully tell your unit test how to wait and proceed for off thread events without sleeping or polling. Whee!

As a small example, here is how the TestCoordinator would be used to test a mouse click event that happens on a different thread:

TestCoordinator coord = new TestCoordinator()

MyComponent component = new MyComponent()
component.addActionListener({ ActionEvent e ->
   assert "click" == e.actionCommand
   coord.finishTest()
} as ActionListener)

component.click()
coord.delayTestFinish(1, TimeUnit.SECONDS)

If you are living in a Groovy world, then you should consider using GPars Dataflow variables instead of the JConch TestCoordinator. The advantage of using Dataflows is that you get a similar thread coordination API but also get a variable capture API, allowing you to move assertion statements to the end of your unit test where they belong. If you are looking for a clean, minimalist approach to capturing off thread events, then dataflows might be exactly what you need:

import java.awt.event.*
import javax.swing.SwingUtilities
import groovyx.gpars.dataflow.DataFlowVariable

def event = new DataFlowVariable()

MyComponent component = new MyComponent()
component.addActionListener({ ActionEvent e ->
   event << e
   } as ActionListener)

component.click()

ActionEvent e = event.val
assert e.source != null

assert e.actionCommand == "click"

A DataFlowVariable is a little like a Java Future mixed with an immutable, thread safe object. The DataFlowVariable has an underlying value which can be set once, and only once, and trying to access that value will block the caller until the value is available. In this example, setting the value is done in the event handler with the "event << e" line, and accessing the value is done in the "event.val" line. Getting the "val" will block the caller until a value is available. (Remember, in Groovy .val property access is translated into a getVal() method call).

The advantage of using Dataflows in testing, when compared to Java, is that you get a nicer thread synchronization API than what the java.util.concurrent primitives provide. The advantage of Dataflows, when compared to JConch, is that you get to move your assertion methods back to the end of the unit tests. William Wake described a format for unit tests called Arrange-Act-Assert, and it is still one of the best guidelines to writing clear and understandable unit tests. Assertion methods belong at the end of your test and arrangement code belongs at the beginning (note how many mock object frameworks subvert this ordering!). Capturing a variable is a good way to move the assertions to the end of the method, but in most frameworks it requires copious amounts of accidentally complex code.

The downside of using Dataflows is that you once again have to set an @Timeout value on your unit test in the event that you never get a value (otherwise your tests will hang). OH WAIT, that is totally wrong. The GPars authors did provide a timeout value on DataFlowVariable#getVal():

ActionEvent e = event.getVal(4, TimeUnit.SECONDS)

I had you there for a second though, right? Be warned, if the timeout value is exceeded then getVal() returns null, it does not throw an exception.

The only downside to using DataFlow concurrency that I can see is that you actually need to understand a little bit about DataFlow concurrency. Who would have thought? DataFlowVariable is just the tip of the iceberg when it comes to GPars DataFlow support, and is not even the primary use case. The Dataflow User Guide is a good read and will explain a lot more about the concepts and GPars' implementation.

Feel free to try this yourself. You don't even need to download any jars or change your project files. Just add the correct Grapes to the top of your unit test:

@Grab(group='org.codehaus.gpars', module='gpars', version='0.9')
@GrabResolver(name='jboss', root='http://repository.jboss.org/maven2/')

If you need more help, then check out the full example.

Happy Testing!

Asynchronous Unit Test Coordination with JConch 1.2

2010-02-09T10:00:00.000-06:00

Slowly but surely, JConch Java Concurrency Library is becoming a depot for multithreaded and asynchronous testing on the Java platform. First there was SerialExectorService, allowing you to test with a Java ExecutorService that never started threads (example, javadoc). Then there was assertSynchronized, allowing you to make easy unit test assertions about your object's synchronization policy (javadoc, example).

And now JConch 1.2 offers TestCoordinator: a tool for testing asynchronous code and multi-threaded callbacks. The TestCoordinator solves the problem of how to properly wait for asynchronous method calls without littering your unit tests with wait/join calls or CyclicBarrier/CountDownLatch API calls. In this regard, it provides a useful unit testing abstraction over the great Java 5 concurrency primitives.

Note: If you'd like to skip all the prose then go straight to the example and unit tests that are part of JConch 1.2.

Writing good unit tests requires the same input as writing good production code: practice, dedication, and about 10,000 hours hours experience. Almost any component framework that requires some sort of event listener invariably has a unit test that looks like this:

MyComponent component = new MyComponent()

ActionEvent event
component.addActionListener({ ActionEvent e ->
   event = e
} as ActionListener)

component.click()
Thread.sleep(1000)

assert event.source != null
assert "click" == event.actionCommand

OK, usually it's in Java and not Groovy, but you get the point. Life's too short to write Java at home. The simple format is 1) create a component with a listener that captures the input, 2) activate the component, and 3) put the thread to sleep for a while and hope that the event fires before your assertions get run. The obvious problem is that you end up with either slow or intermittently failing unit tests depending on how long you sleep (no, there really isn't a perfect sleep number that avoids both).

In the Java 4 days this was "solved" by monkeying with Object#wait and Object#notify method calls. Ewww. Java 5 introduced CyclicBarrier and CountDownLatch which solved the problem nicely albeit in a primitive way. It is an improvement, but we can do better than this:

@Test(timeout=5000)
public void testClick() {
 CyclicBarrier gate = new CyclicBarrier(2)

 MyComponent component = new MyComponent()

 component.addActionListener({ ActionEvent e ->
    assert "click" == e.actionCommand
    gate.await()
  } as ActionListener)

 component.click()
 gate.await()
}

This approach allows us to move the assertion methods inside the callback, because we're insured that either the callback will be called or the test will fail with a timeout (the @Test(timeout=5000) bit). It is simpler and hides some complexity, but there is still a bunch of primitives distracting the reader from the core content of the test... what is that (2) parameter on the barrier constructor? Is the timeout value clear or is it hidden within a method annotation? And just what does await() mean? There is a lot of primitiveness involved here, which is what TestCoordinator abstracts over:

import jconch.testing.TestCoordinator

TestCoordinator coord = new TestCoordinator()

MyComponent component = new MyComponent()
component.addActionListener({ ActionEvent e ->
  assert "click" == e.actionCommand
  coord.finishTest()
} as ActionListener)

component.click()
coord.delayTestFinish(1, TimeUnit.SECONDS)

As you can see, TestCoordinator acts alot like a barrier or latch, but without all the low level API. When you've activated your class under test then you call delayTestFinish(...); the unit test will wait unit the listener calls finishTest(). And if finishTest() has already been called (a frequent possibility with multithreaded systems), then delayTestFinish just proceeds. If the timeout value expires then the test fails. No funny timeout annotation. No funny API. Just a simple coordinator. You want to delay a test until a callback is finished, and the API is designed around this nomenclature.

There are two other options to delay as well:

coord.delayTestFinish(1000) // milliseconds
coord.delayTestFinish()     // requires @Timeout value!

For those of you coming from GWT... yes, this is almost exactly the API of GWTTestCase. Use what works, I say.

For more info on TestCoordinator, check out the UnitTest and the example. Happy Testing!

New in Groovy 1.7.1: Constructor Mocking and Half Mocks

2010-02-07T09:58:00.002-06:00

The Groovy MockFor object got some fun new features this weekend: constructor mocking and "half-mocks". The tickets are marked for Groovy 1.7.1, so these features are available in the nightly builds or in the next few weeks as 1.7.1 is released. The easiest thing is, of course, to just build from source.

The first feature is Constructor mocking. It is now possible to specify a return value for mock constructor calls. Here is an example that uses a "Person" object. The "tom" variable is a real Person object, while the "mary" object is not. Thanks to this new feature, instantiating Mary returns to you Tom.

import groovy.mock.interceptor.MockFor

class Person {
 String name
}

def tom = new Person(name:'Tom')

def mock = new MockFor(Person, true)

mock.demand.with {
 Person() { tom }
}

mock.use {
 def mary = new Person(name:'Mary')
 assert tom == mary
}

There are two parts to getting this working: the "demand.with" block, where the person constructor is specified to return tom, and the true parameter passed to MockFor. To make MockFor backwards compatible, a flag was needed to control when to allow constructor mocking and when not to allow it. As you can see, after doing this, the tom and mary objects are equal (in fact, the same reference).

The problem with this feature alone is that calling tom.getName() results in a mock exception saying no demand is set. If Tom is a real instance of Person then why would you get a demand exception? Well, within the mock.use block Tom is a mock. The behaviour you probably expect is to have the real Tom methods pass through to the real implementation, which is why "half-mocks" were introduced. Check out how you can specify methods to ignore and pass through to the underlying object:

mock.ignore(~'get.*')
mock.demand.with {
 Person() { tom }
}

mock.use {
 def mary = new Person(name:'Mary')
 assert mary.name == 'Tom'
}

The ignore method takes a pattern to match, and the mock ignores those methods, passing them on to the underlying implementation. So your mock is still a mock, not a real object, but methods are passed through as needed. This is the opposite of how EasyMock partial mocking works, where a partial mock is technically a subclass of the target class, and methods are routed by default to the real implementation. In Groovy, the method are routed by default to the mock.

Fun stuff, and thanks to Paul King for the work!

Sharing IntelliJ IDEA Project Files in Version Control

2010-01-22T15:45:00.008-06:00

There are quite a few benefits to having IDE project files properly stored under version control:

most people want to use an IDE and will need to configure the project somehow
a shared environment reduces costs within a corporate workplace
a VCed project means that the project file branches with your codebase. Need to work on an old branch? No problem, the project file already exists.
casual contributors will have an easier time making patches for your project
an IDE can provide checkstyle and coding standards to your code

And when the files are stored improperly in version control, you create a nice changelist headache for yourself and turn away the casual contributor.

Here are three simple rules on how to properly share an IntelliJ IDEA project file in version control:

Rule #1 - Check-in the .ipr and .iml files
Rule #2 - Set an ignore flag for the .iws file (your workspace file). All proper VCSs allow you to do this.
Rule #3 - Make IDEA Project Variables for all the variables that vary (it's kinda the definition of variable, you know). Some common variables you'll probably need are "Tomcat Home", "Groovy Home", and "JDK Version".

Creating variables is really easy. Open the .ipr/.iml in a text editor and scan for hardcoded paths or values. When you find one just replace it with $variable_name$. Now when IDEA starts, it prompts you for those variables if they are not defined. These are IDEA variable stored in your HOME directory; they have nothing to do with your operating system environment variables. Here's what a prompt looks like:

See that red dialog? Clicking it opens a screen where you can add values to the variables:

Adding a value should be pretty self-explanatory.

The Gotchas - There are a few gotcha's to be aware of...

Using multiple versions of IDEA is not well supported. Most new versions, even minor releases, try to write some extra info to the project files. Not a problem in the corporate world, but more so in the OS one. You might try checking in a backup copy when the project lead upgrade.

Making the JDK Name variable can be somewhat confusing, as it is not a path on your disk. IDEA keeps track globally (for all projects) of your installed JDKs, and each one is given a name. It is easy to find the place in the .ipr where this is specified, but it is not easy to explain to users what their's should be set to. It needs to be the name of an entry from your SDKs in Settings:

Also, changing the inspection profiles can modify the project. The inspections are the codestyle to the project, so it may make sense to have them baked into the .ipr. However, this is an optional place to put them. An inspection profile can be shared or not. If it is shared then it is written to the .ipr, otherwise it is an external file. The Settings screen should be self-explanatory once you understand the Shared checkbox.

Lastly, adding words to the spell checker will add the word to the .ipr file, with a key of your username. I opened a bug for this, as I feel a user's name should never go in the project file, but the JetBrains guys feel that this is OK considering that switching to the ".idea" alternate project layout will separate the files out and avoid the issue. For now, I guess the best advice is to spell words correctly and, if you're not a committer, live with the fact that your project file may be checked out because of this issue.

Is all this worth it? I recently opened the GPars project files, defined an SDK and a Groovy 1.6.7 library, and ran the tests successfully. For me, it was great.

Rule #4 - Profit!

Accelerated GWT Report Card

2010-01-11T13:26:00.001-06:00

Student Name:	Accelerated GWT
School:	Apress
School Year:	2008
Teacher:	Hamlet D'Arcy
Overall Grade:	A-Strong on client side, i18n, and advanced topics.

Grading Scale: A=Outstanding, B=Good, C=Average, D=Poor, F=Fail

Teacher Comments

This was a good book and a worthwhile read. It was very strong on the client side and documented many edge cases of the GWT framework without reducing itself to an overly specific "recipe" book.

Client Side Programming

A-

Knowledge of Standard GWT Widget Set

A-

Great coverage of basic widget set. Contains examples and advice about how to layout widgets in various containers, including composing widgets in the Composite class. Sometimes the widget API was explained too completely and it felt like reading Javadoc, but these sections can easily be skimmed. Also, this book was written for GWT 1.4, so UiBuilder is not covered at all.

Knowledge of CSS Styling

Great coverage of how to work with the CSS of widgets including several good examples. Best coverage of the topic I could find.

General Client Concerns

Overall, the client sections are excellent. Serializing your own classes is covered, the output of the GWT compiler is given fair due, and generators for creating custom JavaScript code at compile time are covered. So far, this is the best book on client side GWT that I've found, even though it doesn't cover the 2.0 features.

Knowledge of JSNI (JavaScript Native Interface)

JavaScript Native Interface is barely covered. In comparison, GWT in Practice covers it maybe too much. Embedding JavaScript in multiline Java comments is the most controversial part of GWT, but it's not the most important. Still, a little more info would be helpful.

Server Side Programming

Knowledge of GWT RPC (Remote Procedure Calls)

GWT RPC is covered in 14 pages. Do you need more than that? Maybe not, it's pretty straight forward. The appendix does cover some of the newer 2.0/1.5 features, like generics.

Knowledge of Internationalization

A+

Over 30 pages on i18n, including an example using localization at either compile time or run time. Very good chapter.

Knowledge of Security Concerns

B+

Same origin policy and cross site scripting: check. Not as lengthy coverage as in other books, but adequate.

Knowledge of GWT Testing

GWT Testing is surely the easiest chapter of the book to write. GWT ships with GWTTestCase for testing asynchronous calls and a benchmarking tool for measuring performance, and they are clearly explained all over the Internet. Speed Tracer, the 2.0 profiler that runs in Chrome, is of course not covered.

Creative Code Samples

Decent enough code samples (with an occasional error) covering browser history and server sessions, as well as the other topics mentioned. Some of the code samples were a little long and ImageBundle was given too much space given that it is deprecated in 2.0.

Knowledge of Alternative Server Stacks

Nothing on the server side beyond GWT RPC is covered.

GWT Best Practices

B+

Tools to Write GWT Clients

The Enterprise means constraints: existing domain objects, maybe a crummy DAO, etc. Enterprise GWT apps will need things like custom serializers and custom generators to work around their legacy constraints, and Accelerated GWT covers these topics well.

How to Design GWT Services

General advice about domain objects and writing RPC services is given and a DTO approach is avoided, which pleases me. Overall, however, not a lot of information is given on building "Enterprise" backends, which is part of the book's title.

60 Second Agility: ROTI Meetings

2010-01-10T04:37:00.003-06:00

Always in search of the absolute minimum of ceremony, my last team "discovered" a useful agile practice that takes 60 seconds from start to end: the ROTI Meeting.

After every meeting, on the way out the door, draw a diagonal line on the whiteboard with the labels 0, 2, and 4.
Each person in turn gives a number on how the meeting performed as a "Return on Time Invested" and the person with the marker draws in the rating. Here is the rating scale we used:

0 = "I'd have been better off making a Starbuck's run. Complete waste of time"
1 = "You really should have let me stay at my desk and code"
2 = "This was an OK meeting. About as valuable as if I'd been coding"
3 = "Surprisingly, this was more valuable than if I'd been writing code"
4 = "Wow, this meeting saved me tons of time. Thank goodness I didn't skip it to code"

And then each person answers the same question, "What could be done to improve your number by one point?"

To do this in 60 seconds means there is no discussion. The feedback is what it is; no debating, no fixing problems, and no hurt feelings.

ROTI meetings create tacit, organization knowledge that can be acted upon by team members in the future. It drives a team towards less meetings (almost always a good thing), pushes team members to be more respectful of each others time and expertise, and influences meeting organizers to craft more succinct, on topic, and meaningful gatherings. It takes only 60 seconds so you might as well try it a few time!

... and now the historical details.

ROTI analysis is nicely described in Esther Derby's great book "Agile Retrospectives". The practice in the context of iteration retrospectives takes more lie 5 to 10 minutes. Our team found ROTI to be so effective in retrospectives that we shortened it and held one at the end of every meeting.

The actual ROTI scale is a bit more formal than what we created:

0 - Lost Principle: No Benefit Received for Time Invested Break-Even:
1 -
2- Received Benefit Equal to Time Invested High Return on Investment
3 -
4 - Received Benefit Greater than Time Invested

Lastly, ROTI charts are covered in detail a few other places as well.

For a mere 60 second investment, this practice is worth trying on your team.

"GWT In Practice" Report Card

2010-01-09T15:48:00.007-06:00

Student Name:	GWT In Practice
School:	Manning
School Year:	2008
Teacher:	Hamlet D'Arcy
Overall Grade:	B (solid effort; time well spent; not enough on client side)

Grading Scale: A=Outstanding, B=Good, C=Average, D=Poor, F=Fail

Server Side Programming	A
Knowledge of GWT RPC (Remote Procedure Calls)	A+
How Ajax calls are made from the client to the server using GWT RPC is clearly explained in a variety of use cases (almost too many!). The difficult to understand relationship between service interfaces, server/servlet implementations, and the client asynchronous interface is explained in words, with diagrams, and within the IDE. Explaining GWT RPC is where GWT in Practice really shines... and this topic is still relevant today in GWT 2.0.
Knowledge of Alternative Server Stacks	A
GWT in Practice demonstrates how to consume REST and POX services with GWT, how to integrate Flash for SOAP services (not a good idea IMO), how to use Java Applets(!) for a SOAP client, and how to use Comet for streaming data to clients. The book focuses so much on server side technologies that perhaps it should have been called "GWT Server Stack Recipes".
Knowledge of Security Concerns	A
The Manning "In Practice" imprint means the author is targeting real world developer concerns rather than giving a generic overview, and security is one of these real world concerns. GWT in Practice does a great job explaining all the cross site scripting concerns (and how to work around them), security certificate issues (and how to work with them), and how to configure Tomcat security (which isn't even unique to GWT). It's clear the authors have real experience with securing a GWT app in a variety of contexts.
Knowledge of GWT Testing	B
GWT ships with both GWTTestCase (for easing the testing of asynchronous and multithreaded components) and a performance benchmarking tool. The tools are straightforward and covered in every other GWT text as well. While a testing section is essential to any GWT book, this one didn't stand out against the other material. I'd have liked to see more testing best practices. Plus, GWT ships with a new performance testing tool, so this section is starting to age.
Creative Code Samples	B
The book deomonstrates how to turn on history to support the browser back button and how to record and playback certain Comet messages. The former is useful (and part of the GWT project) while the latter seems unlikely to be needed by almost any other GWT developer (and is custom implemented for the book).

Client Side Programming	C+
Knowledge of Standard GWT Widget Set	C
Granted, the "in Practice" title is not meant to be a basic introduction, but there is almost no material on the standard GWT widget set. If you want to know all the user interface widgets supported by GWT then look elsewhere. However, some widget layout and CompositeWidget material was covered, the GWT in Practice salvages a C grade.
Knowledge of JSNI (JavaScript Native Interface)	B
Surely the most controversial part of GWT is the ability to embed JavaScript within multiline comments of Java files. Sounds nasty even if IDEA does support refactoring and highlighting, right? (And as of 2.0, Eclipse does as well). This books contains a ton of JSNI. Pages of it, actually; which isn't always a good thing for the book or for GWT. GWT in Practice will definitely show you how to wrap JavaScript libraries, even if it makes for poor reading. The best part of the JSNI examples was the recipe for adding drag and drop, which appeared twice in the book for some reason. If you need drag and drop or to wrap a JavaScript library then this is the book for you!
Knowledge of CSS Styling	B
CSS styling gets a whole 2 pages. Which means there isn't much to say beyond what the browser offers already.
General Client Concerns	C
There simply wasn't enough information about client side programming in GWT in Practice. The example that were shown left me wondering, "Is that something I'll really need to do?"

GWT Best Practices	B+
How to Design GWT Services	A
This is the only book I found that included best practices about how to design services, which the author recommended using DTOs and keeping interfaces seperate from implementations to decouple physical services. However, some of the diagrams seemed forced and pointless (and they were, did you know Manning requires a visual aid on every spread?)
How to Design GWT Clients	F
What should client side design be in an Ajax application that consists of pretty much just one web page? This wasn't covered at all. Note to authors: please include this in your GWT 2.0 update! Also, there was no coverage of how to deal with degenerate browsers like IE6. This is the bane of my current project and any hints would be appreciated.
Knowledge of the Best Tools for the Job	B+
GWT in Practice shows NetBeans, Eclipse, and IDEA in action. The most helpful diagram of how a GWT project is structured was actually an IDE screenshot. For builds, the Maven plugin is covered ad naseum, which is understandable considering the authors wrote it. But still, Maven is dead to me so it was time wasted (now where is that Gradle GWT plugin)?

Teacher Comments

Even in light of the GWT 2.0 release, GWT in Practice holds up as a solid GWT book for newcomers. It definitely favors the server side and gives the client side short shrift. My biggest complaint was that it seemed to be a lot of recipes bundled into an "in Practice" format, and many of the recipes I didn't need (Flex? Applets?). Given the book is from 2008 when GWT had just been released, I wanted to dislike it. But it's held up quite well. It should not be the only GWT book you read, but probably it should be one of them.

Groovy AST Transformations by Example: Adding Methods to Classes

2010-01-07T01:31:00.003-06:00

What can you do with a Groovy AST Transformation? A difficult question, considering the answer is "almost anything". One frequent use of AST Transformations is to write Groovy changes into the JVM .class files so that standard Java tools will know about them. For instance, the bytecode changes produced by @Delegate are visible and usable from plain old Java classes. This example is going to walk you through how to add synthesized methods to a class at compile time using a Groovy AST Transformation and the AST Builder. It makes an interesting test case since it explores some of the edge cases and common pitfalls. This should be fun...

(And for those of you who want to skip all this and just see the code, check out the latest Groovy example in SVN and just run the Ant script).

svn co http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/examples/astbuilder

To start, let's better define our example. Consider the class with a public API and also a standard main() method. Frequently, the main() method simply instantiates the class and calls a single method. Wouldn't it be nice to annotate a method so that it is treated like a main method? What if this were possible:

class MainExample {

   @Main
   public void greet() {
       println "Hello from the greet() method!"
   }   
}

At compile time, this annotation would synthesize a proper main method and run the same code as greet(). (We'll worry about calling an instance method from a static method later). Transforming this annotation in the AST means that both Java and Groovy would see the main() method. Let's do it; there are three things you need:

An @Main marker interface
An ASTTransformation implementation make the AST changes
A test harness (trust me, you want this)

The @Main marker interface is the easiest piece. The marker interface is used to a standard Java 5 annotation to an ASTTransformation implementation:

@Retention (RetentionPolicy.SOURCE)
@Target ([ElementType.METHOD])
@GroovyASTTransformationClass (["MainTransformation"])
public @interface Main {  }

This is just a standard Java 5 annotation. The retention policy is SOURCE so that other objects don't see the annotation at runtime when using reflection (there is no need). The ElementType is METHOD because we only want to apply the annotation to methods. And the GroovyASTTransformationClass is "MainTransformation". This value is the fully qualified classname of the ASTTransformation implementation that the Groovy compiler is going to call when an @Main annotation is found. In practice, you will need to specify a package... I left one off here for simplicity.

So this annotation will wire @Main into a class called MainTransformation. I suppose we'd better go make this class then...

The ASTTransformation interface defines one method:

void visit(ASTNode[], SourceUnit)

The best way to understand the contents of the two parameters is to write several tests and inspect them in a debugger, and then use the AST Browser to analyze the syntax tree. In general, the ASTNode array contains what you want to manipulate and the SourceUnit provides services like error and warning reporting. Here is the skeleton of our MainTransformation that is going to add a synthesized main() method to the class:

@GroovyASTTransformation(phase = CompilePhase.INSTRUCTION_SELECTION)
public class MainTransformation implements ASTTransformation {
  
   void visit(ASTNode[] astNodes, SourceUnit sourceUnit) {

       if (!astNodes) return
       if (!astNodes[0]) return
       if (!astNodes[1]) return
       if (!(astNodes[0] instanceof AnnotationNode)) return
       if (astNodes[0].classNode?.name != Main.class.getName()) return
       if (!(astNodes[1] instanceof MethodNode)) return

       MethodNode annotatedMethod = astNodes[1]
       ClassNode declaringClass = astNodes[1].declaringClass

       // makeMainMethod synthesizes a method!
       MethodNode mainMethod = makeMainMethod(annotatedMethod)
       declaringClass.addMethod(mainMethod)
   }
   ...

Again, there are a few interesting things to notice here...

The compiler phase chosen was INSTRUCTION_SELECTION, which was a completely arbitrary decision. Please, if you want to know which phase to target then ask on the groovy-user mailing list (and asking will give us better hints as to what we need to document on the wiki).
The body of the transformations contains many guard clauses as a safeguard against NullPointerExceptions and unexpected input. The AST of classes changes from version to version. Defensive programming and thorough test cases highly recommended.
The MethodNode that was annotated comes through in the ASTNode array, and from there you can navigate most anywhere within the syntax tree.
The makeMainMethod is going to copy the body of the annotated method and then we will add that method back onto the declaring class.

The last thing to do is define the makeMainMethod that actually creates the main():

MethodNode makeMainMethod(MethodNode source) {
   def ast = new AstBuilder().buildFromSpec {
       method('main', ACC_PUBLIC | ACC_STATIC, Void.TYPE) {
           parameters {
               parameter 'args': String[].class
           }
           exceptions {}
           block { }
       }
   }
   MethodNode newMainMethod = ast[0]
   newMainMethod.code = source.code
   newMainMethod
}

Again, a few bits and pieces to make special note of:

The AstBuilder.buildFromSpec is a DSL convenience over the concrete constructors of ASTNode and its subclasses. Thus, the API of buildFromSpec mirrors the API of the ASTNode classes. If you want to understand the arguments and parameters types, then look at the ASTNode Javadoc. In this regard, the buildFromSpec is more about making the code easier to read rather than easier to write.
The return type of the method is Void.TYPE and not Void.class. Void.class is a real class with real usages, Void.TYPE is the symbol for "little vee void".
The modifiers are ACC_PUBLIC | ACC_STATIC. Notice the use of | and not &. Let me editorialize briefly: I don't understood bitmasks, I never understood them, and people who design APIs around parameters typed as "int modifers" will be first against the wall when the revolution comes. If null was the billion dollar mistake then bitmasks at least deserve to be in the tens of millions range. Anyway...
The codeblock of the new main method is just a reference to the codeblock of the method annotated with @Main. This will cause duplicate bytecode to be written out into the .class file (bad) but does make for a small, clear example (good). In production, you probably want to invoke the @Main method instead of copy the body.

Now, as you can see from the javap output, the compiled MainExample.groovy contains a real, live Java main class:

hdarcy:$ groovyc MainExample.groovy
hdarcy:$ javap MainExample
Compiled from "MainExample.groovy"
public class MainExample extends java.lang.Object implements groovy.lang.GroovyObject{
   ...
   public void greet();
   ...
   public static void main(java.lang.String[]);
   ...
}

Phew! That was a lot of information. The last thing to add is a simple test case using Groovy's TranformTestHelper class. A debugger helps AST Transformation development immensely, and this integration test is one way to hook one up to your transformation. I tested this in IDEA but it should work fine in Eclipse/Netbeans as well:

class MainIntegrationTest extends GroovyTestCase {

    public void testInvokeUnitTest() {

       def file = new File('./MainExample.groovy')
       assert file.exists()

       def invoker = new TranformTestHelper(new MainTransformation(), CompilePhase.CANONICALIZATION)

       def clazz = invoker.parse(file)
       def tester = clazz.newInstance()
       tester.main(null)       // main method added with AST transform
   }
}

This test case takes the example class as a simple File and then compiles it using the MainTransformation we defined. This is a great way to invoke both local and global transformations from unit tests. I'm sure there is an easier way to invoke the static main method without instantiating the class, but I'm tired and don't care all that much. It's just an example!

And that's it... we created a marker interface to trigger an annotation, created an ASTTransformation to add a method onto a class, and wrote a test harness. There are just two loose ends to tie up:

How can a static main method invoke a non static method like Greet? We'll, it can't really. For this example we just copied the method body into the main method. In practice, you'd want to make sure only static methods were annotated.

How are inner classes handled? Short answer: not the same way. You have a test case, so just write a test method and find out!

All the code was checked into the Groovy examples folder along with an Ant script. There's no better time to get the source:

svn co http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/src/examples/astbuilder

cd examples/astbuilder and away you go.

Java 5 + GPars: Throttling Action Processing

2009-12-12T22:42:00.002-06:00

An interesting question came up on the GPars mailing list today: In a system that generates events, what is the best way to throttle back event processing to one event per second? I thought about an answer... then thought some more... and finally decided to write it all up in this blog post. The example uses Groovy and GPars, but it is easily adapted to a generic Java solution. Don't let the actors scare you! (or the lack of semi-colons, for that matter).

The example is the classic "Sleeping Barber" problem (I hadn't heard of it either). Basically, there is a barbershop. The barber is asleep. Customers walk into the waiting room periodically, and the barber wakes up to give each of them a haircut. When he's done he returns to his slumber. It's a lesson in reaction: something is asleep, then awakens to do some work, then returns to sleep.

The GPars docs provide a decent Actor based solution to this problem: there is a waiting room and a barber, and both are actors. When the barber is free, a customer from the waiting room is moved into the barber's chair. The barber and waiting room communicate via actor messages. But what if our barber is a bit of a diva, and no matter how busy the shop gets, he wants to give one haircut every 15 minutes and never any more (otherwise, he might get burnt out you see). That is the throttling problem: how do you make sure events are processed (a haircut is given) no more than x number of times in a given time period?

My solution: keep a work queue, and have a scheduled executor pull work off the queue at a specified interval. Java 5 gives you all the tools to do this without resorting to busy waiting, polling, or writing scheduling code. The classes you need to know about are ArrayBlockingQueue and ScheduledThreadPoolExecutor.

ArrayBlockingQueue is a FIFO (First-In-First-Out) queue that supports blocking instead of busy waiting. When you take() an item from an ABQ, the call blocks until an item is available... no polling or sleeping to see if an item is ready to available. Just call take() and your code won't proceed until there is an element found.

The ScheduledThreadPoolExecutor supports executing both Runnable and Callable objects at a fixed interval. If you're looking to execute the same task every 1 second then STPE is what you need... Timer, for all intents and purposes, has been deprecated.

So here's the barber that just won't stand to be over-worked... setting it all up we need customers, a barber, and a waiting room:

class Customer {
 String name
}

// waiting room can't hold more than 12!
def waitingRoom = new ArrayBlockingQueue(12)

def barber = Executors.newScheduledThreadPool(1)
barber.scheduleAtFixedRate({
     println 'Barber: Next customer please!'
     Customer customer = waitingRoom.take()
     println "${customer.name} gets a haircut at ${new Date().format('H:m:s')}"
 } as Runnable, 0, 15, TimeUnit.MINUTES)

Customer is a simple bean; nothing interesting here. The waiting room is an ArrayBlockingQueue filled with customers that need a haircut. And the barber is an executor service with a scheduled task to give haircuts. The number of threads in the scheduled thread pool is 1 because there's only one barber. The barber takes customers from the waiting room and cuts their hair once every 15 minutes. The call to waitingRoom.take() is blocking... if there is a customer ready then he is serviced immediately, and if one is not, then the call blocks until someone is available. Once thing to note... the waitingRoom has a size of 12... if a 13th customer is added then the calling code will either block until there is enough room or throw an exception. There is an API to do either case.

So how do customers get into the waiting room? That's where GPars actors come in. The barber shop is a "reactor" in GPars terminology. Messages can be sent to the barbershop ("Enter" the waiting room), and the reactor adds the customer to the waiting room. A maître d' of sorts. Here it is in action:

def barberShop = new PooledActorGroup().reactor {message ->
 switch (message) {
     case Enter:
         println "${message.customer.name} waits for a haircut..."
         waitingRoom.add(message.customer)
         break
 }
}

class Enter {
 Customer customer
}

barberShop << new Enter(customer: new Customer(name: 'Jerry'))
barberShop << new Enter(customer: new Customer(name: 'Phil'))
barberShop << new Enter(customer: new Customer(name: 'Bob'))
barberShop << new Enter(customer: new Customer(name: 'Ron'))

The barberShop is a PooledActorGroup, a GPars object, and the "actor framework" just means adding a closure to the reactor() method of that group. The closure, or actor, and responds to Enter messages by adding the customer to the waitingRoom. At the bottom you see the nice << syntax for posting events to the ActorGroup.

So there you have it. There are many ways to do this, but I think the Java 5 Concurrency libraries are some of the best options. I'd be interested to hear other ideas too. Now go give those hippies some haircuts!

Groovy Shell gets some Color

2009-12-12T09:07:00.002-06:00

Well, well, it looks like Groovy Shell (groovysh) is starting to catch up to the other IDEs. Yesterday, support for colored output was added!

The Jansi library has added to the Groovy 1.8 snapshot, and now you get color coded output on all platforms, including Windows. Here is an Ubuntu screenshot:

And here's how to make it work in 3 easy steps:

1) Download Jansi and put it in your classpath
2) Get the nightly Groovy install*
3) Invoke groovysh with the -C parameter!

* As an alternative, it is incredibly simple to build Groovy from source:

svn co https://svn.codehaus.org/groovy/trunk/groovy/groovy-core
ant install -DskipTests=true

Happy Evaluating!

Groovy 2009 Year in Review and 2010 Predictions

2009-12-10T20:07:00.003-06:00

Groovy in 2009

Groovy 1.6 Released
Groovy 1.6 was released at the beginning of the year, and the most exciting new features have turned out to be Grape and AST Transformations.

Grape allows a developer to declare dependencies within their Groovy source code and then, at runtime, Groovy will download and install the dependencies using Ivy repositories. Want to ship scripts to your operations team? You no longer need to email or build JARs! It's no Jigsaw, but maybe that's a good thing.

AST Transformations allow developers to hook into the Groovy compiler and alter the way the code is compiled. This has enabled great work like the @Bindable and @Delegate annotations, as well as many forward thinking (read: visionary) libraries like the Spock testing framework and the CodeNarc static code analysis tools. Watch for more cool libraries and frameworks to use these features in 2010.

Griffon Released
While Flex and JavaFX duked it out for developer mindshare in the coveted (and hyped) RIA space, a groovier team quietly forked the Grails codebase and adapted it for Swing desktop applications. Griffon is way more than a dynamically typed builder pattern on top of Swing components. Griffon gives you property binding to widgets (you're not the only one Flex), a standard and simple MVC architecture without a spaghetti monster diagram (that's you PureMVC), more components than just a TextBox (the complete JavaFX widget set last Winter), a plugin system that allows you to decompose problems into reusable addons, and an easy way to deploy your app via webstart. If the Griffon team can keep the energy they had in 2009 then 2010 should be the year of the lion. Or eagle... or dog. What the hell is a griffon anyway?

Groovy Tooling Explosion
Looks like IntelliJ IDEA has some competition for best Groovy IDE. The Groovy Eclipse plugin got new life as Andrew Eisenberg revived the project, and SpringSource released better Grails support in SpringSource Tool Suite. Was the open sourcing of IDEA a response to the new competition? Who cares, it's free now! While IDEA is still the best IDE for Groovy, Groovy users will surely benefit from each IDE maker trying to outdo the other.

VMWare Buys SpringSource
Seriously, who saw this coming? This week at Groovy/Grails Exchange, Graeme Rocher demoed deploying to the cloud from his IDE (according to Twitter). Easy cloud deployment is good news... it will end the monthly "who do you use for Java hosting?" questions on user groups. Now if only the price would come down.

GR8 Conference
A Groovy conference created by the community, for the community, and priced for the community. It's great to see not for profit additions to the Groovy conference scene, and next year sees two GR8 events: one in Europe and one in North America (Minneapolis!). In other community news, Chicago Groovy Users Group started posting video sessions to blip.tv. Can we get some other GUGs to do the same?

A Groovy 2010

Don't think of these as predictions... think of them more as premature facts.

Gradle
To be clear: Gradle is not a Groovy technology. It is an enterprise build system written in Java with Groovy used as a build script. Anyway, the 0.9 release will include "Smart Execution/Incremental Compile" and 1.0 will support multi-threaded builds. These are enterprise level features that would be totally unique to Gradle... and they're sure to intice a lot of unhappy Maven developers. If the Gradle team can hit 1.0 and publish a book(!), then a lot of people will migrate.

GPars
I can't find anyone with anything bad to say about either the Fork/Join Framework or BMW Motorcycles... and I just sold my bike to spend more time coding. GParallelizer hasn't been widely adopted to date, in my opinion because it chased the Actor-Model hype a little too strongly. But now it's been rebranded GPars and an all-star team has been assembled to work on it. This isn't a good project to just follow, it's a good project to download and play with. Get the bits and join the mailing list. Your opinion counts! It's too bad that they hate cool the logo I made for them.

Griffon
The decision by the Grails team to make almost everything a plugin was genius. It provides a standard mechanism for everyone to modularize their own applications, and provides an easy path for users to push their non-business-critical plugins back into the community. If Griffon users embrace the plugin system, and then push their plugins back to the community, then Griffon could be a real alternative to JavaFX/Flex by the end of the year.

Java Closures
I wish the JDK team the best of luck meeting their September 2010 release deadline. While I have doubts that JDK 7 will ship in 2010, I do believe the closure syntax will be decided upon. And Groovy will be the first Java language to support that syntax. Whether a version of Groovy containing Java closures actually ships before JDK 7 is probably dependent on how the Groovy team wants to address the modularization issues of Jigsaw, which sounds like a much harder problem to solve.

Groovy IDE Support Improves... a little
IntelliJ IDEA still leads in features by a wide margin, but Eclipse and STS aren't stealing IDEA users, they're stealing TextMate users. People just want to debug without parsing all the files in the world. IDEA will remain the sole provider of Groovy refactorings, intentions, and auto-completions. But Eclipse will finally become a better alternative than a text editor. However, the open sourceing of IDEA should make it easier (and faster) to get IDEA support for newer frameworks, which is a good thing.

Groovy-User Mailing List Shuts Down
For the entire month of June, all posts to groovy-user will be answered with the same response: "This is covered in Groovy in Action 2nd Edition". By the end of the month admins will simply replace the mailing list home page with an advertisement for the book.

What Won't Happen

InvokeDynamic
InvokeDynamic is set to greatly improve implementing and running dynamic languages on the JVM. Only it won't be released until September 2010 at the earliest. Sure you can get the OpenJDK now, but most users won't do that. I predict InvokeDynamic being the story of 2011, not 2010... and even then I bet the JRuby guys beat us to it.

The Java Store
Let me get this straight... you want me to pay US$50 yearly so that I can give my Griffon app away free on the Java Store? You gotta be kidding. Griffon + the Java Store could be a match made in heaven: Griffon makes JNLP Webstart simple to configure and the Java Store handles hosting the files. But Sun/Oracle is turning too many hobbiest and small time developers away with their entrance fee. Are you listening Mr Ellison? I said I want... oh wait, you're not listening. How hard would it be to make the Groovy Store?

This was fun. What are your Groovy 2009 highlights and 2010 predictions?

Simple XML Testing from Java

2009-12-03T17:32:00.001-06:00

2009 is coming to a close and I'm writing a blog post about Java and XML. The blog's called "behind the times" for a reason: no fancy dynamic languages, JSON, or REST. Straight up XML on Java action.

One of my favorite editions to our enterprise toolset this year was XMLUnit. We'd been using Groovy in testing, and multi-line strings led to a ton of expressiveness in test cases... but then we stopped using Groovy. I switched back to Java testing and was left with making String based assertions and doing String#contains() calls. Not ideal; enter XMLUnit.

XMLUnit supports XML based equality comparisons so that things like these two snippets are equal:

<root/>

<root></root>

As well as these:

<root>data</root>

<root>
    data
</root>

In fact, there's a whole gaggle of options around what is a meaningful versus a meaningless difference. For instance, I often consider two dates equal as long as they are formatted correctly (ignoring those pesky timestamp issue is nice).

The problem with XMLUnit is that the API is not as nice for the simple case as it could be. What I want is a simple custom assertion that asserts two XML snippets as similar. Something like this:

XmlAssertions.assertXmlSimilar("<root/>", "<root></root>"); 
XmlAssertions.assertXmlSimilar("<root>data</root>", "<root>\ndata\n</root>\n");

This was as simple as creating a small utility class and hiding the XMLUnit API behind a nicer custom assertion:

import java.util.List;
import junit.framework.*; 
import org.custommonkey.xmlunit.*;

public class XmlAssertions {

    private static final String ERROR_MSG = "XML comparison failure. \nExpected: %s\nReceived: %s\n%s"; 

    static {
         XMLUnit.setIgnoreWhitespace(true);
    }

    public static void assertXmlSimilar(String expected, String actual) {
        try {
            Diff diff = new Diff(expected, actual);
            List differences = new DetailedDiff(diff).getAllDifferences();
            Assert.assertTrue(
                String.format(ERROR_MSG, expected, actual, differences), 
                diff.similar());
        } catch (Exception ex) {
             Assert.fail(String.format(ERROR_MSG, expected, actual, ex.getMessage())); 
        }
    }
}

The raw XMLUnit code is clearly not something you want to see within the body of a test method. The custom assertion seems simple, but it's made a real pleasure of writing XML based functional tests around all our web services. Feel free to steal my code here: http://svn.assembla.com/svn/SampleCode/xmlunit/XmlAssertions.java

The True and False of Groovy's AST Builder

2009-10-25T13:08:00.002-05:00

Last week saw three(!) conference sessions on Groovy AST Transformations. At SpringOne2gx I started with an AST Transform deep dive, Venkat followed up with a more gentle, and better performed, overview of Transforms, and I finished out with a video taped AST talk at StrangeLoop (watch infoQ for the video).

At the sessions there was some confusion about the API of the new AstBuilder in Groovy 1.7 (currently in RC1). Hopefully this post can clear that up.

The AstBuilder helps you create abstract syntax trees, and the User Guide is the best place to start learning about it. It lets you create the AST for code by simply passing the code into a method:

List<ASTNode> x = new AstBuilder().buildFromCode { " a constant " }

The big question everyone has is, "What's in the List of ASTNodes?"

The question is mostly easily answered by looking at GroovyConsole's AST Browser. Just type in "a constant" into GroovyConsole and press Ctrl+T/Cmd+T to view the AST:

Do you see that there are two root nodes? A BlockStatement and a ClassNode? The reason for this is that all Groovy scripts are compiled into classes of type Script by the compiler. Remember, to the JVM there are no scripts, only classes. If you expand out the AST, you can see that the script (in this case, the ConstantExpression) is in two places in the AST: within the BlockStatement and within the Script#run method:

So this is why the AstBuilder returns a List of ASTNodes. You'll be returned both the script BlockStatement and the Script ClassNode. For the most part, you're probably only interested in the script and not the ClassNode. The boolean parameter on the buildFromCode and buildFromString methods exist so that you don't have to see that ClassNode. The default value of this 'statementsOnly' parameter is true, so be default you'll only be given the script BlockStatements.

List<ASTNode> x = new AstBuilder().buildFromCode { " a constant " }

assert x[0].class == BlockStatement
assert x[1] == null

But when you need it, you can specify 'false', meaning you want the ClassNode as well as the BlockNode:


List<ASTNode> x = new AstBuilder().buildFromCode(
       CompilePhase.CONVERSION,
       false,  // get the ClassNode
       { " a constant " })

assert x[0].class == BlockStatement
assert x[1].class == ClassNode

For a fun, educational exercise, create multiple classes in GroovyConsole and see what the AST looks like. Hopefully you won't be surprised. But please don't enter an anonymous class because then you'll break the AST Browser! (It's a defect in 1.7 RC1 that I promise to fix soon).

Enjoy!

Y-Combinator in Groovy

2009-09-04T15:15:00.004-05:00

File under uselessly cool (for now at least).

A Y-Combinator in Groovy!

def Y = { le -> ({ f -> f(f) })({ f -> le { x -> f(f)(x) } }) }

Update: @mrdillon was kind enough to point out that my original Y used recursion. His revision is better.

And to prove it works, you can define a recursive factorial function without using recursion in the language.

def factorial = Y { fac ->
    { n -> n <= 2 ? n : n * fac(n - 1) }
}

assert 2432902008176640000 == factorial(20G)

The one-liner closure in the middle is the meat of the method definition and is a classic (i.e. horribly inefficient, known deficient) factorial implementation. I wouldn't write code this way but it makes a nice terse example when written in one line.

So is this useless? Not at all! But getting to an explanation of its use requires just a few examples.

Consider a recursive function that determines if an element is a member of a list. I guess you should also consider that List#contains(Object) does not exist. Here is a first draft of such function:

def isMember(element, list) {
    if (!list) false
    else
        if (list.head() == element) true
        else isMember(element, list.tail())
}

assert isMember(2, [1, 2, 3])

Let's just step through it briefly.

if (!list) false
... a simple terminating condition, an empty list never contains the element

if (list.head() == element) true
...if the first item in the list is the element then yes, we found it!

else isMember(element, list.tail())
....otherwise, check all the other items in the list for the same condition

Over time, the isMember will be recursively applied to each element in the list, and call stacks will build up in memory until the element is found, the list is fully checked, or you run out of stack space.

The problem with isMember is the redundancy. Notice how 'element' is always passed as a parameter? The value never changes over the life of the function application, so we shouldn't bother passing it every time. We have closures, so why don't we just 'close over' the value of element and remove the redundant data passing?

We can do it, but because of Groovy's limited recursion support, the implementation is kinda ugly. What I'd like to do is create a recursive closure that captures element once, but Groovy forces me into declaring an inner looping function on one line and defining it on another:

def isMember(element, list) {
    def inner
    inner = { lat ->
        if (!lat) false
        else
            if (lat.head() == element) true
            else inner(lat.tail())
    }
    inner(list)
}

assert isMember(2, [1, 2, 3])

This is where Y comes into play. I want a recursive closure and that is what Y does! Check out how Y allows me to define a recursive function locally within a method. No more split on declaration/definition.

def isMember = { element, list -> {
    def inner = Y { inner ->
        { lat ->
            if (!lat) false
            else
                if (lat.head() == element) true
               else inner(lat.tail())
        }
    }
    inner(list)
}

assert isMember(2, [1, 2, 3])

It's not uselessly cool any more is it? Oh wait, without tail call optimization on the JVM you will surely run out of stack space with a nasty StackOverflowException by writing code like this. So don't do it!

I guess it's pretty useless after all.

And if you're looking for a fun weekend project, why don't you do a different one of the Rosetta Code challenges
that are missing for Groovy: http://rosettacode.org/wiki/Tasks_not_implemented_in_Groovy

Tschüs!

The System Metaphor is Still Lame

2009-08-31T07:17:00.003-05:00

The eXtreme Programming System Metaphor practice makes a pretty easy target. I've seen large development teams grind to a halt because of persistent broken builds without any continuous integration. I've seen teams' productivity drop to a crawl under the burden of technical debt and lack of test driven development. But what happens when a team lacks a good metaphor for how the system works? Can you really name a reasonable, realistic negative impact from a lack of system metaphor? Just last week I was dismissing it in an internal agility training as an unneeded XP practice. So I was surprised to see Joshua Kerievsky's and Brain Foote's "System Metaphor Revisited" talk at Agile 2009. Seriously, the guy who wrote Refactoring to Patterns is also hyping the System Metaphor? We'll see...

The thesis of the talk was that a System Metaphor provides illumination, inspiration, and integrity (...and alliteration). Briefly and in my own words, it illuminates an unfamiliar design by providing a reference point in the familiar. It inspires unexpected design ideas by providing a frame of reference in which to apply your creativity. And it builds integrity by suggesting a single, coherent, and consistent framework and set of names into the system.

And the Industrial Logic case study is just so cool. I've loved their marketing since they started with the "Let the Good Designs Roll" idea. I want to be a rock star and I want to take their training classes. And now their trainings are all hinged around the ideas of albums, tracks, and playlists. Very cool. I guess it helps to have a great product.

I have no doubt that the guys presenting have deemed their system metaphor a qualitative success for the project. But I still think the practice is totally lame.

A metaphor embedded in code is rigid. Having a great marketing metaphor is fine; marketing campaigns sell products. But they're also easy to change and refine, while an IT system is not. How many defects will be introduced if you change your marketing metaphor from a rock and roll concert to a book store? Not many. How many defects are introduced by changing the code to reflect this? Is none really the answer?

A metaphor is a 2nd ubiquitous language. One universal domain language is of clear benefit to a project. And that language should be the language of your users, domain experts, and programmers. A system metaphor is usually not this same language. For my projects, it's hard enough to try to get all the stakeholders to agree on terms of use. Introducing another layer of abstraction and the associated verbiage isn't going to help things. Ubiquitous language is important; but create it out of your domain not your metaphor.

A metaphor illuminates in the large, but confuses in the small. A metaphor guides you towards understanding when contemplating projects as a whole. It provides familiar signposts and a framework on which to hang ideas. But a lot of programming involves making small changes to systems you only partially understand. When this is the case, the metaphor becomes accidental complexity in the system. Having postage stamp, postage meter, and postage machine objects in your system helps the original design team, but doesn't help the contractor being asked to figure out why reports aren't printing.

A metaphor will likely become a mixed metaphor over time: consider a system that treats all data as streams. It's consistent and elegant. But eventually the Enterprise knocks on your door and you develop some sort of messaging service. Now your streams have mailboxes and can be put in message queues. This undermines the benefit of integrity via consistency. We live with this sort of ambiguity every day, but it doesn't mean we should accept it.

Metaphors do not decompose well. You system will grow, it will take on technical debt, and eventually it will need to be refactored, decomposed, and modularized. This is the natural rhythm of agile systems. When it comes time to break out components, will your modularization boundaries be natural boundaries within your metaphor? At a minimum, you'll probably end up discussing how the two don't line up correctly, and now you're spending time solving a self-created problem... surely a bad process smell. It seems like only the most dependency-obsessed team leads could foresee and prevent this. A YAGNI approach may not be your ally here.

In the end, I see a metaphor as a set of constraints. Humans can do their best and most creative thinking when provided rigid constraints. But constraints limit design ideas as much as suggest new ones. I'm sure some people have had success with a system metaphor, but I find success because of system metaphor a much harder pill to swallow. To all those espousing the use of system metaphor as a useful practice, I quote a famous and apocryphal doctor and say, "Clearly I can see you're nuts".

And just for fun, I used eight metaphors in this blogpost. There is no prize for finding them all.

Counter-intuitive Agile Coaching Tips

2009-08-24T19:05:00.002-05:00

Just finished the "What Do Agile Coaches Do?" session from Rachel Davies and Liz Sedley at Agile 2009. It was a great workshop based on their new book and J. Richard Hackman's "Leading Teams".

Hackman defines three types of coaching intervention:

Motivational - intended to improve team or team member's effort
Consultative - intended to improve a team's process
Educational - intended to improve a team's understanding

Today's first insight is that I personally focus on educational and consultative interventions, to the almost complete exclusion of motivational intervention. I'm not interested in giving people pep talks, so clearly I'm lacking some motivational skills and techniques. Anyway, Hackman defines three key times when these interventions should be used within a project:

Beginning - focus on motivational intervention
Midpoint - focus on consultative intervention
End - focus on educational intervention

Delaying education until the end of a project sounds ludicrous. Wouldn't the begining be better, when people need it? The answer is to think of a project as an iteration, not a full release. Iteration start is when the team needs energy to commit to work and kick off the sprint. Makes sense that the "sprint" start is when you want to get a good start out of the blocks, huh? So the beginning is a good time for motivation after all. The iteration midpoint is when the team is neck deep in implementation and sees that not all their practices are working in their best interest. What's the solution? A small process tweak via consultative intervention. The end is the retrospective is where the whole team has a forum and opportunity to make big changes to how they work. That's the time where education on a new and unknown practice can make the biggest impact, because the team can commit to it and take action immediately. Hackman's description of when the interventions are most effective sounded fishy at first, but when viewed against an iterative process it makes sense!

But non of this is counter-intuitive coaching. The most interesting part of the workshop was our group's breakout session where we discussed coaching in the context of some manufactured scenarios from the speakers. We came up with these unique coaching strategies:

1) Coaching Through Absurdity
In my group was Alex Chin, and in order to get his team to estimate at the task level he added huge estimates to each task. These grossly absurd estimates forced the team to come back and add better estimates. Awesome idea. Next time my team won't make a decision I'm going to make an absurd one for them and wait for a reaction.
2) Letting People Fail
We spent a lot of time discussing how to guide a team down the correct path. If your team insists on walking precariously close to a metaphorical cliff, then don't waste your time each week shepherding them off it. Let them fall of the cliff once. Surely that will teach them to avoid it next time. Right?
3) Doing Nothing is Doing Something
You can lead a horse to water but you can't make him drink. Likewise, you can't force a feedback loop. Coaching is not just about knowing when to intervene, but when not to. If there is a team issue that needs resolving, sometimes you just need to get the people together and wait. Do nothing and let the team decide. The most effective way to coach is to set the stage for team emergence and then get out of the way! Agile isn't about the coach solving your problems, it's about the coach getting you to solve your own.

Agile 2009 Day 0: Kanban Bag Stuffing

2009-08-24T09:28:00.004-05:00

Spent about 4 hours today packing 1500 conference bags for Agile 2009. Luckily, several guys from the Kanban-Dev mailing list showed up to turn a scene from Modern Times into an unique exercise in Lean production.

I won't explain much about the "Pull" system or queuing theory backing Lean production; rather, I'll just explain what we did.

We had about 1500 bags to pack with more than 25 items each: a lot of flyers, several books (including past volunteer Ahmed Sidky's Becoming Agile), and even some agile hand lotion (wtf, right? But it beats the face creme I got at a triathlon called, I kid you not, "every manjack"). Anyway, it all needed to be bagged, carted, and trolleyed to the reception area. Uff da.

The group split into two teams. Our goal was obviously to get the bags packed as quickly as possible. The Kanban experiment was used to create a system of flow, in which each group had a steady stream of product through their production line. Bottlenecks meant the upstream production unit should slow down and not having work to do meant the upstream unit had to speed up. But it wasn't about working more or working less to achieve flow. It was about tweaking the process so that everyone working hard would result in the flow. The group had to self organize and evolve so that this steady state existed. Over two shifts of people this did all happen to a certain extent. We didn't have measurements to see who was fastest, but each process did improve over time. Here are some observations, along with how they might apply to software teams:

Having no leader meant that every team member contributed new ideas to process improvement. Does the leader of your software team sometimes stifle innovation?
Having two teams meant that competition fueled more frequent improvements. What external forces are driving your software team to evolve?
Having no roles created a system where shouting "We're out of Agile Hand Lotion" meant that the person most capable of grabbing more lotion at that point in time immediately pitched in to do the job. Are the roles in your organization lightweight enough to allow the right person to contribute?
Having one set of workers out-of-sight (running trolleys through elevators and hotels) meant that it was more difficult for the group to self correct their entire process. Finished bags did pile up and we were at a loss as to how to make the trolley's move faster. What parts of your SDLC are out-of-sight and out-of-mind?
Not producing a Value Stream Map meant it was difficult to see the whole and fix the real problem, which was slow elevators. VSMs are much lower ceremony than most people realize, and we could have done one. Can your team truly see the whole?
Many improvements were small and non-obvious in retrospect - "Why don't we pack the books last because they are heaviest?" and "Open the bags more quickly by pulling the handle like this." Does your company have a process group that focuses on the macro view to the exclusion of the micro view?

At the end of the day, pro facilitator Jean Tabaka of Collaboration Explained fame hosted our retrospective for us. The format was ORID:

Objective – Focus on the facts, hard evidence data
Reflective – Focus on how that made people feel or other associations
Interpretive – Focus on the impact and significance
Decisive – Focus on next steps

It was a typical retrospective in that we gathered data and created a shared understanding of the event in the Objective phase, talked about our feelings and personal experiences in the Reflective phase, generated insights in the Interpretive phase, and voted with dots in the Decisive phase to create a list of action items for next year. She and the rest of the team did a great job, and both Kanban and the retrospective were superb ways to kick off the volunteer sessions!

And just because I get a kick out of having foreign characters in my posts: 看板 is the symbol for Kanban. Whee!

Exploring Immutability

2009-08-23T17:25:00.001-05:00

When did immutable objects become idiomatic Java? I suppose it depends on your frame of reference and what your experience brings to coding. It may not be idiomatic Java to you. In fact, Java Beans and fluent interfaces seem to rely on in-place side effects and mutability to function properly. Exactly what place do immutable objects have in the Java world?

The immutability of String surprised me when I was learning Java. One of the first programs I wrote assumed String#replace did an in-place edit of the String, one of the first debugger sessions I conducted clearly showed me that String#replace was not editing the String, and one of the first Sun Forum questions I posted was about String#replace being broken in Java. BROKEN, I say! This represents the one time when an answer to a Sun Forum question clearly solved the poster's problem without a lengthy chain of "please post your code" responses. The point is that immutability has been with Java forever. And now that the days of Object Pools and persistent performance concerns are behind us, it's time to embrace immutability in our own code.

The benefits of immutability are many and well-documented. Effective Java (either edition) is a good start. I'll touch the highlights with the example of a Person object:

class Person {
 def firstName
 def lastName
}

HA! Tricked you, it's Groovy. In the Groovy compiler, the Person class gains two methods for each property defined:

public void setFirstName(Object)
public Object getFirstName()
...

The problem with the Class (as the JVM sees it), is that many threads may access Person and see the same instance in different states. It is difficult to share this object correctly across threads. The problem with the Java source is the cruft of writing getters and setters. The problem with the API of Person is that working effectively with the object requires many calls to setX and setY mutators just to get simple interactions working. The problem with maintaining this code is that the object is somewhat non-deterministic and difficult to reason about. Groovy uniquely solves only one of these issues: the source level cruft. Sure, you don't have to write out getters and setters, but that is the least important problem to solve.

Fluent APIs solve the issue of having to munge up the source code with repetitive calls to setX and setY. Instead of following the Bean pattern of setters being void methods, instead have all your setters return a 'this' reference:

class Person {

 def firstName
 def lastName

 def setFirstName(firstName) {
   this.firstName = firstName
   this
 }

 def setLastName(lastName) {
   this.lastName = lastName
   this
 }
}

Now you have a nicer API to work with at the expense of not having a standard Java bean. Remember the Java language change proposal to make void methods always implicitly return a 'this' reference? I feel like I'm the only guy wanting that in Project Coin. Oh well. Here is what consuming the API now looks like:

def person = new Person()
   .setFirstName('hamlet')
   .setLastName('darcy')
assert 'hamlet darcy' ==  "$person.firstName $person.lastName"

Groovy has a solution to this same problem: the with block. All object have a with method, and when inside the closure parameter, all method and field references are resolved back to the object having with invoked. Visual Basic had this feature and I always missed it in Java:

def person = new Person()
person.with {
   firstName = 'hamlet'
   lastName = 'darcy'
}
assert 'hamlet darcy' ==  "$person.firstName $person.lastName"

So both Groovy and Java have an answer for the boilerplate involved with writing and consuming APIs. But again, these aren't the big issue with mutable objects: non-determinism and thread safety are. To help solve that you can use immutable objects.

Can we get nice, fluent interfaces in Java without sacrificing mutability? The go-to solution in Java is the Builder idiom. Make your Person object immutable (all final fields), and the constructor private. Then define a mutable Builder class that wraps the constructor in a better API. In the end, creating Person instances looks like this:

def person = new PersonBuilder()
       .setFirstName('hamlet')
       .setLastName('darcy')
       .build()
assert 'hamlet darcy' ==  "$person.firstName $person.lastName"

You instantiate the Builder and the builder instantiates the Person. This is nice... I guess maybe. Remember the idea of a "meaningless abstraction"? PersonBuilder is definitely a meaningless abstraction. It does not exist to solve any of the problems the system was designed to do. It exists solely to solve a problem in the implementation. Suspect. Very suspect.

The Builder has by far the longest implementation, and the DRY principle is glaringly violated in the duplicate fields:

class Person {
   private final def firstName
   private final def lastName

   Person(firstName, lastName) {
       this.firstName = firstName
       this.lastName = lastName
   }
}

class PersonBuilder {
   def firstName
   def lastName

   def build() {
       new Person(firstName, lastName)
   }

   def setFirstName(firstName) {
     this.firstName = firstName
     this
   }

   def setLastName(lastName) {
     this.lastName = lastName
     this
   }   
}

It should be said, though, that this idiom does address the determinism and concurrency issues with mutability. Person can now be shared freely across threads, cached safely, and behaves in a more deterministic manner.

The best approach has not yet been mentioned. Simply create an immutable object without a builder, and also provide a nice fluent API around it. The calling code can look exactly the same as the previous fluent API example:

def person = new Person()
       .setFirstName('hamlet')
       .setLastName('darcy')
assert 'hamlet darcy' ==  "$person.firstName $person.lastName"

You can do this by slightly tweaking the fluent API example. Don't return a 'this' reference from setters, just return an entirely new object. And don't worry about all those little objects you're creating and tossing away.

class Person {

   private final def firstName
   private final def lastName

   Person() { }

   Person(firstName, lastName) {
       this.firstName = firstName
       this.lastName = lastName
   }
  
   def setFirstName(firstName) {
       new Person(firstName, lastName)
   }

   def setLastName(lastName) {
       new Person(firstName, lastName)
   }
}

Is a Person instance threadsafe? Yes. It is immutable.
Is a Person instance deterministic? Yes. No race conditions here.
Is a Person instance API easy to consume? Yup, it has a fluent API.
Does the Person class introduce a meaningless abstraction? Nope, and good thing!
Does the Person class require an exotic programming language to implement? No, this works fine in Java.
Does the Person implementation require tons of boilerplate? Maybe. The good part is that every implementation of a setter is exactly the same. The bad part is that every implementation of a setter is exactly the same. Not very DRY, but it could be automated in Groovy at runtime with a little metaprogramming.

That's it, immutable exploration over. And what place should immutability have in the Java world? It should be front and center. I hope I've shown that immutability and good APIs do not need to conflict with each other. You can get both with a tiny bit of extra work.

How to Tell Your Co-Workers Have Discovered Functional Programming

2009-08-22T08:47:00.005-05:00

Three ways to tell that your co-workers are learning a Functional Programming language at home:

1. You've found the number of .class file in your project quadrupled in the last month.

Here is an output directory with 17 .class files, only 2 of which are not inner classes, most of which are anonymous.

See all those dollar signs indicating inner/anonymous classes? Uh oh, looks like someone discovered shitty lambdas. Your chaos meter should be increasing... will your project be maintainable of it's not idiomatic Java?

2. You've found a class named Pair

public class Pair<T, U> {

   private final T left;
   private final U right;

   public Pair(T left, U right) {
       this.left = left;
       this.right = right;
   }

   T getLeft() { return left; }
   U getRight() { return right; }

   static <T, U> Pair from(T left, U right) {
       return new Pair<T, U>(left, right);
   }
}

It's not a tuple, but it's about 2/3 of the way there! There's already jfun.util.Pair in JFun and gnu.lists.Pair in Gnu Lists, and now you've got your own too. Hooray. But wait, how will you maintain your code base if the declared name of your data structures don't tell you how it's meant to be used? Chaos... level... rising...

3. You've found a widely implemented one-method highly generic interface

An interface like this has over 100 implementations, many of which are anonymous:

public interface Transformer<T, U> {
   U transform(T input);
}

This is equivalent to F in Functional Java and Transformer5 in JConch.

At this point your chaos meter should be off the chart. These functional programmers are surely ruining your system and livelihood. So, I'm hesitant to mention what happens when you combine all of them together:

Transformer identityfunction =
   new Transformer<Pair<String, String>, Pair<String, String>>() {
       public Pair<String, String> transform(Pair<String, String> input>) {
           return input;
       }
   }
}

Voila, an anonymous function object that uses the Pair class to make a 1-arity method into a 2-arity method, returning multiple values.

These are the trials and tribulations of finding the appropriate use of new programming paradigms in old languages. Looking at version control history, the person that committed all these sins was... me. Will this be the Java code I write in two years time? Certainly not. But which of these example is the growing pains of learning FP and which of these will still be with me in the future?

Comments are open!

Testing OSGi Bundles with Pax Exam, Groovy and Gradle

2009-08-15T16:07:00.003-05:00

You can't be an expert on everything. There are a few topics in life I've given up on, forever to be ignorant:

Botany - Don't ask me the name of any plant or tree. I won't know.
Hardware Specs - Telling me the model number and speed on your new laptop just makes me confused. I stopped reading the Computer Shopper years ago.
Maven - I'm happy to use it IF SOMEONE ELSE CREATES THE POM. Life's too short for that much XML.

So what was I to do with Pax Exam? This framework allows you to write JUnit or TestNG tests and then executes them in the OSGi container of your choosing. It's a cool framework but much of the documentation assumes a willingness to scroll through lines and lines of Maven XML. Ugh. Life's too short to grapple with build systems.

So the nut is cracked. I've got Groovy unit tests running across Equinox, Felix, and Knopflerfish from both Gradle and IntelliJ IDEA. No Maven in sight.

Writing Pax Exam Tests in Groovy

Once it was all configured, writing Pax Exam tests were simple. Start with a plain old JUnit 4 test and do two things:

Annotate it to run with the Pax Exam JUnit runner
Configure it to provision the Groovy-all JAR

Here it is, complete with imports and ready for a little copy-paste action.

import org.junit.runner.RunWith
import org.ops4j.pax.exam.junit.JUnit4TestRunner
import org.junit.Test
import org.ops4j.pax.exam.junit.Configuration
import org.ops4j.pax.exam.Option
import static org.ops4j.pax.exam.CoreOptions.*

@RunWith (JUnit4TestRunner)
class GroovyIntegrationTest {

  @Configuration
  public Option[] configure() {
      [
          provision(
              mavenBundle().groupId('org.codehaus.groovy').artifactId('groovy-all').version('1.6.4')
          )
      ] as Option[]
  }

  @Test
  public void testFramework() {
      println 'Hello from Pax-Groovy!'
  }
}

A few things to notice... CoreOptions was statically imported, so the methods provision() and mavenBundle() are resolved. The configure() method returns an Array of Option objects, so the "as Option[]" cast is needed. And by default, the tests are going to run on Felix 1.8 (or thereabouts).

Before Pax Exam becomes useful you're going to want to provision your bundle as the system under test, get access to the BundleContext it was loaded with, and specify more containers to run on.

Provisioning your bundle and running on multiple containers is a matter of configuring the test differently:

@Configuration
public Option[] configure() {
  [equinox(),
  felix(),
  knopflerfish(),
  provision(
      bundle(new File('./../out/production/Filter4osgi.jar').toURI().toString()),
      mavenBundle().groupId('org.codehaus.groovy').artifactId('groovy-all').version('1.6.4')
  )] as Option[]
}

Here I'm specifying Equinox, Felix, and Knopflerfish. There's also options for allEquinoxVersions, allFelixVersions, and allKnopflerfishVersions, as well as allFrameworks and allFrameworksVersions to test on the world. That options takes a few minutes to execute! Provisioning my own bundle can be done by loading it as a file URL. This example isn't very robust but it proves the concept.

The BundleContext is available as an injectable bean from Pax Exam. If you want the BundleContext in your unit test, to perhaps test that services were registered correctly, then add this as a field:

@Inject
BundleContext bundleContext

And here is a simple test to make sure that my filter4osgi library was correctly installed:

@Test
public void test_BundleIsLoaded() {

  def found = bundleContext.bundles.find {
      'filter4osgi' == it.symbolicName
  }
  Assert.assertNotNull('filter4osgi bundle not loaded!', found)
}

Running Pax Exam Tests from Gradle

The Gradle build was slick. Tell Gradle to load your bundle as a file URL and the rest is just boilerplate dependency management:

usePlugin 'groovy'

repositories {
  mavenCentral()
  flatDir dirs: [
      './../out/production',   // filter4osgi jar built earlier
  ]
}

dependencies {
  groovy group: 'org.codehaus.groovy', name: 'groovy-all', version: '1.6.4'
  compile(
      'org.ops4j.pax.exam:pax-exam:1.0.0',
      'org.ops4j.pax.exam:pax-exam-container-default:1.0.0',
      'org.ops4j.pax.exam:pax-exam-junit:1.0.0',
      'junit:junit:4.5',
      'org.osgi:org.osgi.core:4.0.1',
      ':filter4osgi:',
  )
}

"gradle test" is the only target you'll ever need.

Running Pax Exam Tests from IntelliJ IDEA

Your unit tests will show up once for each container you're targeting. So my two tests run against 3 containers shows up as 6 tests in the test runner window, plus a sweet ASCII art logo:

One nice side effect of Maven is that IDEA can generate files off it, so you don't need to worry about the transative dependencies in the IDE, it's all configured for you. When using Gradle you do need to set up the IDE manually. I added the following JARs as dependencies and it all worked fine (with one exception):

commons-discovery-0.4.jar
commons-logging-1.1.jar
easymock-2.4.jar
junit-4.4.jar
log4j-1.2.12.jar
ops4j-base-lang-1.0.0.jar
ops4j-base-monitors-1.0.0.jar
ops4j-base-net-1.0.0.jar
org.osgi.core-4.0.1.jar
osgi-3.4.0.jar
pax-exam-1.1.0-SNAPSHOT.jar
pax-exam-container-default-1.1.0-SNAPSHOT.jar
pax-exam-container-rbc-1.1.0-SNAPSHOT.jar
pax-exam-container-rbc-client-1.1.0-SNAPSHOT.jar
pax-exam-junit-1.1.0-SNAPSHOT.jar
pax-exam-junit-extender-1.1.0-SNAPSHOT.jar
pax-exam-junit-extender-impl-1.1.0-SNAPSHOT.jar
pax-exam-runtime-1.1.0-SNAPSHOT.jar
pax-exam-spi-1.1.0-SNAPSHOT.jar
pax-exam-testng-1.1.0-SNAPSHOT.jar
pax-exam-tutorial-1.1.0-SNAPSHOT.jar
pax-runner-no-jcl-1.1.0.jar

You can surely discard a lot of these JARs. But I added the list of completeness. It's just the IDE setup. I did have one issue with running the tests in the IDEA. Between runs I needed to manually delete my $TEMP_DIR/paxexam_runner_[user] folder. It sounds like no one else is experiencing this issue and I'm using a snapshot I built myself from source. The mailing list has been pretty responsive but it's a mystery error for now.

That's the end of it, folks. I'm going to the back yard to sit in the kiddie pool with my daughter. It's hot as a mutha in my office.

Groovy + OSGi: Gradle Makes it Easy

2009-08-12T22:34:00.004-05:00

I'm exploring the tooling around Groovy and OSGi for the September Groovy.MN meeting, where I'm presenting Groovy+OSGi Jumpstart (and then on to 2GX!).

In the end, creating OSGi bundles from Groovy classes was pretty darn easy with Gradle. The User Guide documentation wasn't exactly clear, and the current 0.7 sample project didn't do much of anything... so here's the goods while you wait for the updates to released.

A good background on using Groovy and OSGi is the Beginner's Guide to OSGi on the Desktop. It explains bundles and Activators are, and what the Import-Package and Export-Package metadata specifies. In a nutshell: an OSGi bundle is a JAR file with extra data in the MANIFEST.MF. It's dependencies are specified in the Import-Package attribute and it's exported API is specified in the Export-Package attribute. On bundle startup, a method called Activator.start() is called, and on shutdown Activator.shutdown() is called. The bundle replaces the JAR as a unit of deployment, the dependency management metadata replaces JAR Hell, and the Activator replaces "public static void main".

This post will show you a few things:

how to implement an Activator in Groovy
how valid bundle metadata looks
how to create the bundle from Gradle
how to load and execute the bundle in Eclipse Equinox

A Groovy Activator
This couldn't be much simpler. Implement the org.osgi.framework.BundleActivator interface in Groovy. You now have knowledge of 4% of the OSGi API (it's only 27 classes, you know).

package org.gradle

import org.osgi.framework.BundleActivator
import org.osgi.framework.BundleContext

public class GradleActivator implements BundleActivator {

 public void start(BundleContext context) {
     println "Hello from a Groovy Gradle Activator"
 }

 public void stop(BundleContext context) {
 }
}

Notice, I put this in the org.gradle package (it's important later). Now think for a moment about all the public methods on this object... can you name all 16 methods Groovy adds?.

Valid Bundle Metadata
The bundle metadata in the JAR's MANIFEST.MF file needs to specify, at a minimum, a unique SymbolicName, all the ImportPackages that the class will need to run, and all the ExportPackages that other classes will need to run it.

What does this activator need for ImportPackage? It needs to import org.gradle (itself), and it needs to import org.osgi.framework (the interface). You also need to import groovy.lang and a few other Groovy packages so that Groovy method dispatch works correctly.

What does the activator need for ExportPackage? Certainly org.gradle, so that the container can invoke the Activator. What else? This hidden dependencies are in the silent API of Groovy objects. All Groovy objects have a get/setMetaClass that operates on type groovy.lang.MetaClass. So to use the Activator you need to export groovy.lang and a few other packages, even if you don't invoke those methods!

A Gradle Generated MANIFEST.MF
Luckily, the Gradle OSGi plugin handles all of the ImportPackage and ExportPackage dependencies for you. Here is the full MANIFEST that Gradle generates:

Manifest-Version: 1.0
Ant-Version: Apache Ant 1.7.0
Created-By: 1.6.0_04 (Sun Microsystems Inc.)
Export-Package: org.gradle;uses:="groovy.lang,org.codehaus.groovy.refl
 ection,org.codehaus.groovy.runtime,org.osgi.framework,org.codehaus.gr
 oovy.runtime.callsite";version="1.0"
Bundle-Version: 1.0
Tool: Bnd-0.0.255
Bnd-LastModified: 1250128067203
Bundle-Name: Example Gradle Activator
Bundle-ManifestVersion: 2
Bundle-Activator: org.gradle.GradleActivator
Import-Package: groovy.lang;version="1.6",org.codehaus.groovy.reflecti
 on;version="1.6",org.codehaus.groovy.runtime;version="1.6",org.codeha
 us.groovy.runtime.callsite;version="1.6",org.gradle;version="1.0",org
 .osgi.framework;version="1.4"
Bundle-SymbolicName: gradle_tooling.osgi

It's a mouthful! Gotta love the wrapping rules on manifest files. Did you know the wrapping is defined not on the number of characters in a line but the number of bytes? Ugh.

Anway, as you can see, the Export-Package and Import-Package are correct. Groovy was discovered and specified. You can also see in the Tool attribute that bnd is used under the covers.

In Gradle, you use the OSGi plugin to simplify the generation of the manifest. The following build file specifies the Equinox dependency (to resolve the OSGi classes) and the Groovy dependency (to compile the Groovy). The "configure(jar.osgi)" begins the OSGi configuration. By specifying the * for import and export packages, bnd is invoked to determine the dependencies. 22 lines of build script gives you a "gradle jar" target that creates a valid bundle:

group = 'gradle_tooling'
version = '1.0'

usePlugin 'groovy'
usePlugin 'osgi'

repositories {
  mavenRepo(urls: 'http://repository.jboss.org/maven2/')
}

dependencies {
  groovy group: 'org.codehaus.groovy', name: 'groovy-all', version: '1.6.0'
  compile( 'org.eclipse:osgi:3.4.3.R34x_v20081215-1030' )
}

configure(jar.osgi) {
  version = '1.0'
  name = 'Example Gradle Activator'
  instruction 'Bundle-Activator', 'org.gradle.GradleActivator'
  instruction 'Import-Package', '*'
  instruction 'Export-Package', '*'
}

Proof Positive
It's a hello world bundle, right? So let's run it!

Invoke the OSGi container in console mode:

java -jar %osgi_jar% -console

The Gradle script will stick the OSGi jar in the .gradle cache. Mine is at $home/.gradle/cache/org.eclipse/osgi/jars/osgi-3.4.3.R34x_v20081215-1030.jar

Now just install and start the bundle:

osgi> ss

Framework is launched.

id      State       Bundle
0       ACTIVE      org.eclipse.osgi_3.4.0.v20080605-1900

osgi> install file:/d:/libs/groovy-all-1.6-beta-2.jar
Bundle id is 27

osgi> install file:/d:/libs/osgi-1.0.jar
Bundle id is 28

osgi> ss

Framework is launched.

id      State       Bundle
0       ACTIVE      org.eclipse.osgi_3.4.0.v20080605-1900
27      INSTALLED   groovy-all_1.6.0.beta-2
28      INSTALLED   gradle_tooling.osgi_1.0.0

osgi> start 28
Hello from a Groovy Gradle Activator

osgi>

Wheee! It works; hooray for Gradle, Groovy and OSGi... a trifecta of un-suck.

30 Days Locked in a Team Room

2009-08-06T20:33:00.011-05:00

Spent the last 30 days in a conference room converted into a shared work area for me and four of my (now) closest co-workers. We had asked to be moved to a shared space but none of us actually expected it to happen. After a month of it, all I can say is:

I do not want work in a cubicle ever again for as long as I live. Ever. Period.

One dude whistles. One dude crunches mini carrots all day long. One person is negotiating a house sale on the phone. And I pop my gum incessantly (if I don't I get an overwhelming urge for tobacco products). Yet it is the best work environment I've had in years.

Today I had to write up a summary of the team's past 8 retrospectives. Almost every single action item from the retrospectives over 4 months had to do with improving communication in some form, talking to each other more frequently, or getting better feedback. Until we got the team room. After moving to a shared space, the action items had nothing to do with communication. Empirically speaking, the team room unequivocally solved our project's biggest problems.

A team room has absolutely not lead to more interruptions. Side conversations are very easy to ignore. And any question you have is usually answered within seconds. Compare that to sending someone an IM seeing if they are available and then wandering over to talk. Speaking of, it's been wonderful not having to run some stupid instant messenger program in the background all day.

Cubicles are inhumane. Solitary confinement is punishment under most systems. Closing ourselves off from human contact and than holding a meeting to discuss ways to improve collaboration is laughable if we didn't actually hold those meetings every few months.

The insanity ended for me 30 days ago. I never, ever want to go back to working in a cubicle.

IntelliJ IDEA 9 OSGi Support Explored

2009-08-04T20:44:00.000-05:00

IntelliJ IDEA 9 (Maia) purports to support OSGi bundles... so I dug in the weekend to find out just what this means.

This blog posts provides a quick start guide for two things: 1) getting the IDEA 9 EAP to publish your library as an OSGi bundle, and 2) getting a second bundle published and running to test your library.

General Impressions
The IDEA 9 OSGi support is a port/conversion of the Osmorc plugin which was available in version 8. In general, the functionality seems stable but sparsely documented. I could find neither a discussion group nor documentation for Osmorc. A lot of the configuration was a process of trial and error. But it does all seem to work, and after investing a few hours it has started to seem straight-forward in hindsight. You'll have to judge for yourself how it compares to Eclipse (that's next on my list!). Osmorc tries to do a lot of auto-magic generation for you, and when I stepped outside the norm (like including Groovy as a test-only dependency) I was confronted with hard to fix failures. However, this may not be a fair criticism. For any serious project, you will want to write a bnd script, not simply use an IDE, and IDEA lets you use a bnd script to drive the OSGi configuration. So at points where the config wizard started to get difficult, I really should have stepped back and created a bnd script.

Getting a Bundle Published
While Eclipse seems to be focused on creating a Plugin (ie Bundle) from scratch using a wizard, IDEA is more focused on including OSGi support on one of your existing libraries using an OSGi facet. There is no "Create OSGi Project" wizard. You simply need to add an OSGi facet to one of your projects. I find this approach more reasonable. But let's not get too far before describing the basics of publishing a bundle:

1. Installing IntelliJ IDEA 9 and OSGi

Download and install the Maia EAP
Download and install an OSGi SDK (not just the framework jar). I had trouble with Eclipse Equinox, but KnopplerFish worked just fine

2. Configuring IntelliJ IDEA for OSGi
The next step is to tell IDEA about your OSGi SDK. In Settings (Ctrl+Alt+S), go to OSGi->IDE Settings->Framework Definitions and add your framework:
Now tell IDEA to use the framework you just registered. In Settings (Ctrl+Alt+S), go to OSGi->Project Settings and select your framework in the dropdown:

Done!

3. Publishing Your Library
Now let's publish your Java library as an OSGi bundle. In this example, I'm using my filter4osgi library that provides a nice API for LDAP style filters (shameless plug: I'll pay you for bug reports). This step will create the OSGi .jar file and required manifests.

Open an existing IDEA project... the fewer dependencies the easier it will be to publish
Add the OSGi facet to the project module. Clicking the little plus sign and picking OSGi should do the trick.

A note about the OSGi facet... currently, you're not able to add more than one OSGi facet to a module. For me, this forced me to create a multi-module project in order to produce several OSGi bundles from the same codebase. This seems like a considerable limitation to me but maybe my projects are structured strangely.

The Manifest Generation tab allows you to specify the exported packages for the bundle. I needed to manually add the package I wanted to export, Osmorc did not seem to discover this for me. Again, using bnd is probably the best option here.

There are plenty of other options, but they aren't as important right now. And note, I didn't add an Activator, because my bundle is just a library.

To verify that you published your library correctly, rebuild the application. The jar file should be written to disk. The manifest file with the OSGi headers is pretty simple:

Manifest-Version: 1.0
Export-Package: org.filter4osgi.builder;version="0.0.1"
Bundle-Version: 0.0.1
Tool: Bnd-0.0.337
Bnd-LastModified: 1249412068156
Bundle-Name: filter4osgi
Bundle-ManifestVersion: 2
Created-By: 1.6.0_12 (Sun Microsystems Inc.)
Import-Package: org.filter4osgi.builder;version="0.0"
Bundle-SymbolicName: filter4osgi
Include-Resource: ..\Filter4osgi

4. Running Your Bundle
Running the bundle is simple. Create a new Run/Debug Configuration using the little plus sign:
Just use the Add button to specify your bundle. You can also specify dependent bundles too.
On the Parameters tab, you'll need to specify the OSGi framework to run within:

Now just click Run, and your console output window should have something like this written to it:

Knopflerfish OSGi framework, version 4.1.7
Copyright 2003-2009 Knopflerfish. All Rights Reserved.

See http://www.knopflerfish.org for more information.
Failed to remove existing fwdir C:\Documents and Settings\hdarcy\.IntelliJIdea90\system\osmorc\runtmp1249413066328
Framework launched
Installed: file:///D:/dev/filter4osgi/out/production/Filter4osgi.jar (id#1)
Started: file:///D:/dev/filter4osgi/out/production/Filter4osgi.jar (id#1)

It worked! (Although I have no idea why there is a failure removing "fwdir", whatever that means). Alternately, you can load the jar file into any other container and interact with it that way. One feature that is missing is the ability to interact with the container from IDEA. You cannot issue the stop or uninstall commands. In fact, the only way I found to stop the container is with the big red Stop button. It would be better if you could issue text commands to the container. This doesn't seem like it would be difficult to add.

5. Testing Your Bundle with an OSGi Test Harness
There are a few test utilities for OSGi, such as Pax Exam and the Knopflerfish Junit Support, but all I wanted to do was write a bundle that imported filter4osgi and made sure the classes were available. So I simply added a new module to my project called "filter4osgiIntegrationTest". To this I added the Knopflerfish framework.jar since I needed to create an Activator. Take note, the prior example of exposing a library as a bundle required zero design or runtime dependencies on any OSGi implementation.

Without adding the OSGi facet, I created a new Java class that implemented BundleActivator. IDEA flagged me with a missing facet error and asked if I'd like to create a facet, which I did. Nice!

The Manifest Generation tab was mostly filled in, but again I needed to hand edit the MANIFEST.MF data so that it imported the package from the original filter4osgi bundle and added the root '.' to the bundle classpath (otherwise there were ClassNotFound exceptions).

Now I just implemented the activator to construct a new filter object:

public class ActivatorForTesting implements BundleActivator {
 @Override
 public void start(BundleContext context) throws Exception {
   FilterBuilder filter = or(
     and(
       eq("mailbox", "default"),
       eq("lang", "EN_US")
     ),
     and(
       eq("mailbox", "dflt"),
       eq("lang", "EN_CA")
     )
   );
   assert filter.toString().equals("(|(&(mailbox=default) (lang=EN_US)) (&(mailbox=dflt) (lang=EN_CA)))");
 }

 @Override
 public void stop(BundleContext context) throws Exception { }
}

Hey, that's not a bad API for working with LDAP filters, is it?

And now it's time to create the Run/Debug Configuration, which was probably the hardest step in all this. In the Run Config, I needed to specify that both the filter4osgi and filter4osgiIntegrationTest need to be started up, and that the filter4osgi must be started first. You do this in the Bundles tab, and ordering is done by indicating a Start Level:

Now, when I run the config, I should see that both bundles were started. And if I don't get a ClassNotFound error then I know that the filter4osgi exports lined up correctly with the filter4osgiIntegrationTest imports. Which they do:

Knopflerfish OSGi framework, version 4.1.7
Copyright 2003-2009 Knopflerfish. All Rights Reserved.

See http://www.knopflerfish.org for more information.
Failed to remove existing fwdir C:\Documents and Settings\hdarcy\.IntelliJIdea90\system\osmorc\runtmp1249414635312
Framework launched
Installed: file:///D:/dev/filter4osgi/out/production/Filter4osgi.jar (id#1)
Started: file:///D:/dev/filter4osgi/out/production/Filter4osgi.jar (id#1)
Installed: file:///D:/dev/filter4osgi/out/production/filter4osgiIntegrationTest.jar (id#2)
Started: file:///D:/dev/filter4osgi/out/production/filter4osgiIntegrationTest.jar (id#2)

Perfect!

All in all, the support was quite usable. I think starting with a bnd file would have made things easier for me, as well as if IDEA had let me interact directly with the container.Definitely looking forward to the official release!

10 Second Code Review

2009-07-29T22:31:00.001-05:00

Weighing in on code reviews are all the rage.

Along the lines of getting a group to start having code reviews... when I hear that code reviews are too time consuming my reply is, "Hogwash! I can do a code review in 10 seconds without even scrolling down the source file."

Just look at the import list in this worst case example:

import com.mycompany.services.UserService;
import com.mycompany.services.UserServiceImpl;
import javax.persistence.EntityManager;
import javax.swing.JFrame;
import com.thirdcompany.COMBridge;
import javax.swing.JLabel;

Dependency Inversion Principle - The presence of a service interface and a service implementation (UserService and UserServiceImpl) means the developer is probably instantiating the concrete dependency somewhere. Reject the code on grounds of not using the IoC container.

Separation of Concerns - The presence of EntityManager (persistence) and a JFrame (user interface) means that code working with the database is intermingled with user interface display. Reject the code on grounds of poor separation of concerns.

Programming to Interfaces - The COMBridge import from a 3rd party package indicates that risk-laden dependencies have not been abstracted out of the core business logic. Reject the code on grounds of no programming to interfaces.

Broken Windows - Imagine that the JLabel import is gray in the IDE because it is unused. Code should compile with no warnings, and all warnings should be cleaned before the code review begins. Reject the code on grounds of sloppiness.

OK, so this is a little facetious. I don't really do code reviews in 10 seconds. The point is that there's a wealth of information available with little effort on the reader's part. Code reviews are not a big commitment. Hopefully, you'll have more than 10 seconds budgeted for this. Maybe you have as many as 30 seconds! In that case all you'll need to do is look at the type signatures within the class.Easy.

IntelliJ IDEA 8 - The Good Parts

2009-07-29T21:18:00.000-05:00

I keep getting asked if the upgrade from IDEA 7 to IDEA 8 is worth the cost.

While all the new features are listed on the IDEA site, these are my favorite 5 features of IDEA 8, listed in the order I like 'em best.

Database Support
The new SQL Console means you now get code completion, syntax highlighting, and error highlighting in your SQL scripts. Plus, no more launching a 2nd application to run queries. Very useful, but does seem to work better in MySQL than MS-SQL.

7 new Java Refactorings
The most useful of the new rafactorings are Extract Method Object, which allows you to extract method on blocks with more than one result type, and "Introduce Parameter Object" to compact long parameter lists. Use with care, you may not end up with a meaningless abstraction in your object model. Also, Introduce Constant got a big upgrade that lets you extract just parts of big static strings. Nice!

UML Diagrams
You can create diagrams right in the IDE now. I think there were some plugins in the past that allowed this, but now a nice version of the feature is available in IDEA. You can generate diagrams, and can also create new entities, refactor, and navigate directly to source code. The diagrams look great; I've used them in some presentation already.

Groovy & Grails Support
Improved Groovy support adds a couple notable improvements: better code completion, more inspections, two new refactorings including "Convert method parameter to map-parameter entry", debugger support for Gant(!), and a whole bunch of Grails stuff.

Debugger for Flex (and Javascript)
We're using FlexBuilder for official Flex development at work, but for me, who does mostly Java, I've found it useful to use the IDEA Flex debugger when I don't feel like launching a 2nd huge application on my workstation. We evaluated IDEA 8 as a replacement for FlexBuilder and decided not to, but we'll take another look at IDEA 9 Flex support and maybe switch then. There's also a Javascript debugger but I haven't touched that language in a year or so.

Honorable Mentions

Dataflow to This - Shows you how a certain variable got the value it did. Think of it as a better call stack window.

Thread Dump Analyzer - The new window is way easier on the eyes than scanning through text thread dumps. Nice.

Spring 2.5 support - Use it every day so that some of the new Spring annotations are recognizable and navigable.

IDE Settings Synchronization - Share all your IDE settings between several workstations.

Those are the things I use the most... there is, of course, way more new than that. If you know me and want a 60 day trial license (normally they are only 30) then drop me an email!

Or check out the IDEA 9 EAP.

GUM, OTUG, 2GX, and Strange Loop, oh my!

2009-07-29T20:26:00.003-05:00

I'm going to be out on the road September and October, talking about programming and testing and maybe trying to make people laugh a little. Here's my schedule so far:

September 8th - Minneapolis, MN - Groovy Users of MN - Groovy+OSGi Jumpstart
September 15th - St. Paul, MN - Object Technology User Group - Thinking in Functions
October 19th - 22 - New Orleans, LA - 2GX Groovy/Grails Conference - All 4 talks for 1 low price!
October 23rd - St. Louis, MO - Strange Loop - Groovy Compiler Metaprogramming for Fun and Profit

If you haven't registered for 2GX, get on the stick... early bird pricing ends soon. And if you haven't heard of Strange Loop, then be sure to check out this super affordable unique and esoteric event.

I'm definitely interested in practicing these talks as much as possible, so if you need a speaker for a JUG or work group just let me know. The legacy code talk has turned out to be a really fun event, and it can have as little or as much Groovy as you want. Southwest has super cheap airfare right now... all I need is an invite.

Here are the summaries for the talks:

Groovy Compiler Metaprogramming and AST Transformations (aka Groovy Compiler Metaprogramming for Fun and Profit)
'A language should have access to its own abstract syntax' John McCarthy, Father of Lisp. Well, now Groovy 1.6 does! This talk is about why AST transformations are important, what you can do with them, and where the language world is headed. We'll dive into some of the useful Groovy annotations and libraries being written that harness AST transformations, see how to write our own, and work with the AST tools coming out with the next version. At the end we'll prognosticate about the future of programming languages in general, and hypothesize about where the Groovy features fit into the history of languages. Fun!

Functional Groovy (aka Thinking in Functions)
For many, learning Groovy made you think differently about Java. Now it's time to think differently about Groovy. Although Groovy is not a functional language by many measures, it does support many of the common functional idioms and patterns. Come explore both how far functional programming can be pushed in Groovy, where functional programming can't currently go, and where functional programming is headed in future releases of both the language and the JVM. Learn about morphisms, option types, tail call optimization, pattern matching, and functional composition in the context of solving classic CS problems side-by-side with a more traditional functional language, and decide for yourself how terms like elegance and simplicity should drive your coding.

OSGi and Groovy Jump Start
OSGi, Jigsaw, modularity, service lifecycles, bundles... where do you start? This talk covers the basics of using OSGi and Groovy together. You'll be introduced to the OSGi framework by building a Groovy based application that introduces the core concepts of OSGi such as the module system, service registry, and service life cycles. We'll also cover common pitfalls encountered by mixing Groovy and OSGi, as well the latest OSGi tools available.

Legacy Code, Groovy, and You
Thinking about writing Groovy unit tests for your legacy Java code? This session is an honest discussion about what Groovy will gain youand what it won't. Come learn the engineering practices and tools that you can use to battle tight coupling, monolithic projects, and tangled dependencies, and then decide for yourself whether Groovy is the answer for your project. Plan on returning to work with a vision of what your team can do to write better software.

Is ignoring private access in Groovy evil?

2009-07-23T23:05:00.004-05:00

The world's worst kept secret has got to be that Groovy ignores the private access modifier in code. You're free to access and invoke private methods, fields, and classes. A well known speaker called this "feature" insanely useful for unit testing.

I'm adding that it's also insanely evil. But it's not both insanely useful and insanely evil... it's either, depending on how you use it. I've spend all night in Gimp making a freakin' sweet graph showing the continuum, but before the big reveal we need to review some vocabulary.

The most useful addition from xUnit Patterns to the testing community is the classification of input and output as either direct or indirect. The easiest methods to test accept direction input (a method parameter) and return direct output (a return value). Think about testing Math#min(int, int). Damnably easy. Indirect input is when your system under test can't be fed input that it uses directly but can be fed an object that can locate that input. Thank about testing a class that uses EntityManager.find(Class, Object) to look up a row on the database. You can provide a mock entity manager that indirectly gives the system input. It's not too difficult but a lot harder than testing Math#min(int, int). Indirect output is when you cannot make assertions about the return value of the method to verify that it worked correctly. Instead you have to look at what side effects the method performed. Any time you are verifying mock method calls, or looking up that correct values were set back on the input values, then you are dealing with indirect output. This is the hardest to test.

So here is testing continuum for using Groovy to ignore private access modifiers:

This image took me like 4 hours to make so don't you dare laugh. Here's a useful key:

Not Using - Clearly, it is best not to use this feature. It's how Jesus would code. Yes, I'm serious. Jesus.

Indirect input in 3rd party library - So you're using a library that is hyper encapsulated and difficult to mock (like the JDK?). The library is unlikely to change frequently and you control when it changes. Sure, setting private values to provide your test with indirect input is useful. It'd be nice if there were more testable options, but at some point you have to wash your hands, stop trying to do the right thing, and just make a decision. A little like Pontius Pilate.

Indirect input in you own objects - Ignoring private in your own libraries can be insanely useful for getting some tests written quickly and out the door. But you're just creating tight coupling to the implementation of other objects... and you control those objects! Clearly there is a better option, and you have the control to perform that option. On an evil scale, it's a little like Saint Paul. Huh? A saint? That's not evil? Let me remind you: Paul killed a dude. With a rock. KILLED HIM. Yeah, go look up how Saint Stephen died. So, like Paul, you are clearly violating your principles in this scenario, but the situation is redeemable. Use private access cautiously, but then come back later and do the right thing. Bad decisions now won't stop you from being canonized later.

Indirect output in 3rd party library - The iteration is late, your test coverage is low, and you're stuck integrating with a shoddy 3rd party API where every method returns void. Slam some tests together and violate private access to write a fragile test coupled to implementation that looks freakishly similar to the production code. Did I just reimplement the system under test in mocks? Yup. What am I really testing? Nothing. Like Judas Iscariot, you've sacrificed your design principles for thirty pieces of silver. In 3 months, when maintenance is a nightmare, deny 3 times that you had anything to do with the code. If that fails, blame it all all on a contractor and move on.

Indirect output in your own objects - Dude c'mon. The only way to test your system that you wrote is to access private fields of objects you own? I don't buy it. This is where the difference between TDD and Test Always shows itself. I challenge anyone to evolve a system using test first that requires this sort of shenanigan. I just don't see how focusing on behaviors and testability would ever leave you like this. Whereas, trying to write tests for code after the fact might. Cain killed his brother and then denied it. That's pretty evil, just like accessing private fields on objects you own. You're given a great tool like Groovy for testing and this is how you choose to wield it? Like Abel, Groovy deserves better. When we catch you doing this, don't try to deny it. We'll know you're lying and we won't forget.

Indirect output in system under test - Old school Satan bad. This is just unjustifiable and with no benefits. What can your test possibly assert? That the code you wrote performs the way you wrote it? On a continuum of useless tests, that's like Pet Rock useless. And before you say, "I'm just laying down some unit tests so that I can refactor later," let me add that almost any refactoring you perform is bound to break the tests. Have fun with that. This usage just seems evil to me.

I don't often ask myself "What would Jesus do?" I try to live my live more like Pilate. I try my best but, hey man, sometimes stuff happens. Don't get hung up on it. Although, I'll admit to doing some pretty nasty stuff in the past, so a poll of my co-workers might reveal I'm more in the Paul range than I'd like to admit. How about you?

Groovy+Testing: Little Things, Big Impact

2009-07-23T19:59:00.003-05:00

When it comes to unit testing, it's the little things that drive me nuts. Or maybe it's the little things that make testing enjoyable. Optimism is better, I suppose. Now I don't buy into all that tipping point garbage, but there are two Groovy features that I miss immediately when returning to Java. Almost every other rising language has them, they seem like trivialities to add to a language, and they aren't likely to be in Project Coin. (Beware, all 3 of the previous statements are suppositions). And these two features are... multi-line strings and default parameter values.

As ridiculous as it sounds, I'm going to spend several paragraphs extolling the virtues of two language features that are so simple they can be explained in a single sentence each.

Multiline Strings

Why am I constantly testing with XML snippets? This is a question I don't often ask myself in Groovy because XML snippets look really nice embedded in a test method:

def garage = new Garage(
 new Car(CAR_NAME1, CAR_MAKE1),
 new Car(CAR_NAME2, CAR_MAKE2),
 new Car(CAR_NAME3, CAR_MAKE3)
)
  
def actual = serialize(garage, Garage)
  
def expected = """
 <garage>
   <car name="$CAR_NAME1" make="$CAR_MAKE1"  />
   <car name="$CAR_NAME2" make="$CAR_MAKE2"  />
   <car name="$CAR_NAME3" make="$CAR_MAKE3"  />
 </garage>"""
  
assertXmlEqual(expected, actual)

This test method clearly shows a garage full of cars being created and a garage full of cars being expected. There is symmetry and conciseness, and I find it pretty easy to see how the input relates to the expected output. Compare the Java version:

Garage garage = new Garage(
 new Car(CAR_NAME1, CAR_MAKE1),
 new Car(CAR_NAME2, CAR_MAKE2),
 new Car(CAR_NAME3, CAR_MAKE3)
);
  
String actual = serialize(garage, Garage.class);
  
String expected =
 "<garage>" +
 "  <car name=\"" + CAR_NAME1 + "\" make=\"" + CAR_MAKE1 + "\" /> " +
 "  <car name=\"" + CAR_NAME2 + "\" make=\"" + CAR_MAKE2 + "\" /> " +     
 "  <car name=\"" + CAR_NAME3 + "\" make=\"" + CAR_MAKE3 + "\" /> " +
 "</garage>";
  
assertXmlEqual(expected, actual);

Whee! Escaping quote characters is fun! I can't see how the expected block lines up with the input anymore, it's lost in a sea of accidental complexity, back slashes, and quotes. Yuck. And this is some pretty simple XML. The tests as documentation idea starts to suffer without multiline strings because the only thing that finds this format easy to read is the compiler. Speaking of documentation, how about copy and pasting the XML snippet into a user document or email? Or heaven forbid generating user documentation off the test case! You're just not going to do that with the Java version. In fact, I'd say most the time I don't even write the Java version like this... the escapes and quotes bring so much clutter that I just store XML snippets in separate XML files and read them in during each test:

Garage garage = new Garage(
 new Car(CAR_NAME1, CAR_MAKE1),
 new Car(CAR_NAME2, CAR_MAKE2),
 new Car(CAR_NAME3, CAR_MAKE3)
);
  
String actual = serialize(garage, Garage.class);
  
String expected = readFromFile("MyTest.sample.xml");
  
assertXmlEqual(expected, actual);

It's no longer abrasive on the eye, but you also completely lose the ability to see the relationship between the input and the output. You aren't helping the tests document the system by hiding all the input from the reader!

Such a small thing and I find it so valuable, and it's nothing more than a notational convenience, a little syntactical sugar.

Default Parameter Values
Tiny feature; big effect on unit testing.

Unit testing is about building tons of data, all in different configurations or states of construction, and then throwing that data at the system under test, observing and verifying the results. Unit tests look a lot different from production code. In production code, subroutines exist to perform smaller pieces of a larger, decomposed problem. In test code, subroutines exist to build objects. The design principles between production and test are different, and it makes sense that some language features might be very useful in test but of more limited value in production. I find default parameter values fit within this category. Consider a set of test methods that need to build a login form (username, password fields) in different states:

JFrame frame1 = makeLoginWindow()
JFrame frame2 = makeLoginWindow("some username")
JFrame frame3 = makeLoginWindow("some username", "some password")

Making a function to build a Login Window in these various states can be done within a single function in Groovy, you just have to use default parameters.

private JFrame makeLoginWindow(String username = null, String password = null) {

 JTextField userField = new JTextField()
 JTextField passwordField = new JTextField()
 if (username != null) userField.text = username
 if (password != null) passwordField.text = password

 JFrame frame = new JFrame()
 frame.contentPane.add(userField)
 frame.contentPane.add(passwordField)
 return frame
}

Under the covers, this is still an overloaded method to the JVM. But that's alright. There is a single source for all the test data configuration, and navigating (ie reading) the test case is made easier by having a single source.

In a Java test case, you're stuck defining at least 3 methods. The real rock star programmers are going to chain the method calls together... ohhh:

private JFrame makeLoginWindow() {
 makeLoginWindow(null, null);
}
private JFrame makeLoginWindow(String username) {
 makeLoginWindow(username, null);
}   
private JFrame makeLoginWindow(String username, String password) {

 JTextField userField = new JTextField();
 JTextField passwordField = new JTextField();
 if (username != null) userField.setText(username);
 if (password != null) passwordField.setText(password); 

 JFrame frame = new JFrame();
 frame.contentPane.add(userField);
 frame.contentPane.add(passwordField);
 return frame;
}

Conceptually, we've just added 3 entities to the test case that didn't exist before. Navigating around labyrinths of production code is a necessary evil. There are other design constraints at play and stuffing everything into one place isn't always the right thing to do. But in the test tree, labyrinths of chained code are just evil, nothing necessary about it. Having to chase down dependencies undermines one the chief principles of testing: show clearly and simply how input is transformed to output. Default Parameters for the win.

Sadly, when advocating using Groovy as a testing language, these two features are unconvincing evidence of Groovy's superiority. They just don't seem like much. They aren't very exciting. So why are they the first things I miss when creating new Java test cases?

By the way: PHP has both these features. Ha!

When Testability and Readability Collide

2009-07-22T21:39:00.004-05:00

Walter Harley was kind enough to leave a comment on my last post about testing busy constructors:

"...some of the attributes that support good testability also make for poor readability. For example, loosely coupled objects make it easy to supply mocks for testing; but they also make it hard to see what's going on in the production code."

I like any man who knows how to properly use a semi-colon. Grammar aside, this is a fair statement for a lot of projects. Why do so many programmers learn to loath dependency injection and IoC container configuration? Is it perhaps, that our default object model looks something like this:

In words, for each simple object (a user service in this example), you'll usually have an interface, a concrete class, a unit test, and a mock implementation. So we've created 4 entities, 3 of which are nonessential to solving the problem at hand. Oh, you're using a mocking framework so have one less than the diagram? A synthesized subclass is still kinda a subclass, isn't it? Whatever.

In my experience, this quad or triad of objects repeats itself fractal like across a Java project. It's uncommon to see an interface with multiple useful implementations and it's downright rare to see suites of components that operate on those abstractions. Names seem to spread out horizontally in my projects. Runnable, Callable, ActionListener, Transformer5, F, Predicate... aren't these all the same thing? Why are there so many names in the world?

In Java we write in names and think in names. There is no such thing as a function that accepts something and returns something. You always have to give it a name (Callable, Runnable, Transformer). And you write your APIs to those names. In Groovy you have closures to solve this problem, and you end up seeing a lot of code that works on that one unit of abstraction. But don't be fooled, it's not a problem with types. In F# and other functional languages it is entirely natural to reason about features by type instead of name. It's easy to write general algorithms that accept something with a certain signature as a parameter rather than a specific named. So my first assertion about readability and testability is that readability suffers in Java because of our culture of naming things, and we write our abstractions based on name instead of type.

Another factor to consider is the difference between a meaningful abstraction and a meaningless abstraction. Writing OO software is, on some level, about building models, whether it be of the problem domain, or the solution space, or the real world. An abstraction allows you to hide details that aren't needed. It allows you to write a general purpose algorithm or component that operate at a higher level than it's dependency. What possible complexity is the above UserService interface hiding from the programmer? None: it is an absolutely meaningless abstraction. And a system riddled with meaningless abstractions is just arbitrarily and needlessly complex. The problem might be in our perceived best practices...

Test Driver Development pushes you to use dependency injection as a way to vary the behavior of dependencies at test time. Fine. Using mock object frameworks guides you towards declaring interfaces for objects. Now the namespace starts to get cluttered, but modern IDEs easily support navigating hundreds, even thousands of classes, so it is no real problem to have a cluttered namespace. However, long-term TDD and mocking are creating a readability beast that results in a frustrating game of "find the dependencies" when changes need to be made. And the IDE masks the problem well into the days in which your codebase undeniably becomes "legacy code". Now the code is unreadable and retrofitting the legacy code with any modicum of meaningful abstraction makes those sections of the code look alien and feel out of place. We lost the battle and testability as a technique to plug the sinking ship rather than to guide an elegant design. So my second assertion about readability and testability is that readability suffers in Java because we have become supremely efficient and versed at producing meaningless abstractions.

Long term readability depends on finding type based abstractions, rather than name based ones, and then being diligent and intentional about coding to those abstractions. I don't know of any way to do this other than to think, and think hard, about the code we write every day. Sadly, some days I just don't feel up to the task.

Busy Constructors (are no problem to test)

2009-07-20T21:51:00.002-05:00

For whatever reason, I was browsing Miško Hevery's old testability blog entries and found a nice top 10 list of developer practices that make code hard to test.

Too bad I totally disagree with Item #3: Doing work in constructor. There's nothing untestable about big long constructors. No, really. Consider this horrorshow constructor, in which a global DataSource is retrieved from a mingleton and a concrete class is instantiated:

import javax.sql.DataSource;

public class UserService {

   private final DataSource dataSource;
   private final UserSettings settings;

   public UserService() {
       dataSource = DatasourceFactory.getDefault();
       settings = new UserSettingsImpl();
   }
}

Don't get me wrong, this is a horrible design. For one, retrieving the data source from a global static method is an example of a mingleton, which is global state by any other name: near impossible to inject any sort of variable behavior or polymorphism into. And instantiating a concrete implementation class (new UserSettingsImpl()) tightly couples this service to that class, another undesirable trait. This service is incredibly tightly coupled to the dependencies it requires. You'll never be able to reuse this service in another context where the datasource and user settings are different. And writing a test would be reusing this service... so what possibly makes this easy to test?

Package local constructors mixed with an Extract Parameter refactoring is what makes this easy to test. Simply create a new constructor for the object and leave out a visibility modifier. This makes the method only visible to objects in the same package (like your unit test)! Then add all the fields created in the original constructor to the parameter list. Final fields are your friend here because then the compiler will check to make sure you did this correctly. If you want to get fancy, chain the original constructor call into the new one:

public class UserService {

   private final DataSource dataSource;
   private final UserSettings settings;

   public UserService() {
       this(DatasourceFactory.getDefault(), new UserSettingsImpl());
   }

   UserService(DataSource dataSource, UserSettings settings) {
       this.dataSource = dataSource;
       this.settings = settings;
   }
}

This transformation preserves all the type signatures that were previously in the class, so all the clients that were compiled against the original code will continue to work just fine without recompiling. So even if the class you need to refactor is depended upon from everywhere, this is still a local refactoring. None of your refactor fearing coworkers even needs to know; it'll just be between us.

Did this buy us anything? Yes. All the dependencies of the class are now exposed for mocking and stubbing, without requiring changes to any calling code. Of course, your original constructor can't be tested, so if you have behavior or logic in the constructor that you need to test, then a little more creative refactoring is needed and it may not be supported with just automated IDE refactorings. Some might complain that we've modified the production code simply to enable testability... but to them I ask for alternatives. When weighing time commitments and code impacts, I find this "expose local constructor" refactoring to be of real benefit and frequently used.