Simple Software Risk Estimates

I've worked with a lot of different project managers at a lot of different companies. Each one seems to have their own magic Excel spreadsheets for risk estimation. The good ones seem to track changes in risk on a week-to-week basis, while the bad ones like to give you a 40 hour block at the beginning of the project and call it done. I imagine the really great PMs get together at networking events and trade stories about the holy grail of Excel formulas that allows them to accurately model project risks. I've got no problem with that; I've wiled away many hours fiddling with Excel files. But please, please, can we have changes in risk reflected in comparable changes to the project timeline?

Which is why I fell in love with Steve McConnell's risk estimation method from his 10 Deadly Sins of Software Estimation keynote (registration required, but not for this version). My employer brought him in to deliver several of his keynotes, and this one was probably my favorite. (And if you work with me, you can access the recorded versions on the PDN).

Here is the simple idea... generate a set of risks as you normally would. But instead of giving them a rank (low/medium/high), give them an impact. How many hours will it cost the project if the risk event occurs? Then, estimate the risk's probability of coming true. Is there a 50% chance that the risk will occur? At this point, you can just multiply through (impact * probability) to determine how much time you need to add to the project. Over the scale of many risk items, the Law of Averages* should result in a number as accurate as your initial estimates.

OK, maybe you're confused because I didn't explain that well (three beers means I may have passed my Ballmer Peak). Time for a hypothetical example... let's say you want to replace all your legacy remote services with a new fangled REST based SOA using Axis2 and Groovy behind the scenes. Crazier ideas have been proposed!

Here are your risk exposure using Steve McConnell's method:

Risk	Impact	Probability	Exposure
REST does not offer message level security. Impact is porting all the services to SOAP.	4 Weeks	25%	1 Week
Clients hate SOA and want to use other technologies. Impact is rewriting all the services as FTP transfers	8 Weeks	10%	.8 Weeks
Axis2 does not support true REST semantics and clients demand it. Impact is porting to Restlet.	3 Weeks	25%	.75 Weeks
Groovy is too slow/too hard to adopt. You must port service classes to Java.	1 Week	10%	.1 Weeks
		Total:	2.65 Weeks

Given these risks, 2.65 weeks should be added to the project schedule (see that box on the lower right?). Hopefully you already brainstorm for project risks, so generating the list shouldn't be a challenge. Impact isn't new, because you're probably already assigning a value to the risk; maybe the value isn't an effort measure like this, but it's still a value. The only new parts should be probability (which is admittedly a swag guess) and a total result of how the schedule is effected.

For me, providing an additional swag estimate on probability is well worth the cost when the result is a clear and simple indicator of how additional risks push out the project timeline. It's time we stopped being surprised in project retrospectives that the "unknown variables" pushed out delivery dates when many of those unknowns could be anticipated by the midway point of the project. Crazy idea, I know.

* Oh, and about that Law of Averages? Wikipedia probably says it best:

As invoked in everyday life, the "law" usually reflects bad statistics or wishful thinking rather than any mathematical principle. While there is a real theorem that a random variable will reflect its underlying probability over a very large sample, the law of averages typically assumes that unnatural short-term "balance" must occur.

This seems a good time to say, "Your mileage may vary."