Agile Forecasting
Santa was kind to me this year. I’ve been good and as a reward I found a shiny new Kindle Paperwhite under the tree on Christmas morning. With plenty of downtime over the holidays, the first book I read was Superforecasting by Philip Tetlock and Dan Gardner. It offers a fascinating insight into the psychology of forecasting, and I drew a lot of parallels to my own experience of Agile Estimation.
Estimation vs Forecasting
Being asked to guess the future sounds familiar to any Software Engineer. We don’t make flip-a-coin guesses though, at least I hope not. So what is the difference between Estimation and Forecasting? It can seem subtle at first, but understanding this will help us talk about how we arrive at delivery dates.
Estimating
Estimates are about output. Reality has infinite granularity, so we construct a simplified model with approximations for things that we can’t know for certain. For example, the target story size is 5 days. We have 10 stories, so we estimate 50 days of effort.
Forecasting
Forecasting is about outcomes. We will build “Widget X” and have 50% of customers using it by 2016. A forecast is usually backed by a path to achieve its outcome.
Superforecasters
In the book Superforecasting, the author identifies a special group of Superforecasters. These are forecasters that have demonstrated much better accuracy over time than even domain experts. One of the unique behaviours of Superforecasters (there are others), is that they frequently update their forecasts when new information becomes available. Small tweaks, made regularly, make for more accurate forecasts. This is something we Estimators traditionally don’t do a good job of. Being accurate is not really the point of an estimate, except when that’s how it is interpreted which unfortunately is most of the time. So we need to act more like forecasters.
Superestimating
Typically we estimate a project at the start and then execute to completion. It should be of no surprise then that most projects are late, firstly Engineers tend to under-estimate, and secondly most of the knowledge and expertise on a specific project is learned during its execution. In the same way that the Superforecasters outperformed their peers because they frequently adjusted their forecast due to new information, we should be frequently updating our estimate of project delivery during the execution phase, when vital new information is being learned daily.
By regularly updating the estimates as new knowledge is gained we can significantly improve our accuracy. This affects the consumer of our estimates and may be scary for them if they are not Agile. Remember “customer collaboration over contract negotiation” means it’s better to keep talking to your business, rather than having a fixed date set before most of your learning has happened.
Estimating a range will help your business be prepared for change much better that estimating an exact day which turns out to be wrong. Ideally the range will decrease over time with each tweak, but not always — sometimes new information brings increased uncertainty. Your business needs to be responsive to these changes too. Regular communication of estimates is crucial, but change itself shouldn’t be a surprise.
Avoid Groupthink
Introducing diversity during estimation will also help improve accuracy. It’s too easy for dominant individuals to inadvertently lead the group astray and possibly reduce innovation by being loud and persuasive. Ways to achieve this could be rotating between teams or simply pulling in members of other teams during Planning & Estimation sessions to get a more varied opinion.
Measure
It’s imperative that we keep score of how good our estimates are. Without tracking performance there’s no way to separate improvements or failures from simple regression to the mean.
Takeaways
When it comes time to estimate, improve your predictability by acting more like a forecaster:
- Recalibrate estimates often
- Provide a confidence range, not exact date
- Seek diverse input during estimation sessions
- Know how good your estimates are historically
In my next blog post I’ll talk about how to use your historical performance and the Monte Carlo Algorithm to work out a range and confidence factor for your future estimates.
