I figured it would probably take a bit more than an hour to write up this post – reality intervened and two hours were exceeded. This is probably an experience many people share, whether from school, work, or personal life. Everything takes much longer than expected. Even though I’ve written countless essays, blog posts and papers – and they’ve all seemed to take commonly longer than estimated – I still didn’t see this coming. There’s a term in decision research for this phenomenon: the planning fallacy. In short, it means we are quite idiotic planners: we never see the delays and problems coming.
The planning fallacy is ubiquitous to the point of being scary. As Jonathan Baron recounts in his exquisitely fantastic book Thinking and Deciding (pp. 387-388):
Projects of all sorts, from massive works of construction to students’ papers to professors’ textbooks, are rarely completed in the amount of time originally estimated. Cost overruns are much more common than underruns (to the point where my spelling checker thinks that only the former is an English word). In 1957, for example, the Sydney (Australia) Opera House was predicted to cost $17 million and to be completed in 1963. A scaled-down version opened in 1973, at a cost of $102 million!
When it comes to our predictions of completion times, there seems to be no such thing as learning from history. It seems that every project we begin is considered only as a single instance, not as belonging to a category.
In fact, herein lies one of the keys to overcoming the problem. Relating a single project to a reference class of projects encourages statistical, data-driven thinking and reveals the delays to us. In an experiment (Buehler et al., 1994), students were much better at predicting when they were going to complete assignments when they were told to think about the past and connect it to the present assignment.
In my opinion, there are two kinds of errors underlying the planning fallacy:
1) Thinking we will have more time or energy in the future
2) Thinking we have prevented the errors that have happened before
The first case is especially related to personal projects. All the time there are several projects demanding attention. When we predict the future, we tend to believe that we will have more free time in the future than now. Since all those meetings, unpredictable surprises and events we just want to be at our not yet in view, the future looks promisingly empty. However, once it arrives it is very likely to look much like the present: the Sunday afternoon I was supposed to use for fixing my bike and catch up on reading was spent over a delightful brunch that we agreed on just a few days in advance. And so on. It never turns out like you planned.
In fact, herein lies one of the keys to overcoming the problem. Relating a single project to a reference class of projects encourages statistical, data-driven thinking and reveals the delays to us. In an experiment (Buehler et al., 1994), students were much better at predicting when they were going to complete assignments when they were told to think about the past and connect it to the present assignment.
In my opinion, there are two kinds of errors underlying the planning fallacy:
1) Thinking we will have more time or energy in the future
2) Thinking we have prevented the errors that have happened before
The first case is especially related to personal projects. All the time there are several projects demanding attention. When we predict the future, we tend to believe that we will have more free time in the future than now. Since all those meetings, unpredictable surprises and events we just want to be at our not yet in view, the future looks promisingly empty. However, once it arrives it is very likely to look much like the present: the Sunday afternoon I was supposed to use for fixing my bike and catch up on reading was spent over a delightful brunch that we agreed on just a few days in advance. And so on. It never turns out like you planned.
Plan vs. reality
In the second case – and I think this might be more relevant in complex projects – we believe that we have in fact learned from history, that we’ve plugged the gaps that proved our downfall in previous projects. Unfortunately, this is exactly what we thought previously, too! Surprising problems in project are surprising exactly because they are not the same errors as before. As before, just round the corner is something we didn’t think of, something that catches us off guard. If we only remembered that it’s always been like this…
Of course, I’m deliberately painting a gloomy picture here. We do learn from our mistakes and tend not to make the same mistakes again. But the point is that there will probably always be new mistakes – and this ought to be reflected in planning. I don’t think anybody is better off from the fact that projects run over their deadlines almost certainly. For example, by some estimates only 1% of military purchases are delivered on time and budget (Griffin & Buehler, 1999)! There’s certainly room for improvement!
And that improvement is possible. According to Buehler et al. (1994), others are less susceptible to the planning fallacy than the actors themselves. In simple terms: those not invested in the plan are more likely to have a conservative view of the completion time. Unfortunately, they are also likely to err in the opposite direction by stating too slow completion times! So, it’s probably a feasible idea to estimate the actual time by averaging your estimate and the outsider’s estimate. This is hardly optimal, but it’s better than relying on a single biased estimate.
So the recipe for better plans seems quite straightforward:
- stopping to think
- considering past projects and their results
- getting an outsider opinion and aggregating estimates
Of course, I’m deliberately painting a gloomy picture here. We do learn from our mistakes and tend not to make the same mistakes again. But the point is that there will probably always be new mistakes – and this ought to be reflected in planning. I don’t think anybody is better off from the fact that projects run over their deadlines almost certainly. For example, by some estimates only 1% of military purchases are delivered on time and budget (Griffin & Buehler, 1999)! There’s certainly room for improvement!
And that improvement is possible. According to Buehler et al. (1994), others are less susceptible to the planning fallacy than the actors themselves. In simple terms: those not invested in the plan are more likely to have a conservative view of the completion time. Unfortunately, they are also likely to err in the opposite direction by stating too slow completion times! So, it’s probably a feasible idea to estimate the actual time by averaging your estimate and the outsider’s estimate. This is hardly optimal, but it’s better than relying on a single biased estimate.
So the recipe for better plans seems quite straightforward:
- stopping to think
- considering past projects and their results
- getting an outsider opinion and aggregating estimates
RSS Feed