A new article in the Journal of the American Planning Association offers a damning indictment of planners’ use of forecasts for transportation infrastructure projects. Here’s the abstract:
This article presents results from the first statistically significant study of traffic forecasts in transportation infrastructure projects. The sample used is the largest of its kind, covering 210 projects in 14 nations worth U.S.$59 billion. The study shows with very high statistical significance that forecasters generally do a poor job of estimating the demand for transportation infrastructure projects. For 9 out of 10 rail projects, passenger forecasts are overestimated; the average overestimation is 106%. For half of all road projects, the difference between actual and forecasted traffic is more than ±20%. The result is substantial financial risks, which are typically ignored or downplayed by planners and decision makers to the detriment of social and economic welfare. Our data also show that forecasts have not become more accurate over the 30-year period studied, despite claims to the contrary by forecasters. The causes of inaccuracy in forecasts are different for rail and road projects, with political causes playing a larger role for rail than for road. The cure is transparency, accountability, and new forecasting methods. The challenge is to change the governance structures for forecasting and project development. Our article shows how planners may help achieve this.
This article is a breath of fresh air, and it validates the concerns of those of us who have been warning for years about the extreme mismatch between forecasted and actual demand, particularly for new light rail projects. We repeatedly hear rosy ridership forecasts used as a primary selling point for new light rail systems when real world experience indicates that new systems almost never get close to meeting their promoters’ lofty expectations. Here’s more from the article:
We conclude that the traffic estimates used in decision making for rail infrastructure development are highly, systematically, and significantly misleading (inflated). The result is large benefit shortfalls. For road projects the problem of misleading forecasts is less severe and less one sided than for rail. But even for roads, for half the projects the difference between actual and forecasted traffic is more than ±20%. On this background, planners and decision makers are well advised to take with a grain of salt any traffic forecast that does not explicitly take into account the uncertainty of predicting future traffic. For rail passenger forecasts, a grain of salt may not be enough. The data demonstrate to planners that risk assessment and management regarding travel demand must be an integral part of planning for both rail and road projects. This is especially the case because prediction errors in the early stages of forecasting appear to amplify, rather than decrease, in later stages. . . . . The striking difference in forecasting inaccuracy between rail and road projects documented above may possibly be explained by the different procedures that apply to how each type of project is funded. Competition for funds is typically more pronounced for rail than for road, which creates an incentive for rail promoters to present their project in as favorable a light as possibleââ?¬â??that is, with overestimated benefits and underestimated costs…We speculate further that rail patronage will be overestimated and road traffic underestimated in instances where there is a strong political or ideological desire to see passengers shifted from road to rail, for instance for reasons of congestion or protection of the environment. Forecasts here become part of the political rhetoric aimed at showing voters that something is being doneââ?¬â??or will be doneââ?¬â??about the problems at hand. In such cases it may be difficult for forecasters and planners to argue for more realistic forecasts, because politicians may use forecasts to show political intent, not the most likely outcome. . . . . The policy implications of our findings are clear. First, the findings show that a major planning and policy problemââ?¬â??namely misinformationââ?¬â??exists for this highly expensive field of public policy. Second, the size and perseverance over time of the problem of misinformation indicate that it will not go away by merely pointing out its existence and appealing to the good will of project promoters and planners to make more accurate forecasts. The problem of misinformation is an issue of power and profit and must be dealt with as such, using the mechanisms of transparency and accountability we commonly use in liberal democracies to mitigate rent-seeking behavior and the misuse of power. To the extent that planners partake in rent-seeking behavior and misuse of power, this may be seen as a violation of their code of ethicsââ?¬â??that is, malpractice. Such malpractice should be taken seriously by the responsible institutions. Failing to do so amounts to not taking the profession of planning seriously.
Ouch. The whole article is definitely worth a read, as it also outlines some remedies that would inject some sanity and accountability into the decisionmaking process. Suggested remedies include:
- Transparency and public control (e.g., federal “agnosticism” regarding transport mode, as it relates to funding; independent peer review of forecasts; and professional/criminal penalties levied for professional malpractice);
- Competition and market control (e.g., reducing risk by making project approval contingent on the willingness of private financiers to contribute at least one third of the total capital costs; avoiding full public financing; requiring forecasters to share financial responsibility for covering benefit shortfalls and cost overruns resulting from forecasting bias and misrepresentation); and
- Reference class forecasting (comparing proposed projects with a group of other similar projects)
Kudos to the researchers for bringing this dirty little (not-so-hidden) secret to light, and also to JAPA for publishing it. I doubt that most of the APA rank-and-file will welcome this research with open arms, given that they’ve invested themselves so heavily in light rail advocacy. But often in life the painful truths are the ones that most desperately need to be confronted.