In advance of the passage of the bipartisan infrastructure bill, Wall Street Journal reporter David Harrison wrote a thoughtful article titled, “Billions Spent on Roads and Transit Projects Are Often Based on Optimistic Forecasts.” Citing a recent National Academies study of cost overruns and overestimates on drivers and passenger numbers, his primary point was summarized by traffic consultant Robert Bain: “The implication is rather stark: It’s wasting resources.”
In the former Soviet Union and its Eastern European satellites, post-Cold War analysis of industrial data found that many or most of that bloc’s state-owned enterprises were actually “value-subtracting enterprises”—the value of their outputs was less than the value of their inputs. That is an effective way of making a country poorer; it’s the opposite of what real investment is about.
The WSJ article was followed by an article in Engineering News-Record by Peter Reina: “Study: Cost-Benefit Analyses Are ‘Worse than Worthless.’” It summarized recent research from the University of Oxford’s Bent Flyvbjerg and Dirk Bester, in a forthcoming paper in the Journal of Benefit-Cost Analysis. Flyvbjerg sent me the paper, which is already available online at https://bit.ly/3oVHjRn.
Basically, the authors relied on an updated version of an international database on megaprojects first used in Flyvbjerg’s classic book, Megaprojects and Risk (Cambridge University Press, 2003). That book found that transportation megaprojects routinely cost a lot more than forecast and often serve far fewer customers than projected in the studies used to justify their construction funding.
The new paper (“The Cost-Benefit Fallacy: Why Cost-Benefit Analysis Is Broken and How to Fix It”) draws on a database totaling 2,062 infrastructure projects, both transportation and non-transportation (including dams, power plants, and buildings). The aim is to compare actual and projected (1) construction costs and (2) benefits, especially riders/customers in the transportation megaprojects.
Here is a summary of the average results for the transportation projects:
| Type | n | Cost overrun | N | Benefits vs. projection |
| BRT | 6 | 1.41 | 4 | 0.42 |
| Rail | 265 | 1.40 | 74 | 0.66 |
| Tunnels | 48 | 1.36 | 23 | 0.81 |
| Bridges | 49 | 1.32 | 26 | 0.96 |
| Roads | 869 | 1.24 | 532 | 0.96 |
In other words, the average rail project cost 40% more than estimated and its benefits were only 66% of what was projected. The average roadway project cost 24% more than estimated and its benefits were 96% of the projected amount.
The paper’s next task is to assess whether these results were more likely due to error or to bias. The authors point out that if errors were the cause, overruns and underruns would be about equally likely to occur. But their statistical analysis demonstrates the results are heavily skewed by what Flyvbjerg has previously identified as “optimism bias,” and by what other researchers including the late John F. Kain labeled as “deception” and the late Marty Wachs of UCLA referred to in the title of a journal article as “lying with numbers.”
Flyvbjerg and Bester add, “For not a single one of the eight investment types in [their] Table 1 did forecasters overestimate cost and for not a single investment type did they underestimate benefits, on average.”
From this they conclude that benefit-cost analysis as practiced today for megaprojects is “highly inaccurate and biased…The data show ex-ante cost-benefit analysis to be so misleading as to be worse than worthless, because decision makers might think they are being informed by such analysis when in fact they are being significantly misinformed about the return on the planned investments.”
After further discussion, the authors suggest four steps to reform cost-benefit analysis, as follows:
- Systematically de-bias cost-benefit forecasts using reference-case estimates.
- Introduce skin-in-the-game for cost-benefit forecasters.
- Require independent audits of cost-benefit forecasts.
- Adapt cost-benefit forecasting to the messy, non-expert character of democratic decision-making.
The first may help, but the danger is that the adjusted cost estimate might constitute a new, higher baseline from which change orders could still proliferate. But the authors cite some successes of this approach in the United Kingdom and Denmark, so it may be useful.
Skin in the game is inherent in design-build-finance-operate-maintain (DBFOM) public-private partnerships, with equity investors putting in hard cash and committing themselves to design-build contracts and operational performance measures. For revenue-risk DBFOMs, the investors themselves take on traffic and revenue risk, but that does not work for availability-payment P3s, even hybrid ones where states do the tolling.
Independent audits are essential before committing to the financing of the project, and in the wake of some Australian and U.S. bankruptcies of revenue-risk DBFOM highway and tunnel projects, the rating agencies have adopted more stringent reviews of project companies’ traffic and revenue projections. We are also seeing more conservative financial structures than in the gung-ho days before the 2008 financial market crisis.
I’m not sure I understand what Flyvbjerg and Bester mean in their brief discussion of the democratic process as at least as important as properly vetted cost-benefit analysis. Obviously, in democratic societies, we cannot ram projects down people’s throats. But we also need to recognize that one of the underlying causes of decisions to proceed with some projects whose costs far exceed their benefits is the politicization of infrastructure decision-making. This often includes legislators’ tendency to prefer ribbon-cutting ceremonies for new projects over ensuring proper ongoing maintenance and poorly-supported beliefs about the viability of rail transit regardless of its cost and ridership (cf. Honolulu) and a similar belief in some quarters about the unquestionable need for hugely-expensive high-speed rail (cf. the California high-speed rail project).
The art in crafting public-private partnership laws and practices (e.g., creating dedicated P3 units) is to gain political consent for the process and to protect its details from political interference and micromanagement. Obviously, no transportation megaproject is going to be authorized and built without the sponsoring government’s consent. But to avoid future debacles like Boston’s Big Dig and California high-speed rail project, today’s large-scale politicization of megaproject decision-making is an obstacle that must be overcome.
A version of this column originally appeared in Public Works Financing.