This column was co-authored by Max Eden, a senior fellow at the Manhattan Institute. It originally was published in the Washington Examiner.
Education advocacy groups are lobbying for a $250 billion bailout for K-12 schools and higher education. But Congress and the Trump administration should take a careful look at the research before they decide whether to cut another coronavirus stimulus check.
There’s little doubt that the COVID-19 pandemic will harm learning for many students. A third of a school year was effectively canceled. The massive spike in unemployment and the economic fallout will pose significant stress on families and government budgets. But would school budget cuts harm student outcomes?
A 2018 working paper by Northwestern University professor Kirabo Jackson and his colleagues claimed to find clear evidence of harm caused by budget cuts from the Great Recession. Their first draft, which was covered in places like the Economist, Vox, Chalkbeat, FutureEd, and Education Next found that a 10 percent cut in education spending yielded a 7 percent of a standard deviation decrease in academic achievement, and, over four years, decreased graduation rates by nearly 3 percentage points. A revised draft (featuring a substantially smaller effect on achievement and examining college-going rather than graduation) was recently accepted for publication at American Economic Journal: Economic Policy, where Jackson serves as an editor.
Fortunately for America’s students and taxpayers, the results are largely an artifact of a flawed methodology. The researchers believed that they could isolate the effects of spending cuts by looking at the share of school revenue that came from the state. Some states provide a higher share of revenue to local school districts than others, where schools rely more heavily on local property taxes. Because state revenues are more vulnerable to economic downturns than local revenues, they argued that we could view the Great Recession as a sort of natural experiment: Schools in some states will lose more money than schools in others.
The problem is that the share of funding provided by the state is not a matter of chance. States that agreed to pick up a greater share of the tab did so because of demographic, economic, and political factors, all of which may be related to how students are likely to fare academically. Pretending that this is a random variable when it is actually systematically related to student performance distorts their results and likely yields false findings.
The authors’ data show that reliance on state revenue is negatively correlated with achievement. And research that Jackson has deemed methodologically rigorous suggests that a greater reliance on state revenue independently and directly harms academic achievement, concluding that “a growing state share [of revenue] generally is accompanied by falling average SAT scores.” To explain this, they “hypothesize that the state share in public school revenue captures another dimension of state involvement,” such as regulation.
Given that the researchers knew that this variable could directly affect the outcome, it would have made more sense to incorporate it as a control. The observed relationships in the data, and a simple regression analysis, suggest that doing so would have reduced their estimate of the effect of spending on student achievement. In fact, because the study’s regression results were already so close to zero, it’s plausible that correctly incorporating this variable as a control instead of an instrument could have produced a finding that spending cuts actually improve educational outcomes.
The researchers claim to have found evidence that spending cuts harm students. But perhaps what they actually found was that state regulations harm students. Jackson has argued that “the question of whether money matters is essentially settled.” While most older studies found no correlation between spending and outcomes, he has argued that more rigorous methodologies have provided academics with greater power to reduce bias in causal estimates. That is true. But it is also true that they have provided academics with greater power to increase bias in causal estimates in ways that will be obscure to most readers, even to the editors of influential academic journals.
The broader debate about the effect of spending on student outcomes should and will continue. The new consensus that “money matters after all” is far weaker than advertised.
Congress is now considering its next steps for supporting a flailing economy after an unprecedented $2 trillion in stimulus spending has brought our national debt-to-gross domestic product ratio to the highest level since World War II. We are already borrowing heavily from the next generation. We owe it to them to spend their money wisely, in ways that are most likely to benefit them.