Here we go again. Just a few days ago, two education professors released a shockingly misleading piece on private school vouchers.
Here are the facts.
The authors claim that the “latest evidence” shows that the Washington, D.C., voucher program has “large, negative impacts on academic achievement.” The only problem is that the most recent federal evaluation of the D.C. voucher program does not show any negative effects on student test scores after three years. In fact, the study finds statistically significant positive effects on reports of safety, satisfaction, and attendance. What’s more, the D.C. choice program produces these benefits at about a third of the cost of nearby public schools.
How could anyone get these findings so wrong?
The authors irresponsibly cited the first– and second-year evaluations of the D.C. voucher program which found negative effects on math test scores, but no effects on reading, while completely omitting the most recent third-year results finding no effects on test scores. In other words, students that won the lottery to use the voucher program caught up to their public school peers on math achievement after three years.
But by omitting the most recent D.C. evaluation, the authors were able to further (falsely) claim that other researchers were wrong to think that initial test score losses would disappear since “more recent follow-up studies show that the harm is significant and sustained.” Obviously, citing the most recent school voucher study, showing just that, would contradict their own claim.
Of course, it’s entirely possible that both education scholars missed the most recent school choice results. After all, the study had only been public for a little over three weeks by the time their piece came out. But overlooking such important results when summarizing the “latest research” would be negligent of “researchers who study school choice and education policy.”
The authors cite just three other evaluations (two of which are nonexperimental) to support the notion that “vouchers harm student learning.” But researchers should cite all of the most rigorous existing studies to avoid unintentional cherry-picking. Here’s the entire picture.
Sixteen random assignment studies link private school choice programs to student test scores in the United States. The majority (11) of the 16 gold-standard studies find statistically significant positive effects on test scores overall or for student subgroups. Only 2 of the 16 experiments (both from Louisiana, where school choice is highly regulated) find statistically significant negative effects on student test scores. The most rigorous evidence linking private school choice to student test scores actually leans positive.
We’re not done yet.
The authors cite just one study to support the claim that “some of the most recent research finds that vouchers don’t really lead to better college enrollment, either.” That’s highly misleading. Seven rigorous studies link private school choice programs in the U.S. to college enrollment. Five of the seven find statistically significant positive effects of school choice on college enrollment overall or for student subgroups. Two studies find no effects. Zero find negative effects.
Some school choice researchers, including myself, have summarized empirical evidence showing standardized test scores may not be the best metric of success. But instead of engaging with the scientific evidence directly, the authors simply wave it off by wrongly accusing school choice researchers of goalpost-shifting, even though choice researchers were interested in non-test-score outcomes (and argued against mandating standardized testing) far before any experimental studies found negative effects of voucher programs on test scores.
There are a lot of disagreements in the school choice debate. But we should all be able to agree on one thing: At a bare minimum, we should be able to trust academics to responsibly report (and engage with) the scientific evidence. Yet, here we are.
This article originally appeared in the Washington Examiner.