They say you shouldn’t miss the forest for the trees. Unfortunately, several people in the education policy debate are doing just that. Researchers and journalists are focusing on the effects of education policies on standardized test scores while ignoring more important long-term outcomes such as crime and earnings. That’s obviously a problem.
Here’s a case in point. Matthew Yglesias, co-founder of Vox, just quoted five rigorous studies linking families’ schooling selections to student outcomes. Yglesias quoted snippets from the five abstracts finding no change in students’ test scores. The only problem is that he completely omitted all of the positive effects on long-term outcomes such as health, safety, crime reduction, and earnings.
For example, Yglesias cited a rigorous study of families’ schooling selections finding that “on average, sought-after schools do not improve student test scores.” However, perhaps unintentionally, he left out that getting the chance to go to those same preferred schools reduced teen pregnancies and improved “educational attainment, occupational rank, earnings, and health.”
He cited another rigorous study finding that families’ schooling selections had “no effects on traditional outcomes.” However, Yglesias again forgot to include the positive effects of those same selected schools on students’ reports of safety.
But that’s not all.
Yglesias also cited an experimental study finding that students winning a lottery to attend a public school of choice in Chicago didn’t provide “any benefit on a wide variety of traditional academic measures, including standardized test scores, attendance rates, course-taking, and credit accumulation.” It’s encouraging that he expanded this particular quote to include non-test score outcomes here.
But, again, Yglesias somehow forgot to include the positive effects of the same schools of choice on reducing “self-reported disciplinary incidences and arrest rates.”
This isn’t the first time someone has cited lackluster test score results while completely omitting the important positive effects of attending a chosen school. For example, reporters such as Valerie Strauss focused on initial negative effects of the D.C. voucher program on test scores without even mentioning that the same study found positive effects on student safety.
We can all learn something important from these omissions. The fact that the story changes substantially when long-term outcomes are omitted tells us that standardized test scores are not good proxies for true success in the long-run. My recent peer-reviewed summary of the evidence shows that there are many more examples of disconnects between schools’ effects on test scores and their effects on long-term outcomes.
We should look at all available outcomes when evaluating any education policy. But we should always prefer valuable long-run outcomes to standardized test scores. And we should think twice about judging families who choose schools that don’t increase their children’s test scores. After all, this evidence suggests families know a lot more about their children’s needs than the experts.
A version of this column originally appeared in the Washington Examiner.