Grading the Graders

How advocacy groups mislead public on pollution

Earlier this year, the American Lung Association handed out F grades for air quality to almost 60 percent of U.S. counties.[1] The Sierra Club recently followed suit, failing 70 percent of cities for their efforts to improve air quality with spending on public transit.[2] Based on these poor grades, advocacy groups are recommending billions of dollars in new regulatory costs and substantial new restrictions on Americans— freedom to choose where and how they live and travel. But what do these letter grades really mean in terms of peoples— actual health and welfare?

Simple grading systems are appealing because they boil down a wide range of complex information and analysis into a single letter that anyone can easily understand. Your county got an F for air quality? Then your air must be unsafe to breathe. Want to change those F—s to A—s? Prod your local officials into clamping down on automobiles and —suburban sprawl,— and building more public transit. That—s the idea anyway. Unfortunately, these grading systems turn out to have little to do with the actual environmental health and safety factors they purport to evaluate.

Sierra Club Mistaken On Link Between Transit and Air Quality

In its report —Clearing the Air with Transit Spending— the Sierra Club graded cities based on the fraction of transportation funding spent on public transit versus highways, and motor vehicle pollution per capita. Cities with greater transit funding and less per capita motor vehicle pollution received better grades.[3]

Though the Sierra Club claimed that cities with better grades are also doing a better job at reducing air pollution, it appears the report—s authors never checked to see if their grading system had any relation to actual air quality. It doesn—t. Figure 1 compares the Sierra Club—s grades for metro areas to actual pollution levels.[4] Note that cities that got the best grades from the Sierra Club have the worst air pollution—just the opposite of what one would expect if the grades tracked actual health risk.[5]

The Sierra Club recommends increasing transit funding, reducing road funding, and requiring higher density, transit-oriented development to fight sprawl, decrease driving and thereby reduce air pollution.[6] But as we—ve seen, the data show that there is no relationship between transit funding levels and air quality. That—s because in most American cities, more than 90 percent of people commute to work by car regardless of transit funding levels. Even in cities like Chicago and Washington, DC, which have extensive public transport, more than 80 percent of people commute by car. In the New York metro area, with by far the best public transit system in the country, two-thirds of commuters still drive to work.[7] There is also no correlation between trends in per-capita transit use and transit funding levels in American cities. Per-capita transit use declined in most metropolitan areas between 1990 and 1999, regardless of funding levels.[8]

Denser development also has little effect on the amount people drive. Using data from the National Personal Transportation Survey, researchers from the Georgia Institute of Technology found that nearly tripling metropolitan density decreases driving by only about seven percent.[9] People drive because the automobile can—t be beat for flexibility and convenience. Even in Europe, with its dense cities, extensive public transport, and $4 to $5 per gallon gasoline, automobile use has skyrocketed during the last few decades.[10] Europeans, like Americans, embraced the automobile when rising incomes put auto travel within reach of most households.

Number of high ozone days is average for all cities receiving given letter grade. Ozone data from EPA AIRData Web site,

American Lung Association Inflates Air Pollution Exposure

Just as with the Sierra Club, the American Lung Association—s (ALA) air quality grades also have little to do with actual air pollution levels in American cities. In —The State of the Air 2001,— ALA graded counties— air quality by tallying the number of days each year in which at least one monitoring location in a county registered an ozone level of at least 0.085 parts per million (ppm).[11] Unfortunately, this method vastly overestimates real air pollution exposure. For example, 99 percent of people in San Diego County live in areas that meet EPA—s ozone health standards (see Figure 2).[12] Nevertheless, ALA gave San Diego County as a whole an F grade for air quality, because one rural location (Alpine) with 0.3 percent of the county—s population has elevated ozone. ALA may thus have needlessly scared the 3 million residents of San Diego County into thinking their air is unsafe. ALA made the same error for dozens of other populous counties around the U.S., counting tens of millions of people as being exposed to high levels of air pollution, when in fact their air is clean.

Areas with fewer than four days per year exceeding 0.084 ppm ozone meet EPA—s 8-hour, 0.085 ppm ozone standard. See endnote 12 for explanation. San Diego County ozone data are from California Air Resources Board, —California Ambient Air Quality Data CD-ROM,— CD Number PTSD-00-014-CD, Planning and Technical Support Division, November 2000.

Overlooking a History of Progress

What is also lost in these simplistic letter grades is context. The Sierra Club and ALA discount America—s success in combating air pollution during the last 20 years. According to Environmental Protection Agency (EPA) monitoring data, average ozone levels in the U.S. dropped 24 percent between 1980 and 1999.[13] Areas with the worst air pollution have had the greatest success. For example, Los Angeles and Houston reduced high ozone days by 80 and 50 percent, respectively, between 1980 and 2000.[14] The nation achieved these reductions at the same time that driving increased by 75 percent.[15] Even as air quality continues to improve, ALA nevertheless claims that the number of people exposed to high air pollution levels is increasing.

The Sierra Club also errs in its claim that any air pollution improvements achieved so far are being —cancelled out,— because people are driving more, and more and more motorists are purchasing big Sport Utility Vehicles.[16] In reality, technological progress is winning the war on air pollution. On-road pollution measurements show that even with more SUVs on the road, the average new vehicle starts out cleaner and stays cleaner as it ages when compared with older models, and average emissions of the vehicle fleet are dropping several percent per year.[17] EPA predicts, based on regulations already on the books, that average motor vehicle per-mile emissions will drop by 85 percent during the next 20 years due to ongoing fleet turnover to cleaner vehicles. [18] This means that air pollution will continue to decline even with large increases in driving.[19]

Technological advances are bringing us cleaner air along with the single-family homes, automobile-based travel, and suburban lifestyles that most Americans desire. Unfortunately, the ALA and Sierra Club are only two among many advocacy groups using phony grading systems to foment public fear and increase support for unnecessary and undesirable restrictions on where and how we live and travel. In recent months, other groups have released equally misleading reports on the supposed dangers of suburban development for human health and welfare.[20] These reports, with their quantitative data analysis and university-style grades, have the patina of rigorous science, but none of the substance. Though —anti-sprawl— advocates earn failing grades in Science and Policy, by bombarding and bewildering the media and the public with specious analysis, they may just hoodwink their way into an A in Politics.

Joel Schwartz is an adjunct fellow at Reason Foundation and visiting scholar at American Enterprise Institute.


[1] American Lung Association, The State of the Air 2001 (Washington, DC, 2001)

[2] Sierra Club, Clearing the Air with Transit Spending (Washington, DC, 2001)

[3] Ibid.

[4] Air pollution data downloaded from the Environmental Protection Agency—s AIRData Web site,

[5] Number of high ozone days is the number of days in 2000 exceeding an ozone level of 0.084 parts per million, based on the most polluted location in each metropolitan area. The Sierra Club used —+— and ——— grades, such as —B+— in its grading system. We—ve collapsed all grades into the letter grade alone for simplicity of presentation.

[6] Sierra Club, Clearing the Air with Transit Spending

[7] Data from Federal Highway Administration and 1990 Census, summarized in R. O’Toole, The Vanishing Automobile and Other Urban Myths (Bandon, Oregon: Thoreau Institute, 2001), p. 248-249.

[8] The Sierra Club report includes data by metropolitan area on both the number of dollars spent on public transit per $100 spent on roads, and per capita public transit spending. Combining these data with trends in per capita transit use between 1990 and 1999, we found a correlation coefficient of zero between both of the Sierra Club measures and trends in transit use. That is, cities with higher levels of transit funding experienced similar (downward) trends in transit use as cities with lower transit funding. Data on per capita transit use are from U.S. Department of Transportation, summarized in W. Cox, —Average Per Capita Public Transport Boardings Ranked by US Metropolitan Areas: 1999,— and W. Cox, —Annual Per Capita Boardings: 1980-1997 & Market Share,— both in Urban Transport Fact Book, on the Web at

[9] C. L. Ross and A. E. Dunning, Land Use Transportation Interaction: An Examination of the 1995 NPTS Data (Atlanta, Georgia: Georgia Institute of Technology, 1997)

[10] Miles driven per capita increased by factors of 2 to 6 in major European cities between 1960 and 1990. See J. R. Kenworthy and F. B. Laube, An International Sourcebook of Automobile Dependence in Cities, 1960-1990 (Boulder, Colorado: University of Colorado, 1999), cited in R. O’Toole, The Vanishing Automobile and Other Urban Myths, p. 266.

[11] American Lung Association, The State of the Air 2001.

[12] EPA has two ozone health standards. To meet the current standard, no monitoring site in a region can exceed an ozone level of 0.124 ppm (averaged over a 1-hour period), more than 3 times in any consecutive three-year period. To meet the new standard, the average of the 4th highest ozone level (averaged over an 8-hour period) from each of the last three years must be less than 0.085 ppm. In practice, this means that, on average, no location can exceed 0.084 ppm ozone more than about four times per year and still comply with the standard. The new standard is not yet legally binding, but EPA hopes to implement it in the near future. All San Diego County locations except Alpine comply with both of the EPA ozone standards.

[13] Environmental Protection Agency, Air Quality Trends 1999 (Washington, DC, 2000).

[14] Based on ozone monitoring data from the California Air Resources Board and the Texas Natural Resource Conservation Commission.

[15] Vehicle use trends come from the U.S. Bureau of Transportation Statistics ( and the Federal Highway Administration (

[16] Sierra Club, Clearing the Air with Transit Spending.

[17] F. Gofa, et al., —Changes in on-Road Emissions and Emission Factor Model Predictions for the Van Nuys Tunnel: 1987 to 1995,— Submitted to Journal of the Air and Waste Management Association (2001).

[18] M. Beardsley, —MOBILE6: EPA’s Highway Vehicle Emissions Model,— 11th On-Road Mobile Source Emissions Conference, San Diego, California (Coordinating Research Council, Atlanta, Georgia).

[19] For example, if per-mile motor vehicle emissions drop by 85 percent, and driving increases by 50 percent during the next 20 years, then total motor vehicle emissions would still drop by about 77 percent.

[20] See, for example, Surface Transportation Policy Project, Mean Streets 2000: Pedestrian Safety, Health, and Federal Transportation Spending (Washington, DC, 2000); R. J. Jackson and C. Kochtitzky, Creating a Healthy Environment: The Impact of the Built Environment on Public Health (Washington, DC: Sprawl Watch Clearinghouse, 2001).