Can We Rely on NAEP Rankings?

Wyoming is talking. In fact, many are happy. What they're jubilant about is Wyoming schools' 2015 National Assessment of Education Progress (NAEP) scores. There is room to be pleased about where Wyoming ranks versus other jurisdictions in reading and math in both Grades 4 and 8.

• Reading Grade 4 – Ranked 8th of 52 (50 states plus D.C. and Department of Defense schools).
• Reading Grade 8 – Ranked 12th of 52.
• Math Grade 4 – Ranked 6th of 52.
• Math Grade 8 – Ranked 12th of 52.

Honest, unbiased analysis tells us, unfortunately, the rankings mean very little. There are several reasons. The first one is about how our ranking drops in both subjects between the two age groups. We lose traction versus other states as the kids get older. That's a matter for concern because if the pattern persists our older kids won't do well in upper grades. We've seen this already in ACT scores.

Our second issue is about the underlying scores themselves. Look at Graph 1 which shows Math Grade 4 where we rank 6th of 52. First, we see one jurisdiction is in the 250-260 bracket (Massachusetts at 250.5658) on the scale of 0 to 500. That's pretty low for a best score. If we're to compete internationally, we've got to do better than this.

Interesting, too, is how tightly grouped all the jurisdictions are. In Graph 1^[1], all are tucked into three brackets spanning only 30 points (the median average score is 240.4). Granted, we're talking about overall state averages, but it's still very narrow dispersion given that the scale is 500 points. It clearly indicates there is very little difference between jurisdictions. Graph 2 shows the same general state of affairs for Reading Grade 8.

That takes us to our next quibble. See Graph 3 for Math Grade 4. It, too, underscores how little the ranking among the states means. As mentioned above, Wyoming scored 246.7598. We "beat" Virginia in ranking, but it was by 0.1471 points. Does such a small margin on a scale of 500 means anything, especially in something as important as K-12 education? In Graph 4, Wyoming whupped Pennsylvania by 0.0572 points. The rankings mean next to nothing and the celebration we hear does nothing but direct our attention away from how poor our scores are. Not only that, but there has been hardly perceptible improvement over time. Wyoming needs better scores and credible reporting of those facts, not obfuscation.

^[1] SOURCE: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), 2015 Mathematics Assessment.