Can We Rely on NAEP Rankings? Part 2

Wyoming Liberty Group recently wrote Can We Rely on NAEP Rankings Part 1 Wyoming's rankings versus other states in National Assessment of Education Progress (NAEP) scores. We pointed out that, while our rankings look very good, appearances are deceiving. The scores were:

• Reading Grade 4 – Ranked 8th of 52 (50 states plus D.C. and Department of Defense schools).
• Reading Grade 8 – Ranked 12th of 52.
• Math Grade 4 – Ranked 6th of 52.
• Math Grade 8 – Ranked 12th of 52.

What's deceiving is how little the rankings mean because the differences between states are very small. For example, in Math Grade 4 Wyoming "beat" Virginia in ranking, but it was by 0.1471 points on a 500 point scale. Similarly, in Reading Grade 8, Wyoming whupped Pennsylvania by 0.0572 points.

Wyoming needs better scores and credible reporting of meaningful facts, so we offer a thought experiment for a different view. We want to know what each state's cost of achieving its NAEP scores is, i.e. which jurisdiction gets the most NAEP points for in return for its expense.

Our experiment takes each state's average cost per year per pupil and divides it by the sum of the state's average math and reading scores for Grade 8. Doing this provides us with a crude measure of cost per NAEP point achieved. Graph 1 presents the results. Wyoming is highlighted in brown. Neighboring states are highlighted gold.

Wyoming does not rank well here at all. It is 7th highest cost (7th least bang for the buck) in the nation. We've all heard various reasons, for example, sparse population, long travel distances, teachers don't want to come to Wyoming because of poor quality of life, and Wyoming children are difficult to handle.

Those explanations are debunked by Graph 2. It condenses Graph 1 to seven states, Wyoming plus its six neighbors. All of them have similar cost of living (certainly much lower than Massachusetts'). Wyoming achieved a total sum NAEP score of 1,030.4 points (out of a possible 2,000) and paid $15.33 for each. Utah scored 1,024.3 points and paid $6.35 per point. Therefore, Wyoming is paying an exorbitant amount to obtain bragging rights.

Readers should also note this analysis only shows cost of school operations. It excludes the cost of buildings and building maintenance. The State of Wyoming is obligated to pay 100 percent for those as well. Our neighbor states do not pay anything close to that.

In addition, readers should look at Wyoming Liberty Group's recent blog "Reality Check" lest they think Wyoming's trend in scores is positive and is cause for reassurance. Improvement has been minimal.

We're pleased that Wyoming ranks in or near the top ten jurisdictions in math and reading NAEP scores; we just wish it meant more. And we wish we weren't paying such a high price for such modest bragging rights.

Wyoming's K-12 "basket of goods" is breaking us financially and achieving little in measurable outcomes. Our legislature must aggressively constrain its costs because we have more of a spending problem than a funding problem.