Schools, in the Eyes of the Beholders’ Websites

You know how parents don't always agree about school evaluation? Neither do school ratings websites.

In an era of increasing school choice, websites like GreatSchoolsNiche and SchoolDigger rate thousands of schools in all 50 states to help parents choose the best public and private schools for their children. But just as different families apply different criteria for their educational choices, a Choice Media investigation reveals disparities in the sites’ results.

Take the school district of Plano, Texas in the Dallas/Fort Worth area. Niche gives it a superb ranking of #10 out of the 995 districts for which it has figures in the state and a grade of “A+.” Meanwhile, SchoolDigger’s page on the same district ranks it a far less impressive 130th in Texas. Or, if you think the Nashua, New Hampshire school district is an above average performer for the Granite State, you’ll tend to agree with Niche, that puts it at #26 out of the 70 districts it reviewed, and a letter grade of ‘B.’ Nashua district haters, however, will side more with SchoolDigger, which ranks the same district well below average, at 111 out of the 155 it’s reviewed and a rating of just one lonely star.

Turning from districts to schools, while the gentleman’s “C” that Niche gives to George Washington High of Ogden, Utah doesn’t sound great, that conveys quite a different impression than you’d get from SchoolDigger, which rates Washington the single worst high school of the 125 it’s reviewed in all of Utah. Or if you want information about Granite City High, in Granite City, Illinois outside St. Louis, (which sounds like it could have been Fred Flintstone’s alma mater), Niche’s grade of “B-” doesn’t seem so bad. Meanwhile, both GreatSchools and SchoolDigger give it terrible ratings, 1/10 and one star out of five, respectively.

The business of comparing the school analysis websites, or judging the judges, gets tricky quickly. Start with the fact that Niche only publishes the numerical state rankings for “B”-or-better-rated schools. In other words, Niche doesn’t release the numerical ranks for any schools it considers “B-” or below, which is half of them. Add to that, Niche’s methodology may reflect a degree of letter grade inflation that not every reader realizes. Ask yourself, what letter grade would you assign to an absolutely dreadful school that you believe performs all the way down at the 3rd percentile (out of 100)? Niche calls that school a “C-.” (Niche reserves the letter grades “D+” and below for only the worst 2.5%.)

Further, in layman’s terms, GreatSchools rates “on a curve” for each state, while Niche says its ratings conform to a single, national norm. This disparity alone could present a confusing scenario for parents moving to a new state, or moving to a metropolitan area that spills across state lines. It means that among two theoretically identical schools in different states, the school located in the higher-scoring state would be more likely to get a better rating from Niche, whereas the one in the lower-scoring state would be more likely to get kinder treatment from GreatSchools.

Niche’s single national standard for school ratings allows their aggregated grades to be used as cross-state comparisons, which are always interesting to peruse. Even within Niche’s particular evaluation paradigm, however, different choices about factor weighting or cut scores can produce very different results. To demonstrate this point, Choice Media obtained and aggregated Niche’s school ratings data to produce such state rankings, shown in the embedded spreadsheet below, alongside figures from the 2017 Nation’s Report Card (NAEP). Try clicking on different column headings to reveal different state rankings. For example three states, Pennsylvania, Arizona and New Hampshire, appear in the top ten when ranked by the percentage of schools with a Niche “A” rating or better, but don’t even make the top 20 if the criterion changes to percentage of schools getting a “B” rating or better.

The growth of school ratings websites is undoubtedly healthy, providing parents far more insight into the heretofore secret world of education than had previously been possible. But in the final analysis, even the most carefully calibrated school quality assessment that somehow exactly matches a particular family’s values will fail to predict how an unusually great new teacher or specific pack of bullies could affect a particular child. In short, statistical data are great, and we need more, even though they will never be enough.

 

State Rankings for School Quality Based on Niche Ratings

StatePercentage of Schools Graded ‘A’ or Better by NichePercentage of Schools Graded ‘B’ or Better by Niche2017 NAEP: 8th Grade Math2017 NAEP: 8th Grade English
NJ20.71%59.0%292275
CT18.53%57.0%284273
MA17.81%61.4%297278
VA17.73%64.5%290268
TX16.86%57.4%282260
NY16.64%51.1%282264
PA15.30%49.7%286270
MD14.84%54.8%281267
AZ12.17%49.7%282263
NH12.17%49.1%293275
MO11.92%47.6%281266
NE11.76%59.8%288269
CA11.66%45.8%277263
VT11.58%48.1%288273
OH11.34%47.5%288268
DC11.11%39.7%266247
DE10.77%54.4%278263
GA10.75%46.3%281266
IL10.73%45.3%282267
WA10.59%49.5%289272
IN10.14%49.6%288272
MI9.76%42.1%280265
IA9.48%53.0%286268
MN9.44%55.1%294269
WI9.32%53.6%288269
KS9.25%49.5%285267
OK9.06%50.1%275261
CO8.31%48.3%286270
SC8.28%51.4%275260
RI8.10%47.5%277266
LA7.98%52.0%267257
NC7.57%49.8%282263
AL7.02%43.4%268258
MS6.73%42.4%271256
ME6.62%42.4%284269
AR6.32%47.0%274260
FL5.98%48.7%279267
KY5.80%45.1%278265
TN5.76%41.0%279262
AK5.26%60.5%277258
UT4.52%51.2%287269
OR4.49%38.5%282266
WY4.21%49.4%289269
ND3.72%44.7%288265
ID3.46%41.3%284270
MT3.32%51.8%286267
NM3.23%35.6%269256
NV3.01%39.1%275260
SD2.84%37.4%286267
WV1.66%28.5%273259
HI0.72%43.0%277261

Bob Bowdon

Bob Bowdon has been a television producer, reporter, political commentator, documentary filmmaker and satirist. He is currently the Executive Director of Choice Media, an education news service, website and smartphone app.

Pin It on Pinterest

Share This
Close