In an era of increasing school choice, websites like GreatSchools, Niche and SchoolDigger rate thousands of schools in all 50 states to help parents choose the best public and private schools for their children. But just as different families apply different criteria for their educational choices, a Choice Media investigation reveals disparities in the sites’ results.
Take the school district of Plano, Texas in the Dallas/Fort Worth area. Niche gives it a superb ranking of #10 out of the 995 districts for which it has figures in the state and a grade of “A+.” Meanwhile, SchoolDigger’s page on the same district ranks it a far less impressive 130th in Texas. Or, if you think the Nashua, New Hampshire school district is an above average performer for the Granite State, you’ll tend to agree with Niche, that puts it at #26 out of the 70 districts it reviewed, and a letter grade of ‘B.’ Nashua district haters, however, will side more with SchoolDigger, which ranks the same district well below average, at 111 out of the 155 it’s reviewed and a rating of just one lonely star.
Turning from districts to schools, while the gentleman’s “C” that Niche gives to George Washington High of Ogden, Utah doesn’t sound great, that conveys quite a different impression than you’d get from SchoolDigger, which rates Washington the single worst high school of the 125 it’s reviewed in all of Utah. Or if you want information about Granite City High, in Granite City, Illinois outside St. Louis, (which sounds like it could have been Fred Flintstone’s alma mater), Niche’s grade of “B-” doesn’t seem so bad. Meanwhile, both GreatSchools and SchoolDigger give it terrible ratings, 1/10 and one star out of five, respectively.
The business of comparing the school analysis websites, or judging the judges, gets tricky quickly. Start with the fact that Niche only publishes the numerical state rankings for “B”-or-better-rated schools. In other words, Niche doesn’t release the numerical ranks for any schools it considers “B-” or below, which is half of them. Add to that, Niche’s methodology may reflect a degree of letter grade inflation that not every reader realizes. Ask yourself, what letter grade would you assign to an absolutely dreadful school that you believe performs all the way down at the 3rd percentile (out of 100)? Niche calls that school a “C-.” (Niche reserves the letter grades “D+” and below for only the worst 2.5%.)
Further, in layman’s terms, GreatSchools rates “on a curve” for each state, while Niche says its ratings conform to a single, national norm. This disparity alone could present a confusing scenario for parents moving to a new state, or moving to a metropolitan area that spills across state lines. It means that among two theoretically identical schools in different states, the school located in the higher-scoring state would be more likely to get a better rating from Niche, whereas the one in the lower-scoring state would be more likely to get kinder treatment from GreatSchools.
Niche’s single national standard for school ratings allows their aggregated grades to be used as cross-state comparisons, which are always interesting to peruse. Even within Niche’s particular evaluation paradigm, however, different choices about factor weighting or cut scores can produce very different results. To demonstrate this point, Choice Media obtained and aggregated Niche’s school ratings data to produce such state rankings, shown in the embedded spreadsheet below, alongside figures from the 2017 Nation’s Report Card (NAEP). Try clicking on different column headings to reveal different state rankings. For example three states, Pennsylvania, Arizona and New Hampshire, appear in the top ten when ranked by the percentage of schools with a Niche “A” rating or better, but don’t even make the top 20 if the criterion changes to percentage of schools getting a “B” rating or better.
The growth of school ratings websites is undoubtedly healthy, providing parents far more insight into the heretofore secret world of education than had previously been possible. But in the final analysis, even the most carefully calibrated school quality assessment that somehow exactly matches a particular family’s values will fail to predict how an unusually great new teacher or specific pack of bullies could affect a particular child. In short, statistical data are great, and we need more, even though they will never be enough.
[table id=6 /]