Jason Bedrick, Policy Analyst at the Cato Institute’s Center for Educational Freedom, and John White, Superintendent of Louisiana Public Schools, have held an online exchange on the appropriate level of regulation for private schools.
Update: Click here to read the latest entry in the debate.
Despite a few setbacks, 2015 was a great year for educational choice. By the time the ball dropped in Times Square, 15 states had adopted 21 new or expanded educational choice policies, and courts rejected challenges to choice laws in Alabama, Florida, Louisiana, and North Carolina.
However, the new year brought sobering news for choice supporters. For the first time ever, a random-assignment study found a negative result from a school voucher program.
In a study published by the National Bureau of Economic Research, researchers Atila Abdulkadiroglu of Duke University, Parag Pathak of MIT, and Christopher Walters of the University of California at Berkeley found that students who received a voucher through the Louisiana Scholarship Program (LSP) during the 2012-13 school year were 50 percent more likely to receive a failing score on the state math test than students who applied for but did not receive a voucher. Outcomes for voucher students in reading, science, and social studies were also negative. The researchers report that the “negative impacts of vouchers are consistent across income groups, geographic areas, and private school characteristics, and are larger for younger children.”
Until now, random-assignment studies have almost unanimously found positive impacts for voucher recipients generally or particular subgroups, especially those that had been the most choice deprived (e.g., low-income minorities). Out of a dozen random-assignment studies, 11 found positive impacts from vouchers and just one failed to find any statistically significant impact. Many of the positive effect sizes were modest, but some were considerable and, until now, none had found any harm.
So why were the results from Louisiana’s voucher program so different? Although not conclusive, there is considerable evidence that the problem stemmed from poor program design. Regulations intended to guarantee quality might well have had the opposite effect. The LSP’s high level of private-school regulation appears to have driven away better schools while attracting primarily lower-performing schools with declining enrollments that were desperate for more funding…Read More
January 12, 2016
In a recent blog post, Jason Bedrick of the Cato Institute attributes the apparently troubling results of a recent study on Louisiana’s private school voucher program to the theory that “[r]egulations intended to guarantee quality might well have had the opposite effect. The high level of private school regulation appears to have driven away better schools.”
As the head of the regulatory agency for traditional public, charter public, and non-public schools in Louisiana, I think it’s important to discuss the facts behind the study, as they raise questions about the conclusions reached by both the researchers and Mr. Bedrick.
More important, however, is the larger implication I take from Mr. Bedrick’s thesis: that private school choice advocates in America, Mr. Bedrick among them, have failed to establish a coherent, prevailing belief system about the role of private schools in providing an education of measured quality, at scale, for the nation’s most disadvantaged youth. I’ll spend most of this post on that subject.
First, the facts.
Mr. Bedrick is right that a study from the National Bureau of Economic Research showed very low performance among students in Louisiana’s voucher program compared to the performance of students not offered a voucher (who thus remained in a local public school).
Not highlighted in the post is that the study is based on one year’s data—and that the researchers were offered two additional years of results, but turned them down because of urgency to meet their own deadlines for publishing.
Because of the study’s design, the research within this one year’s data only included students in their first year enrolled in the state’s voucher program…Read More
JASON BEDRICK’S RESPONSE TO JOHN WHITE
Last week, NBER released the first random-assignment study ever to find a negative impact from a school voucher program. Previous gold standard studies had almost unanimously found modest positive effects from school choice, which raises the obvious question: what makes the Louisiana Scholarship Program (LSP) so different?
In an article for Education Next, I argued that, “although not conclusive, there is considerable evidence that the problem stemmed from poor program design.” The LSP is one of the most heavily regulated school choice programs in the nation, and that burden has led to a very low rate of private school participation. Only about one-third of Louisiana private schools accept voucher students, a considerably lower rate than in most other states. From a survey of private school leaders conducted by Brian Kisida, Patrick J. Wolf, and Evan Rhinesmith for the American Enterprise Institute, we know that the primary reason private schools opted out of the voucher program was their concerns over the regulatory burden, particularly those regulations that threatened their character and identity. For example, voucher-accepting schools in Louisiana may not set their own admissions criteria, cannot charge families more than the value of the voucher (a meager $5,311 on average in 2012), and must administer the state test.
We also know from the NBER study that the participating and non-participating private schools differ in at least one important respect. Whereas the non-participating schools experienced modest growth over the decade before the voucher program was expanded statewide (about 3 percent, on average), the participating schools had been experiencing a significant decline in enrollment (about 13 percent, on average). In other words, schools that were able to attract students tended to reject the vouchers while voucher schools tended to be those where enrollment had been dropping.
The difference in enrollment trends suggests that the LSP’s regulatory burden had the opposite of its intended effect: discouraging higher-performing schools from participating, leaving only the lower-performing schools that were so desperate to reverse their declining enrollment and increase their funding that they were willing to do whatever the voucher program required.
Several researchers and education reform advocates reached similar conclusions, including Matthew Ladner, Adam Peshek, Michael McShane, Lindsey Burke, and Jonathan Butcher. However, others expressed skepticism about what I shall call the Overregulation Theory, and proposed alternative explanations for the LSP’s poor results, while a few more raised concerns about the impact of a more free-market school choice system on equity. I will address these critiques and concerns below.
Alternative Explanations to the Overregulation Theory
Writing at Education Week, Douglas Harris of the Education Research Alliance for New Orleans concedes that “regulation probably does reduce the number of private schools, especially the number of higher-performing private schools,” but he still believes the Overregulation Theory is “premature.” Harris instead offers two potential alternatives: 1) the improved public/charter school performance in New Orleans made the performance of the private sector look relatively worse; and 2) the curriculum at most private schools may not have been aligned to the state test, so the poor performance merely reflects that lack of alignment rather than poor performance.
Harris’s first theory is explicitly rejected by the NBER study. On the third page of the study, the authors write: “Negative voucher effects are not explained by the quality of public fallback options for LSP applicants: achievement levels at public schools attended by students lotteried out of the program are below the Louisiana average and comparable to scores in low- performing districts like New Orleans.” In other words, the public school alternatives are not so great and the performance of the participating private schools is considerably worse.
That said, Harris’s second theory, which Jason Richwine also suggested, is plausible as a contributing factor. However, it is no more plausible than the Overregulation Theory. Indeed, whereas the differences in enrollment trends between voucher and non-voucher private schools provide some suggestive evidence for the Overregulation Theory, Harris provides no evidence to support the Nonaligned Test Theory. How many voucher schools were already aligned with the state curriculum and/or administered the state test? At this point, we do not know. Moreover, to the extent that testing nonalignment explains some of the very large 0.4 standard deviation difference in math scores, it is unlikely that it explains all or even most of that difference. Then again, Harris stated that he will be releasing the results of his own research on the LSP, so it’s likely he knows something that I do not.
In addition to test non-alignment issues, Louisiana state superintendent of schools John White seems to imply that the negative impact found in the NBER study was a result of the “transition cost” of students switching to new schools. White writes: “these schools (a) had never worked with these students before and (b) had never been accountable for state test results before.” Certainly, transition costs play a role, but they cannot account for the entire negative impact, which was quite large. Previous research has found that the negative impact of student mobility was only a fraction of the negative impact of Louisiana’s voucher program.
Moreover, if they are correct that test non-alignment and transition costs explain a great deal of the negative results, then merely growing accustomed to the new schools and testing regimen—and not actual improvements in learning—should account for a great deal of any improvement on the state test in the first few years.
Limits to the Overregulation Theory
Harris and White also note that the NBER study only examined the results of one year of one program. They are certainly correct that we need more data over time to draw firmer conclusions, which is one reason I presented my interpretation as “not conclusive” and wrote that “the regulations may have had the opposite of their intended effect” (emphasis added). And, indeed, there is some evidence that voucher schools improved slightly in the third year since the statewide expansion. However, while White touts that, if the voucher schools were their own district, they’d rank 9th for performance improvement, they’d still be the fifth-worst of 76 in the state in actual performance.
Professor Joshua Goodman of Harvard University also offered a salient critique, pointing out that the NBER study’s “estimates show consistent negative impacts irrespective of previous enrollment trends.” If there had been wide variation in enrollment trends among the participating schools, that would count as evidence against the Overregulation Theory. However, if the enrollment trends among participating schools were consistently negative, with the variation mostly limited to how fast the participating schools had been losing students, then this finding would still be consistent with the Overregulation Theory.
In other words, if the participating schools are actually roughly the bottom third in terms of performance, then the lack of correlation between their enrollment trends and performance may not tell us anything about the relationship between the enrollment trends and performance of the top two-thirds of private schools that chose not to participate.
As I noted initially, the evidence for the Overregulation Theory is suggestive but not conclusive. Nevertheless, such strongly negative results should give education reform advocates great pause about the regulatory strategies employed in Louisiana. We know the regulatory burden chased away most private schools, and we have evidence that the voucher-accepting schools had been struggling with declining enrollment. If we want to better understand the LSP’s atypically disastrous performance, the program design is the logical place to start.
School Choice: Equity and Economics
Whereas Harris, Richwine, and Goodman expressed skepticism about the explanatory power of the Overregulation Theory, Neerav Kingsland expressed misgivings about its implications. (White did both.) Noting that I had raised concerns about the unintended consequences of prohibiting voucher schools from setting their own admissions criteria or charging more than the voucher, Kingsland wrote:
The logical extension of Jason’s argument is that an all voucher education system would lead to a public education system where all schools would be allowed to reject students based on wealth, academic performance, and behavior.
Is this right?
Either voucher proponents have very different views of equity than most citizens, or they don’t really view vouchers as a replacement model for the current public education.
I’m curious – which is it?
First, I should clarify that I do not believe we can know from the NBER study the extent to which any single regulation—whether the ones Kingsland listed, the mandatory state test, or others—is to blame for the LSP’s poor performance. The Overregulation Theory only suggests that the whole package of regulations in Louisiana is driving away the better private schools.
That said, there is reason to believe that each of the regulations I enumerated produces some adverse consequences.
Mandating a single test makes it easier for parents to compare performance across schools, but it also stifles diversity and innovation because tests drive what is taught, when it’s taught, and how it’s taught. If Louisiana policymakers want to ensure parents have access to valuable test data without the significant downsides of a single test, a better solution would be requiring schools to administer one of the many nationally norm-referenced exams.
Tuition and the Role of Price
The prohibition against schools charging more than the value of the voucher is intended to ensure that low-income students are not turned away due to inability to pay. However, it can have the effect of reducing access to quality schools for low-income families. For example, if the state offers a $5,000 voucher and a school charges $7,000 in tuition, some low-income families will be able to make up the $2,000 difference (whether on their own or through additional scholarships or tuition aid), although some might be not be able to afford that school. However, if the school refuses to participate in the voucher program, then even students in the first category will not have access to that school.
It’s basic economics: prices provide valuable information and price ceilings cause shortages. If we truly want all children to have access to quality schools, it is imperative that the quality schools have an incentive to expand, which means they need to be able to cover more than just the marginal cost of an additional student. However, particularly if the voucher value is relatively small, price ceilings mean that private schools will likely only fill empty seats rather than expand enrollment. As I noted in my original post, the Wolf/Kisida/Rhinesmith survey found that two-thirds of participating Louisiana private schools intended to maintain their level of voucher enrollment and 13 percent planned to decrease voucher enrollment, while only 22 percent planned to increase it. This is not a recipe for universal access.
Mandating open admissions, like the prohibition against “topping off,” is intended to ensure all children have access to quality schools. And, similarly, it may have the opposite effect. The Wolf/Kisida/Rhinesmith survey found that most private schools that opted not to accept voucher students were very concerned about threats to their “independence, character, or identity.” These concerns may be religious in nature or relate to student behavior, motivation, or other aspects of school culture. I believe these are legitimate concerns, but even if some people disagree, all that really matters is that the schools have these concerns. If most schools aren’t willing to participate then we haven’t done much to expand access.
Now, Kingsland asked about an all-voucher education system, and of course the calculus in that situation would be different than in a voucher system that runs in tandem with a district school system. In the present system, students always have access to a school that must accept them. That would not necessarily be the case in an entirely privatized system. In that case, the argument for mandating open admissions is much more compelling, but it is still not necessary. The government could, for example, directly subsidize schools that adopted an open admissions policy or grant larger vouchers to students attending such schools, while offering smaller vouchers to students attending more selective schools. This or similar approaches (e.g., Kingsland’s proposal to grant larger vouchers for at-risk students) are more likely to yield wider private school participation—and therefore greater access to quality schools—than a strict open admissions mandate.
I outlined above how several regulations intended to guarantee equity could produce adverse effects. It does not automatically follow that they are not worth doing. Perhaps, in some cases, the benefits of a particular regulation outweigh the costs. This is a legitimate debate that extends back at least to Milton Friedman’s dispute with John E. Coons and Stephen Sugarman over their competing voucher proposals in California in the 1970s.
Proponents of particular regulations should make the case that the costs are worth it to achieve some desired end, and both sides should be willing to reconsider based on compelling evidence. However, dogmatically refusing to acknowledge that regulations can have real costs or that Louisiana’s voucher program has (perhaps uniquely among choice programs) failed to deliver on its promises does nothing to serve the interests of disadvantaged children.
So what is to be done? Do policymakers throw up their hands and accept inequity? Of course not. But we must balance idealism with a clear-eyed approach to evidence and humility about what can be accomplished. Policymakers should strive for a system in which every child has access to the education that best meets his or her needs. But they should also be cognizant that intentions do not guarantee results, that there are limits to what any policy can achieve, and that a perfect system is not possible. The perfect should not become the enemy of the good.
— Jason Bedrick