Grannan: Where have all the KIPPsters gone?

Guest post by Caroline Grannan.

“[Charter schools] have a distinct advantage … Their families have already chosen to be at a charter and have often jumped through numerous hoops to get there. This makes it easier for charters to create their own cultures. They can define the length of their days, dictate exactly how children dress and enforce strict codes of conduct. Those students — scholars, in charter parlance — who fall out of line don’t last.”

– Jonathan Mahler in the New York Times Magazine

“I will never follow the lead of those who exclude the kids who need education the most so that their precious scores will rise.”

– John Kuhn, superintendent of Perrin-Whitt Consolidated Independent School District, Perrin, Texas

The highly acclaimed charter school operator KIPP – the Knowledge is Power Program – wins widespread praise for the overall high achievement of the low-income students it serves. It's no wonder that KIPP's practices are watched closely. And that means asking an obvious question: Why do so many KIPP students leave the schools without being replaced, and how does that affect the schools' achievement?

Why is attrition at KIPP schools an issue?

Several studies show that a high number of students who enroll at KIPP schools leave the schools early, and the numbers show that students who leave aren't replaced by new, incoming students. At public schools that serve comparable demographics, students who leave are replaced by new, incoming students.

A 2008 study of San Francisco Bay Area KIPP schools by SRI International found that it's consistently the lower-performing students who leave.

How does this affect KIPP's achievement? The lower-performing students are no longer there to bring down averages – but it would also be valuable to learn more about the impact on the students who don't leave after the lower performers have departed. Do they learn more and achieve more, unfettered by their less successful former classmates? This is difficult to address, since the topic is so often met with denial and distortion. Would those students do as well at public schools if their less successful classmates left and weren't replaced?

KIPP schools' achievement is regularly compared to public schools' achievement (i), but the attrition question confounds those comparisons.

Richard Kahlenberg, senior fellow at the New Century Foundation, wrote: “The issue is important because if large numbers of weaker students drop out of KIPP’s rigorous program, it would be highly unfair to compare the test score gains won by the top KIPP students against the scores of all regular public school students – who include KIPP dropouts.” Unfairness is only one issue. If we want to look at KIPP schools to see how they often achieve academic success, the high attrition confounds efforts to do that.

We frequently hear that KIPP schools have waiting lists, but the attrition puts that supposed situation in a whole different light. If there are waiting lists, why aren't all the departing students immediately replaced?

The movie “Waiting for 'Superman',” a PR tool for charter schools, presented the story of Daisy, who hoped to attend KIPP Los Angeles College Preparatory Academy and was devastated to lose out in the admission lottery. But in real life, the high attrition at KIPP LA Prep means Daisy shouldn't have to wait long for an opening. For example, KIPP LA Prep's most recent 8th-grade cohort lost a third of the students who started in grade 5 by the beginning of 8th grade – figures aren't publicly available for how many students finished 8th grade. For that class, which started grade 5 in the 2006-'07 school year and finished grade 8 in 2010, the number dropped from 97 students at the beginning of 5th grade to 81 by the beginning of 6th grade to 65 by the beginning of 8th grade.

Based on that attrition rate, the school would have room for Daisy and many more hopeful applicants if the administrators filled those spots from the waiting list. Why isn't that happening? It's a mystery.

Does this attrition happen at all KIPP schools?

The available research doesn't provide that information.

• In early 2007, I researched attrition at KIPP's California schools as a volunteer project, using data publicly available on the California Department of Education website. At the time, KIPP had nine schools in California. My research found very high attrition at six of them. I also broke down the attrition by demographic subgroup. At all six of the schools with high attrition, the attrition was much higher in the subgroup that's statistically likely to be the most academically challenged – either African-American males or Latino males, depending on the school. My blog posts about the issue appear to mark the first time KIPP attrition had been publicly discussed. What my findings refer to is the overall drop in the number of students – total and by subgroup – year by year in a grade cohort.
• In fall 2008, the organization SRI International released a study of the five KIPP schools that existed at the time in the San Francisco Bay Area. SRI used data that went deeper than the publicly available data I'd used, and found high attrition at all five Bay Area KIPP schools. SRI reported that overall, 60% of the students who enrolled at the five KIPP schools didn't finish at those schools. SRI also found that the students who left were consistently the lower-performing students. If SRI's research broke the students down by demographic subgroup, that wasn't included in the final report.

(Interestingly, SRI International found the high attrition pattern even at a KIPP school – KIPP Heartwood in San Jose – that didn't show high attrition in the figures I researched. As noted, SRI International had access to more complete data than I did; I used only publicly available statistics.)

• KIPP supporters responded to my research and to the SRI study with this claim: “The San Francisco KIPP schools are outliers.” But that was an invalid, misleading and inapplicable response. I had researched all the KIPP schools in California (nine at the time), and SRI had researched all the KIPP schools in the Bay Area (five at the time) – not just the two San Francisco KIPP schools. The two San Francisco KIPP schools, KIPP San Francisco Bay Academy and KIPP Bayview Academy, did show high attrition, but not the highest among the Bay Area's or the state's KIPP schools. It wasn't true that the San Francisco KIPP schools were outliers, and that claim didn't negate my findings or SRI's findings. It didn't even make sense in context of those findings.
• A 2010 study of KIPP schools by Mathematica Policy Research was framed in a manner intended to refute reports of high attrition at KIPP. The report found that of the KIPP schools studied, one-third showed lower attrition than comparable public schools, one-third showed higher attrition and the rest showed comparable attrition. However, the Mathematica report failed to address the issue of whether KIPP replaces the students who leave, which leaves a giant gap in the report.
• A 2011 study by Western Michigan University researchers, published jointly with Columbia University, found significantly higher attrition at KIPP schools than at comparable public schools – 15 percent attrition per year at KIPP vs. 3 percent at public schools. The study found that 30 percent of KIPP students leave between 6th and 8th grades.

But many KIPP supporters say that public schools have the same high rates of attrition, and the Mathematica study said that too.

That's not valid because of the critical difference that at public schools, the students who leave are replaced by new, incoming students. The numbers show that at KIPP schools, students who leave are not replaced. The cohort of students simply shrinks drastically. Even though Mathematica did indeed make that statement, it's not valid. It flies in the face of logic. The distinction puts the situations in an entirely different light.

To give a clear picture: Let's say 100 students start 5th grade at a KIPP school that serves grades 5-8. Sixty of the students leave along the way, before completing 8th grade, and those are the lowest-performing 60 students. They aren't replaced with incoming students. So that cohort at the KIPP school winds up with only the 40 highest-performing students.

At the public school down the street, 100 students start. Sixty of them leave along the way, but each time one leaves, a new student arrives to replace him or her. So that cohort at the public school consists of 100 students from start to finish. Clearly, those two situations are not parallel, equivalent or comparable.

Low-income, at-risk students are likely to be “high-mobility” – meaning that they move a lot due to the instability that tends to afflict the lives of impoverished families. Those students are also statistically likely to be low academic achievers. With the high mobility that characterizes low-income communities, if if a high-mobility student leaves a public school, he or she is replaced with a similarly high-mobility student. By contrast, if a high-mobility student leaves a KIPP school, the numbers show that KIPP is usually not replacing him or her with an incoming student.

Why do the students leave KIPP schools?

That's not publicly known. KIPP spokespeople and supporters deny that KIPP expels or “counsels out” (ii) low-performing students.

Does KIPP have a policy of not enrolling new students after the starting grade? How do we know KIPP isn't replacing the students who leave?

KIPP doesn't appear to have an actual policy of not enrolling new students. But the enrollment numbers at the KIPP schools studied show that the students who leave are, overall, not being replaced with incoming students. It's not clear why that is, given the widespread reports of “long waiting lists.” Even if some of the students who leave are replaced with incoming students, the numbers still show that a very high number are not, resulting in very high total overall attrition and significant shrinkage of the grade cohorts.

If schools receive state funding based on the number of students, doesn't the attrition mean that KIPP schools lose funding as students transfer to other schools?

Yes, it must mean that, though there's discussion of whether the students leave after they are counted for the year, so that KIPP still receives the funding. (This situation could vary state by state.) The KIPP organization receives an immense amount of private philanthropical funding, which may provide enough of a cushion against the loss of the public funding. Perhaps the tradeoff – losing the less-successful students and also losing the per-student funding – is worth it to KIPP schools.

But there's a high dropout rate in public schools that serve low-income populations, so how can we say that public schools replace the students who leave?

Most KIPP schools are middle schools, serving grades 5 through 8. (Almost all of the KIPP schools that have existed long enough for their attrition to be tracked are middle schools.) Except for a very small number of extreme, problematic outliers, students don't drop out of middle school, so public schools don't suffer from dropout rates at those grade levels. Students who leave KIPP schools would be transferring to other schools. I did comparisons of grade cohorts in demographically comparable public schools, and those schools simply didn't show a pattern of attrition at all.

Why would Mathematica make the misleading statement that attrition was comparable to public schools?

Research organizations are known to negotiate with the funders of the research about just how the findings will be presented. It is not publicly knowable what kind of negotiations went on with Mathematica, whose report was funded by KIPP.

The 2008 SRI International report – the one that found 60 percent attrition at all the Bay Area KIPP schools and reported that it was the lower achievers who left – presented that finding as a secondary one in its report. The report, which was also funded by KIPP, announced KIPP's high achievement as the primary finding. The attrition was the newsworthy finding and the one that is still extensively – and increasingly – discussed today.

Some reports have said it might not be that students are leaving KIPP schools but that they're being required to repeat a grade.

That might be true in some – or many – cases. But the overall numbers still show the grade cohorts shrinking, so many students are clearly leaving the schools.

So what's the conclusion?

KIPP attrition isn't comparable to the flow of high-mobility students in and out of public schools – it simply isn't, no matter how many claims there are to the contrary. Why it happens and what it means are still hard to pin down.

i The usual question is: “But aren't KIPP schools public schools?” I don't believe that schools run by a private operator are truly public schools, even if they receive public funding.

ii “Counseling out” refers to a practice of gently but firmly persuading the student and family to leave the school. The archetypal method would be to tell the family that the school isn't the “right fit” for the student, or vice versa.