A data analyst who worked for the past several years in the New
York City Department of Education wrote the following about Eva
Moskowitz’s Success Academy charter chain. Most of the data he cites
comes from public records maintained by the city or state education
departments. His footnotes are at the bottom of the post.
Building a Charter Chain, and a Mayoral Campaign, on Lies
Over the past few days a deluge of, what for lack of a better word
can only be described as Success Academy propaganda material, has
appeared in the New York City media. The New York Daily News published
an opinion piece written by what they termed “a researcher” and a
“graduate student” that used the veneer of data to argue that Success
Academy is a true success and miracle story.[1] Careful analysis reveals
that many of the claims are outrights lies and the rest are
half-truths.
Let’s start digging in.
Claim #1: “Success Academy schools serve a similar share of special
needs students relative to their zoned counterparts.” This is a lie.
According to the latest public data on the New York City Department of
Education’s Progress Report website [2] the 4 Success Academy schools in
Harlem had a total of 2540 students enrolled. Of these students only 17
were special education students with the highest level of need. That’s
0.6% of their students. By contrast the average percent of special
education students with the highest level of need at community
elementary/middle schools throughout New York City was 9.4% and 14% in
Harlem. The average community school in NYC serves 1,500% MORE of the
highest need special education students than Success Academy.[3] Such a
vast disparity, in what to a significant extent are disabilities based
on neurological, medical and physical differences, can’t be explained by
anything other than Success Academy not serving the neediest students
in the community.[4] You might think it would be hard for the
researchers who penned the propaganda in the Daily News to top this lie.
They will manage to surprise you.
Claim #2: “the school is more successful in teaching students
English…27% of Success Academy students passed the ELL writing exam,
compared to 19% at nearby schools.” This is a lie. The school is not
successful at teaching English Language Learners. The “researchers”
somehow forgot to mention that Success Academy only serves ELLs who are
already amazingly good at English.
According to the New York State data
[5] over 90% of the ELL students at Success Academy are proficient or
advanced in kindergarten (that is before Success Academy would have had
much of an effect). Seems that Success Academy only serves English
Language Learners who already know English. There ARE data suggesting
that Success Academy IS very successful at kicking ELLs out (perhaps the
few intermediate level ELL students who manage to slip through the
lottery).[6]
Claim #3: “Success Academy students scored on average 39 points…
ahead of others from equivalent backgrounds.” This is a lie. It is only
by misrepresenting the “equivalent background” that they can make this
claim. As we have seen in the first two lies that were exposed above,
these “researchers” have not even come close to controlling for
“equivalent backgrounds.” They have not controlled for level of special
education need. They have not controlled for English Language Learner
performance levels. They have not controlled for parent characteristics.
They have not controlled for home environment. They have not controlled
for peer effects of creaming some of the most advantaged students in
each neighborhood.[7]
Claim #4: “the overall rate of attrition at Success Academy is far
from alarming.” This is a lie. The “researchers” make this claim by
comparing the yearly rate of attrition at Success Academy to that of
other, nearby community schools. But that is an absurd comparison to
make. After all, the data show that Success Academy is serving some of
the most economically, linguistically, and academically privileged
students in Harlem.[8] On top of that parents must actively choose to
enter a lottery to get into Success Academy and must put up with Ms.
Moskowitz’s hazing.[9] The fact that even after such a thorough
screening process every single year, year after year, another 10% of the
student body leaves IS rather disturbing.
Claim #5: They dismiss as a “side argument” the notion that “the
choice not to backfill drives up scores.” It is a moral imperative to
point out that only charter schools are given this “choice.” Public
schools serve all students, from all circumstances, at all times (even
when charter schools kick those very students out right after “census
day,” the day districts use to calculate enrollment for budget
purposes). It is disturbing that this difference seems to carry such
little weight with so-called education reformers. It can only make one
wonder how invested they really are in the success of each and every
student.
It is also disturbing that these researchers seem unable to use basic
logic and arithmetic. An Independent Budget Office report showed that
charters are more likely to lose the students who score poorly on the
New York State exams and who are more often absent.[10] Losing 30% of
the students who will test poorly before they enter the first testing
grade (i.e. 10% attrition each year through 3rd grade) can have a huge
effect on test outcomes. Since it is reasonable to assume that Success
Academy replaces those students with ones more likely to do well on the
exams, the whole Success Academy effect can be explained by
attrition.[11]
Let’s use the data from Harlem Success Academy 1 as an illustration.
This year’s 8th grade cohort, the one that started kindergarten in the
2006-07 school year, dropped from 83 students in kindergarten to 63
students in 3rd grade. This means that at least 25% of the cohort
disappeared even before the first exam.[12] As the disappearing students
are the ones least likely to do well on the exams that means that in
3rd grade these students can be expected to score proficient on the New
York State exams at a rate that would be about 25% higher than would be
expected based solely on other factors (such as the creaming and
self-selection noted above). This effect fully accounts for Success
Academy’s, now obviously banal, outcomes.
Claim #6: “growth data from 2013 suggests that in the upper grades,
on average, students maintain their high early achievement rather than
moving further ahead.” Now this one is interesting since the researchers
are more or less admitting that a Success Academy education amounts to
very little. If students are not growing more the more time they are
spending in these schools, what exactly is Success Academy
accomplishing? This little fact shows that the rest of their essay
amounts to little more than apologetics and lies.
It is implausible to
assume that Success Academy accomplishes magic in the early grades and
then barely manages to hold ground in upper grades. In fact, this is
another piece of evidence suggesting that Success Academy amounts to
little more than an accounting trick.
However even this is a half-truth since the most recent (2012-13) New
York City Department of Education Progress Report data show that
Success Academy lagged well behind its “peer” schools in English exam
growth.[13] Success Academy scored in the 39th percentile on English
exam growth for their overall student population and in the 21st
percentile on English exam growth for the students who began with scores
in the lowest 1/3 of students citywide.
Claim #7: “The implication is that, through “drill and kill”
instructional techniques, Success Academy is teaching students only
low-level skills…The reality is much different.” Here the “researchers”
mislead in various ways. They state that “Success Academy students get
more science instruction than their peers.” It is unclear how they know
this since no evidence is cited. They forgot to mention that according
to a Success Academy teacher “We do not teach history or foreign
languages in elementary school.[14]” The teacher also revealed that
“Test prep starts in November.” A former teacher noted that the “Entire
school focused on remaining at top of network schools assessment wise.”
[15] From another interview, “All of the other grades, besides seventh
and eighth grade, have been doing test prep since…the beginning of
November. So that means they weren’t having English class, they were
just doing stupid passages by random authors of no literary basis,
quality, and just doing multiple choice questions for the past two
months or so. [16]”
The “researchers” also somehow forgot to mention that exactly zero
Success Academy students scored well enough on the Specialized High
School Admission Exam to be admitted to one the city’s top high
schools.[17] This forgotten piece of data supports the teachers’ claims
that a very narrow sort of test prep characterizes Success Academy
curriculum, as little of it appears to generalize to other exams.
Conclusion: Given all the other tricks in the Success Academy
playbook including suspension rates 2-3 times the district averages
[18], teacher attrition rates approaching 75% a year [19], the ability
to spend thousands of more dollars per student thanks to deep pocketed
ideological foes of public schools [20], the ability to grade its own
exams (public schools are legally prohibited from doing so), and lots
more instructional time, Success Academy must be considered an utter
failure.
The media, as seen in the “puff piece” in the New York Times Magazine
[21], is clearly getting behind Eva Moskowitz’s planned mayoral
campaign. However, the accumulating lies cannot withstand scrutiny. Eva
has fought audits of her schools and has refused to be transparent with
Success Academy data. These are not the characteristics that we want in
an elected official. Nor in a leader of schools.
[1]
http://www.nydailynews.com/opinion/results-real-article-1.1929656
[2]
http://schools.nyc.gov/Accountability/tools/report/default.htm
[3] By the way, Success Academy did not do any miracles with these
students. Fewer than 20% tested proficient on the NYS English Exam.
[4] Not that that has stopped shills for the charter school industry from trying
http://nepc.colorado.edu/files/ttr-charter-speced-crpe-mead.pdf
[5]
http://data.nysed.gov/
[6]
http://commonal.tumblr.com/post/58209601458/harlem-success-academy-charter-and-attrition
[7] They also convert a 39 point difference in test scores into being “3
years” more advanced than other students. To put this ridiculous claim
in context, let’s note that 39 points is about 12% of the average total
score for general education students in New York City on the New York
State exams. They are essentially claiming that a student who scores a
92% on an exam is 3 years more advanced than a student who scores an
80%.
[8]
http://andreagabor.com/2014/05/09/a-demographic-divide-in-east-harlem-the-neediest-kids-go-to-public-schools-not-charters/
[9] Here is one parent’s description on insideschools.org: “I attended
the orientation & was “turned off” by Ms. E. Moskowitz condescending
& offensive approach. She was bordering on insulting. When
informing the parents of their obligations to attend the after-school
games/activities, she said,“All parents are expected to attend and stay
for the entire time. Don’t think you can come for a little while &
leave to go get your hair done”. “Another comment made in poor taste
was when a parent ask if there was financial asst. 4 uniforms. Ms. M’s
response was, “No.you have six weeks to save up”.”
[10]
http://www.ibo.nyc.ny.us/iboreports/2014attritioncharterpublic.pdf
“Among students in charter schools, those who remained…had higher
average scale scores…compared with those who had left for another New
York City public school.” “Absenteeism is an even greater predictor of
turnover for students in charter schools, compared with its predictive
power for students in nearby traditional public schools.”
[11] Since Success Academy refuses to transparently share its data we
are unable to determine exactly how much of a role attrition plays in
the early grades. The public data only show overall cohort sizes, so
when Success Academy loses 10% of its students a year, if it backfills
those seats in grades K-3, the cohort size appears to remain stable
through those years. As the “researchers” admit to at least the 10%
annual attrition rate we will accept their numbers. But it is also
possible that attrition is even higher in the early grades, since
“survivor bias” makes it reasonable to assume that the longer a student
has been at a Success Academy school the more likely they are to remain.
[12] See the previous endnote for an explanation of why this is likely an underestimate.
[13]
http://schools.nyc.gov/NR/rdonlyres/1550033E-3F15-4746-BD1A-DF3364721785/0/2012_2013_EMS_PR_Results_2014_04_24.xlsx
this data does not account for the selective attrition effects noted
above and is therefore a very optimistic figure for the real outcomes
within Success Academy schools
[14]
https://dianeravitch.net/2013/10/04/mole-in-success-academy-speaks/
[15]
http://www.glassdoor.com/Reviews/Success-Academy-Charter-Schools-Reviews-E381408_P5.htm
[16]
http://honestpracticum.com/exclusive-interview-a-tfa-teacher-working-at-success-academy-part-1/
[17]
http://www.nydailynews.com/new-york/education/gonzalez-success-charter-students-fail-top-city-schools-article-1.1833960
[18]
http://www.nydailynews.com/new-york/education/success-academy-fire-parents-fight-disciplinary-policy-article-1.1438753
[19]
http://www.wnyc.org/story/302768-high-teacher-turnover-at-a-success-network-school/ and
http://www.citylimits.org/news/articles/5156/why-charter-schools-have-high-teacher-turnover#.VA60YP_wvcw
[20]
http://nepc.colorado.edu/files/ttr-charter-rent_0.pdf
[21]
http://www.huffingtonpost.com/diane-ravitch/charter-schools_1_b_5781474.html
