Note: We haven't had the time to add the links on some of the studies mentioned. Will try to update but if any of you find them send them along to normsco@gmail.com.
On the web: http://www.indypendent.org/
Judge Rules in Favor of Releasing Teacher Test Scores; Data Dump Would Promote a Flawed and Cynical Method of Accountability
By Norm ScottJanuary 11, 2011 | Posted in IndyBlog | Email this article
Would you gauge the effectiveness of individual doctors by the percentage of patients who live or die under their care? Should firemen be held accountable when a building burns down? Should individual soldiers in Afghanistan be compared to each other on the basis of “success” or “failure” in controlling the Taliban in a given area?
Any effort to do so would spark a major outcry. But when it comes to teaching, there is a different standard.
Ironically, it was only six months ago that the NY State Department of Education revealed that years of test score advances by city students had turned out to be a mirage causing Bloomberg and his former Chancellor Joel Klein a good deal of embarrassment. No matter – the Mayor is ready once again to wield unreliable test scores as a political weapon and most media in this city have deliberately short memories, having all too often been active partners in attempts to eviscerate teachers.
The value-added approach is the latest attempt to undermine teachers, the teaching profession and the teacher union by measuring teachers based on the performance of their students on standardized tests from year to year.
Crucial backing for such initiatives has come from private foundations led by billionaires like Bill Gates and Eli Broad who assert that data-driven models in the private sector can be transferred to public schools.
A study by Mathematica Policy Research done for the US Department of Education showed that one-fourth of average teachers will be mistakenly identified for special rewards while one-fourth of teachers who differ from average performance by three to four months of student learning will be overlooked.
A recent study by Sean Corcoran of NYU demonstrated that the New York City teacher data reports have an average margin of error of 34 to 61 percentage points out of 100. The National Academy of Sciences has also warned of the potentially damaging consequences of implementing these unfair and inherently unreliable evaluation systems. Even the NYC Department of Education’s own consultants have warned against using data for teacher evaluation.
Perverse Incentives
Value-added measures can not only be in error but they provide incentives for teachers to manipulate scores by using large amounts of classroom time practicing for tests or engaging in various forms of cheating. To the extent a teacher cuts corners one year to deliver improved test scores, a student’s next teacher will face that much greater of a challenge to deliver similar or even better results.
Teachers under the gun of having their very lifelihood threatened will be very careful about working with troubled children who could drag down their value-added ratings. Accountable Talk” wrote about dealing with a request to take a class full of difficult students:
Norm Scott worked in the New York City public school system from 1967 to 2002. He publishes commentary about current issues in New York City public education at ednotesonline.blogspot.com.
Here are some news story links I copied from Gotham:
Any effort to do so would spark a major outcry. But when it comes to teaching, there is a different standard.
On Monday Manhattan Supreme Court Justice Cynthia Kern ruled that the NYC Department of Education was obliged to release the names of individual teachers with “value-added” test score results that purport to measure teacher effectiveness. Judge Kern brushed aside arguments by the United Federation of Teachers (UFT) that the release of unreliable data would unjustly harm teachers’ reputations writing “there is no requirement that data be reliable for it to be disclosed.”
The data dump will affect more than 12,000 classroom educators in Grades 4 to 8. The UFT is expected to appeal. There is some irony here as it was the UFT that signed off on the use of value-added in the first place after Joel Klein promised the results would not be made public, while many skeptical critics in the union raised questions about that deal and warned it would turn into a disaster for teachers and the union.
Feeding Frenzy
If the value-added data is ultimately released, expect a feeding frenzy as teachers are judged and shamed on an individual basis in the media. The larger purpose of such a data dump by DOE would be to further erode public support for teachers and force their union to renounce a seniority-based system just as Mayor Michael Bloomberg and his new Schools Chancellor Cathie Black are talking about having to lay off thousands of teachers due to budget shortfalls.The data dump will affect more than 12,000 classroom educators in Grades 4 to 8. The UFT is expected to appeal. There is some irony here as it was the UFT that signed off on the use of value-added in the first place after Joel Klein promised the results would not be made public, while many skeptical critics in the union raised questions about that deal and warned it would turn into a disaster for teachers and the union.
Feeding Frenzy
Ironically, it was only six months ago that the NY State Department of Education revealed that years of test score advances by city students had turned out to be a mirage causing Bloomberg and his former Chancellor Joel Klein a good deal of embarrassment. No matter – the Mayor is ready once again to wield unreliable test scores as a political weapon and most media in this city have deliberately short memories, having all too often been active partners in attempts to eviscerate teachers.
The value-added approach is the latest attempt to undermine teachers, the teaching profession and the teacher union by measuring teachers based on the performance of their students on standardized tests from year to year.
Crucial backing for such initiatives has come from private foundations led by billionaires like Bill Gates and Eli Broad who assert that data-driven models in the private sector can be transferred to public schools.
Their dream of using data to accurately measure the effectiveness of individual teachers is rooted in a vision of the school as a factory in which teachers are assembly line workers and rising student test scores equals rising workforce productivity. At long last, value-added supporters claim, good teachers will be rewarded and the poor ones forced to improve at risk of losing their jobs.
In reality, value-added measures are seriously flawed. They don’t fully account for external circumstances such as poverty or family turmoil that can affect a child’s performance from year-to-year. Nor can they account for the fact the same child can take tests on different occasions and under different conditions and the results will differ.A study by Mathematica Policy Research done for the US Department of Education showed that one-fourth of average teachers will be mistakenly identified for special rewards while one-fourth of teachers who differ from average performance by three to four months of student learning will be overlooked.
A recent study by Sean Corcoran of NYU demonstrated that the New York City teacher data reports have an average margin of error of 34 to 61 percentage points out of 100. The National Academy of Sciences has also warned of the potentially damaging consequences of implementing these unfair and inherently unreliable evaluation systems. Even the NYC Department of Education’s own consultants have warned against using data for teacher evaluation.
Perverse Incentives
Value-added measures can not only be in error but they provide incentives for teachers to manipulate scores by using large amounts of classroom time practicing for tests or engaging in various forms of cheating. To the extent a teacher cuts corners one year to deliver improved test scores, a student’s next teacher will face that much greater of a challenge to deliver similar or even better results.
Teachers under the gun of having their very lifelihood threatened will be very careful about working with troubled children who could drag down their value-added ratings. Accountable Talk” wrote about dealing with a request to take a class full of difficult students:
“I did something I am still not proud of. I quit,” Accountable Talk wrote. “No, I didn’t quit teaching. I just quit volunteering to teach the very children who needed me most. When my AP [assistant principal] asked me to take them on again (which he would not do unless he knew I’d been successful), I said no. This year, those kids are with another teacher who has difficulty just getting them to sit in their seats.”
One of the political goals of the value-added approach is to break teacher unity by pitting them against each other. The competitive, zero-sum logic of value-added also undermines the spirit of collaboration which is essential to them refining and developing their craft. If sharing tips with fellow teachers will help them improve their value-added rankings, is it prudent to reach out and help teachers you are competing with?
The downside of a value-added approach doesn’t faze leading proponents like Eric Hanushek, a Stanford economist who has written that teachers’ scores should be made public even if they are flawed.Several news organizations including The New York Post, The New York Times and The Wall Street Journal have filed Freedom of Information requests for New York City teacher test score data with which the normally secretive NYC Department of Education has been eager to comply.
Still, it is wise to remember that not everything that counts can be measured and not everything that can be measured counts.
Still, it is wise to remember that not everything that counts can be measured and not everything that can be measured counts.
Norm Scott worked in the New York City public school system from 1967 to 2002. He publishes commentary about current issues in New York City public education at ednotesonline.blogspot.com.
Here are some news story links I copied from Gotham:
- A judge said the city can release teachers’ value-added ratings. (GS, Times, Post, DN, NY1, WNYC, WSJ)
- The teachers union is planning to appeal the release, so it won’t happen yet. (GothamSchools)
- The Post says union president Michael Mulgrew is wrong to appeal the judge’s ruling.
For ten years, The Indypendent has printed truth in the face of power. With political and economic systems faltering, there is an opportunity for real change from the bottom up. But this means having a vibrant independent media. Consider supporting The Indypendent as a monthly sustainer, donating as little as $5 a month. Please visit indypendent.org/donate.
Subscribe to the Indypendent!Below the fold: A list of some of the great pieces on ed that ran in The Indypendent
More Highlights from the Indy's 2010 Education Coverage
"Taking the Public Out of Schools" by John Tarleton
"Inside Columbus High School" by Mary Heglar
"Experience Is the Best Teacher: Bronx School Fights to Save Building Trades Program As DOE Pushes College Prep Over Hands-on Learning" by Mary Heglar
"Think Globally, Privatize Locally: Education Is Under Attack Around the World" by Lois Weiner
"Teaching Under Assualt: Two Visions of Education Clash as Bloomberg Prepares to Layoff 6,400 Teachers" by Norm Scott
"Why Teachers Unions Matter" by Lois Weiner
"Ticket to Ride: Students Win Metro Card Fight" by Jaisal Noor
"Students at Tilden High School Win Last Chance for Diploma" by Jaisal Noor
"An Education at Any Age: A Boy from Baghdad and His Parents Navigate Different Ends of the NYC School System" by David Enders
"Learning the 3 C's: Competition, Corruption and Cheating" by Arthur Goldstein and Lucas Hilderbrand
"A Parent's Guide to School Involvement" by John Tarleton
"Education Rediscovered" by Stanley Aronowitz
"Queer Youth Embrace Fluid Identities" by S. Leigh Thompson
"School Closings Pushback Begins" by John Tarleton
"Experience Be Damned!: Education Department's Self-Inflicted Crisis Leaves More Than 1,000 Veteran Teachers in Limbo" by Marc Epstein
Check out Norms Notes for a variety of articles of interest: http://normsnotes2.blogspot.com/. And make sure to check out the side panel on right for news bits.
We absolutely should evaluate MD's based on patient outcomes. That's how we can identify and fire the bad ones!
ReplyDeletehttp://www.examiner.com/k-12-in-topeka/we-must-fire-bad-doctors
:)