Monday, November 8, 2010
DOE progress reports: The Insane Elephant in the Room
Why is the media not scrutinizing the immense flaw in the DOE's methodology of its progress reports? In what other field of practice, do we keep increasing the standard of any measure of evaluation - and calling that valid?
The DOE every year, increases the "cut" score for determining the letter grade for each school's evaluation. So what would have been an A two years ago is now a C.
Really? To see the farcical nature of this system - let's just imagine hospitals were rated in this manner. So in one year, a hospital can be rated as a top notch hospital, but two years later be rated as a dangerous C, even though the raw scores may have improved?
In baseball, a good batting average is around .300 - which means that a batter manages a hit only 3 times out of 10. We can imagine Bloomberg taking over the Yankees and incrementally increasing that measure to .350, .400., .450,... every year until Alex Rodriguez is out of a job.
Legally speaking, a driver is DWAI, when the blood alcohol content is 0.05 % or higher. Leave it up to Bloomberg, and in a few years, you won't be able to eat that slice of grandma's rum cake before leaving Christmas dinner.
In Bloomberg's world, millions of more cars would fail their yearly inspection, a fever of 98.7 would require antibiotics, and if you happen to skip your vitamin C for the day your t-cell count would require you to be on AIDS medication.
The only rationale the DOE incessantly brings up is the idea of "competition" - that schools need to be evaluated based on how other schools perform. The only problem is - competition is not what's being evaluated here. Schools are not competing with each other, they are competing against a one-size-fits-all cut score - that's ever increasing.
An evaluation system of physicians currently being developed is running into similar problems. They are noticing that physicians in poorer areas have patients with longer recovery times and more complications. Of course, we know that patients with less access to health care will have more acute conditions before seeking treatment.
If Bloomberg were in charge of evaluating military doctor's in Iraq and Afghanistan - he'd fire every army surgeon for having terrible rates of limb loss during operations. Boston general would get an A (this year at least) and military units ...an F.
So, here's the danger. In a world where Bloombergs of the world threaten a professional's ability to feed their children and keep a roof over their heads, massive cheating ensues and real progress ceases - more doctor's will fudge their medical reports, more batters will take steroids, and anti-biotic resistant bacteria will take over.....oh. So much for the business model.
Check out Norms Notes for a variety of articles of interest: http://normsnotes2.blogspot.com/