Cathy O’Neill: Don’t Grade Teachers with Bad Algorithms

Cathy O’Neill is a mathematician who wrote a wonderful book called “weapons of Math Destruction,” in which she showed how math and big data can be misused to reach very bad decisions.

She recently wrote an article for Bkloomberg News in which she explained why VAM is a terrible way to evaluate teachers.

She writes:

“For more than a decade, a glitchy and unaccountable algorithm has been making life difficult for America’s teachers. The good news is that its reign of terror might finally be drawing to a close.

“I first became acquainted with the Value-Added Model in 2011, when a friend of mine, a high school principal in Brooklyn, told me that a complex mathematical system was being used to assess her teachers — and to help decide such important matters as tenure. I offered to explain the formula to her if she could get it. She said she had tried, but had been told “it’s math, you wouldn’t understand it.”

“This was the first sign that something very weird was going on, and that somebody was avoiding scrutiny by invoking the authority and trustworthiness of mathematics. Not cool. The results have actually been terrible, and may be partly to blame for a national teacher shortage.

“The VAM — actually a family of algorithms — purports to determine how much “value” an individual teacher adds to a classroom. It goes by standardized test scores, and holds teachers accountable for what’s called student growth, which comes down to the difference between how well students performed on a test and how well a predictive model “expected” them to do…

“Fundamental problems immediately arose. Inconsistency was the most notable, statistically speaking: The same person teaching the same course in the same way to similar students could get wildly different scores from year to year. Teachers sometimes received scores for classes they hadn’t taught, or lost their jobs due to mistakes in code. Some cheated to raise their students’ test scores, creating false baselines that could lead to the firing of subsequent teachers (assuming they didn’t cheat, too).

“Perhaps most galling was the sheer lack of accountability. The code was proprietary, which meant administrators didn’t really understand the scores and appealing the model’s conclusions was next to impossible. Although economists studied such things as the effects of high-scoring teachers on students’ longer-term income, nobody paid adequate attention to the system’s effect on the quality and motivation of teachers overall.”

Happily, she says, VAM is on the way out. The new federal law does not require it, and courts have been ruling against. She cites Sheri Lederman’s court victory in New York and the recent court victory in Houston, where the judge said the algorithm was so opaque that it should not be used at all.

VAM has ruined the careers, the reputations, and the lives of many educators. One teacher, Rigoberto Ruelas, committed suicide soon after his VAM rating was published online by the Los Angeles Times. This is an example of a deadly use of math to damage real people, not just a game played by economists. Whoever participated willingly in this sham exercise should do penance.

from novemoore


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s