Jersey Jazzman: Why Won’t CREDO Stop Using the Invalid “Days of Learning” Metric?

Mark Weber, aka the blogger Jersey Jazzman, is getting his doctorate in research and statistics while teaching in a public school in New Jersey. He is a sharp critic of shoddy research, especially when it comes to the fantastical claims made on behalf of charter schools.

In his latest post, he asks why CREDO, the charter-evaluating institute at Stanford University run by Macke Raymond, continues to use a metric that has never been validated.

Journalists who have little expertise in evaluating research claims eagerly take up the claim that School X produces an additional “number of days of learning.”

It happened most recently in Texas, where charter schools finally managed to match the test scores of public schools (you know, those “failing schools” for which charter schools are supposed to be the rescuers.)

He shows how the Texas study refers to “days of learning” and this is translated to infer “substantial” improvement. But, as JJ shows, the gains are actually very small, and might more accurately be described as “tiny.”

He writes:

Stanley Pogrow published a paper earlier this year that didn’t get much attention, and that’s too bad. Because he quite rightly points out that it’s much more credible to describe results like the ones reported here as “small” than as substantial. 0.03 standard deviations is tiny: plug it in here and you’ll see it translates into moving from the 50th to the 51st percentile (the most generous possible interpretation when converting to percentiles).

I have been working on something more formal than a blog post to delve into this issue. I’ve decided to publish an excerpt now because, frankly, I am tired of seeing “days of learning” conversions reported in the press and in research — both peer-reviewed and not — as if there was no debate about their validity.

The fact is that many people who know what they are talking about have a problem with how CREDO and others use “days of learning,” and it’s well past time that the researchers who make this conversion justify it.

Jersey Jazzman calls on Macke Raymond and the staff at CREDO to justify their use of this measurement. The “days of learning” inflates the actual changes, he says.

The concept of days of learning, he says, is based on the work of economist Erik Hanushek of the Hoover Institution (Stanford). It may be coincidental that he is Macke Raymond’s husband. They are both very smart people. I hope they respond to Mark Weber’s challenge.

from novemoore


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s