Metrics vs Common Sense

No mention of education, per se, but so applicable …

Welcome to the Age of Metrics — or to the End of Instinct. Metrics are everywhere. It is increasingly with them that we decide what to read, what stocks to buy, which poor people to feed, which athletes to recruit, which films and restaurants to try. World Metrics Day was declared for the first time this year.

The once-mysterious formation of tastes is becoming a quantitative science, as services like Netflix and Pandora and StumbleUpon deploy algorithms to predict, and shape, what we like to watch, listen to and read.

These services are wondrous. They also risk lumping us into clusters of the like-minded and depriving us of the self-fortifying act of choosing. What will it mean to prefer one genre of song when you have never confronted others? It is one thing to love your country because you have seen the world and love it still; it is quite another to love it because you know nothing else.

… In short, what we know instinctively, data can make us forget. [emphasis added]

The whole NYT article here.

Tests and Teacher Pay

An issue brief from the Century Foundation by Gordon MacInnes: “Eight Reasons Not to Tie Teacher Pay to Student Test Scores.”

Reason #1: Tying test scores to teacher compensation suggests that teachers are holding back on using their experience, expertise, and time because they are not being paid for the extra effort.

Reason # 2: The standardized tests in most states are lousy and so are the standards they are designed to measure.

Reason #3: The idea of compensating teachers individually in order to differentiate their performance from their school colleagues defeats a principal tenet of good instruction—that teachers need to learn from one another to solve difficult pedagogical challenges.

Reason #4: Most teachers do not teach a grade or subject that is subject to standardized testing.

Reason # 5: Even reliable standardized tests are valid only when they are used for their intended purposes.

Reason #6: A key assumption of using test scores to judge teachers is that students are randomly assigned, first, to schools, and, second, to classes. Neither is true.

Reason #7: State data systems are in their infancy. It turns out that it is harder, is more expensive, and takes longer for states to produce reliable, accurate, and secure longitudinal data on students and teachers than widely assumed.

Reason #8: The rationale for tying tests to compensation is not clear.  …  if teacher compensation does not keep up with inflation because of poor student performance, then teachers will . . . what? Work harder? Dig deeper? Stay longer? There is no evidence that such measures improve instructional practices or student outcomes.

And as Diane Ravitch points out: to tie test scores to teacher evaluation requires a minimum of three years’ data. At then end … what? Fire the “bad” teachers? And replace them with … who? new teachers? There is no precedent for this in any other developing nation.

Read MacInnes’ entire brief here.

Reading Incomprehension

Fascinating NY Times Op-Ed piece about the vagaries of assessment scoring:

One of the tests I scored had students read a passage about bicycle safety. They were then instructed to draw a poster that illustrated a rule that was indicated in the text. We would award one point for a poster that included a correct rule and zero for a drawing that did not.

The first poster I saw was a drawing of a young cyclist, a helmet tightly attached to his head, flying his bike over a canal filled with flaming oil, his two arms waving wildly in the air. I stared at the response for minutes. Was this a picture of a helmet-wearing child who understood the basic rules of bike safety? Or was it meant to portray a youngster killing himself on two wheels?

I was not the only one who was confused. Soon several of my fellow scorers — pretty much people off the street, like me — were debating my poster, some positing that it clearly showed an understanding of bike safety while others argued that it most certainly did not. I realized then — an epiphany confirmed over a decade and a half of experience in the testing industry — that the score any student would earn mostly depended on which temporary employee viewed his response. [emphasis added]

Read the entire piece here.