aera-l | Relevant to the recent Atlanta cheating scandal, a
byproduct of the draconian consequences of failure to meet AdequateYearly Progress targets of the No ChildLeft Behind act , I point to six empirical
"laws" related to "The Counterproductivity of Targets,"
five of which were listed EvalTalk's Bill Fear (2013) in his EvalTalk
post "The Performance Paradox...da da da!!". These are
epitomized by "Campbell's Law": "The more any
quantitative social indicator is used for social decision-making, the
more subject it will be to corruption pressures and the more apt it
will be to distort and corrupt the social processes it is intended to
monitor."
Fear (2013), wrote:
Fear (2013), wrote:
I first heard Frans Leeuw talk about the "Performance Paradox" in relation to evaluation. Unfortunately I don't think he wrote about it other than a thesis by one of his students.
However, quite by chance I found some of the roots of it today.It seems that MarilynStrathern was one of the first to articulate this phenomenon in recent times. I like her quote for it's brevity.
1. "When a measure becomes a target, it ceases to be a good measure."
- Marilyn Strathern (1997)
2. Goodhart's Law "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes." Also: "Once a social or economic indicator or other surrogate measure is made a target for the purpose of conducting social or economic policy, then it will lose the information content that would qualify it to play such a role."
- Charles Goodhart (1981).
3. The Lucas critique "It is naïve to try to predict the effects of a change in economic policy entirely on the basis of relationships observed in historical data, especially highly aggregated historical data."
- Robert Lucas (1976)
4. "A risk model breaks down when used for regulatory purposes."- Jon Danželsson (2002)
5. Campbell's Law: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
- Donald Campbell (1976)
