Tuesday, May 22, 2012

useful, reliable, and non-obvious predictions from the social science of physics education research...,

aera-l | Rick Froman of the TIPS discussion list has pointed to a New York Times Opinion Piece "How Reliable Are the Social Sciences?" by Gary Gutting at <http://nyti.ms/K0xVQL>. Gutting wrote that Obama, in his State of the Union address <http://wapo.st/JnuBCO> cited "The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood" (Chetty et al., 2011 at <http://bit.ly/KkanoU>) to support his emphasis on evaluating teachers by their students' test scores. That study purportedly shows that students with teachers who raise their standardized test scores are "more likely to attend college, earn higher salaries, live in better neighborhoods, and save more for retirement."

After comparing the reliability of social-science research unfavorably with that of physical-science research, Getting wrote [my CAPS): "IS THERE ANY WORK ON THE EFFECTIVENESS OF TEACHING THAT IS SOLIDLY ENOUGH ESTABLISHED TO SUPPORT MAJOR POLICY DECISIONS?" THE CASE FOR A NEGATIVE ANSWER lies in the [superior] predictive power of the core natural sciences compared with even the most highly developed social sciences."

Most education experts would probably agree with Getting's negative answer. Even economist Eric Hanushek, as reported by Lowery <http://nyti.ms/KnRvDh>, states: "Very few people suggest that you should use value-added scores alone to make personnel decisions."

But then Getting goes on to write (slightly edited): "While the physical sciences produce many detailed and precise predictions, the social sciences do not. The reason is that such predictions almost always require randomized controlled trials (RCT's) which are seldom possible when people are involved. . . . . . Jim Manzi. . . .[[according to Wikipedia <http://bit.ly/KqMf1M>, a senior fellow at the conservative Manhattan Institute <http://bit.ly/JvwKG1>]]. . . . in his recent book "Uncontrolled" <http://amzn.to/JFalMD> offers a careful and informed survey of the problems of research in the social sciences and concludes that non-RCT social science is not capable of making useful, reliable, and nonobvious predictions for the effects of most proposed policy interventions." BUT:

(1) Randomized controlled trails may be the "gold standard" for medical research, but they are not such for the social science of educational research - see e.g., "Seventeen Statements by Gold-Standard Skeptics #2" (Hake, 2010) at <http://bit.ly/oRGnBp>.

(2) Unknown to most of academia, and probably to Getting and Manzi, ever since the pioneering work of Halloun & Hestenes (1985a) at <http://bit.ly/fDdJHm>, physicists have been engaged in the social science of Physics Education Research that IS "capable of making useful, reliable, and nonobvious predictions," e.g., that "interactive engagement" courses can achieve average normalized pre-to-posttest gains which are about two-standard deviations above *comparison* courses subjected to "traditional" passive-student lecture courses. This work employs pre/post testing with Concept Inventories <http://en.wikipedia.org/wiki/Concept_inventory> - see e.g., (a) "The Impact of Concept Inventories on Physics Education and It's Relevance For Engineering Education" (Hake, 2011) at <http://bit.ly/nmPY8F>, and (b) "Why Not Try a Scientific Approach to Science Education?" (Wieman, 2007) at <http://bit.ly/anTMfF>.