Wednesday, April 17, 2013

not only bad statistics, but deeply flawed/erroneous theories, as well...,

wired | The fields of psychology and cognitive neuroscience have had some rough sledding in recent years. The bumps have come from high-profile fraudsters, concerns about  findings that can’t be replicated, and criticism from within the scientific ranks about shoddy statistics. A new study adds to these woes, suggesting that a wide range of neuroscience studies lack the statistical power to back up their findings.

This problem isn’t just academic. The authors argue that there are real-world consequences, from wasting the lives of lab animals and squandering public funding on unreliable studies, to potentially stopping clinical trials with human patients prematurely (or not stopping them soon enough).

“This paper should help by revealing exactly how bad things have gotten,” said Hal Pashler, a psychologist at the University of California, San Diego. Pashler was not involved with the new study, but he and colleagues have previously raised concerns about statistical problems with fMRI brain scan studies in human subjects.

The aim of the new study wasn’t to rake neuroscientists over the coals, but to get them talking about how to change the culture and the incentives that promote statistically unreliable studies, says co-author Marcus Munafò, a psychologist at the University of Bristol, United Kingdom. “We’re really trying to be constructive about this.” Fist tap Dale.


Dale Asberry said...

Sadly, the "thinking like a scientist primes ethical behavior" falls into the low-powered category. I'll try to keep an eye on such things now.

CNu said...

Do you remember Chelly from the Progstone list? I've been doing a lot of refresher reading provoked by a question or two that Chelly sent my way a few weeks ago, and among the things I've made myself reread is Origins of Consciousness. I swear there's something new to learn in that text with every single reading, but I digress. The first two chapters in which Jaynes reviews the science or philosophy of mind remains one of the landmark or signature achievements in deep and careful thought - OF ALL TIME!!!!!

Would that all scientists were so scrupulous and careful with their framing of hypothesis....,