physicstoday | “The case against science,” wrote Richard Horton, editor of the medical journal the Lancet, “is straightforward: much of the scientific literature, perhaps half, may simply be untrue.” Horton’s April commentary appeared weeks before news broke about Science
magazine’s now widely analyzed retraction of a psychology paper about
attitudes towards gay marriage. Much in the current media analysis,
whether or not citing Horton specifically, aligns with his
judgment—sometimes without much of the hopefulness he framed within this
harshness.
Consider, for example, Charles Seife’s Los Angeles Times op-ed.
The science journalist and New York University journalism professor
discerns “a weakness at the heart of the scientific establishment,”
which a “steady drip-drip-drip of falsification, exaggeration and
outright fabrication [is] eroding.”
Science itself eroding? Horton leveled general accusations:
Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness.… The apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world. Or they retrofit hypotheses to fit their data. Journal editors deserve their fair share of criticism too. We aid and abet the worst behaviours.
But in some ways—not always matched in the popular
media—Horton placed hopefulness alongside the harshness. “The good news
is that science is beginning to take some of its worst failings very
seriously,” he wrote, though the “bad news is that nobody is ready to
take the first step to clean up the system.” He described possible
hopeful steps, including one inspired by physicists:
One of the most convincing proposals came from outside the biomedical community. Tony Weidberg is a Professor of Particle Physics at Oxford. Following several high-profile errors, the particle physics community now invests great effort into intensive checking and rechecking of data prior to publication. By filtering results through independent working groups, physicists are encouraged to criticise. Good criticism is rewarded. The goal is a reliable result, and the incentives for scientists are aligned around this goal. Weidberg worried we set the bar for results in biomedicine far too low. In particle physics, significance is set at 5 sigma—a p value of 3 × 10–7 or 1 in 3.5 million (if the result is not true, this is the probability that the data would have been as extreme as they are).
Other scientists have also given the popular media cues
for generalizing harshly about science overall from specific failings
like the retracted Science paper. At Nature, Richard Van Noorden borrowed phrasing from Seife in reporting that delegates to the recent World Conference on Research Integrity in Rio de Janeiro saw in the retracted study only more of the “steady drip-drip of research misconduct.” In December 2013, the American Journal of Neuroradiology published a paper with a powerfully loaded title: “The fraud and retraction epidemic.” Last month at The Conversation, Laureate Professor of Mathematics Jonathan Borwein of the University of Newcastle published a piece
called “The ‘train wreck’ continues: Another social science
retraction.” The train-wreck analogy comes from 2012, when Nobel
laureate Daniel Kahneman called on psychologists to tighten up
replicability in “social priming” research, as reported in Nature. Two years ago, Borwein coauthored a Huffington Post essay
arguing that the “scientific world is suffering through a rash of
examples of the sad consequences of the ‘hype now, hide later’ approach
to scientific news.”
