Analysis of meta-analyses identifies where sciences’ real problems lie

(credit: Harvard University)

Science is in a phase of pretty intense soul-searching. Over the past few years, systemic problems that lead to unreliable scientific results have become more and more obvious. There's a litany of woes for good science: publication bias leads to buried data, single studies don't stand well on their own yet not enough people are replicating them, and flaws in the peer-review process are showing. And that's before we even get to the (hopefully occasional) research fraud.

John Ioannidis, one of the heroes of the science-scrutinizing movement, has some news in PNAS this week that is simultaneously uncomfortable and comforting. Ioannidis, along with colleagues Daniele Fanelli and Rodrigo Costas, scoured thousands of scientific papers to uncover some of the most common causes of bias. Their findings suggest that, for the most part, people are worrying about the right things, including small studies that spark a lot of scientific conversation. But they also pinpoint other causes for concern that haven't attracted much attention so far: early career researchers and isolated scientists.

Data about data about data

Fanelli is a meta-researcher: a scientist whose research is itself about scientific research. In order to get a broad view of the biases at play across all of science, he went hunting for meta-analyses. These are scientific studies that combine the data from a range of separate studies in the same area. Meta-analyses often give a more comprehensive picture of the current evidence than any individual study.

Read 10 remaining paragraphs | Comments