Story 2014-04-26 3J9 Spotting Bad Science

Spotting Bad Science

by
Anonymous Coward
in science on (#3J9)
story imageCompound Interest , a chemistry blog, has posted an infographic guide to spotting bad science. Among the listed things to watch out for are perennial favorites "Correlation & Causation" and "Sample Size Too Small".

The author was inspired to create the guide after running into so many of these gotchas while doing online research for a different article. It is in no way intended to be comprehensive, but instead to act as a brief reminder of what things to be alert for when reading science news articles and research results.

Guides like this might be overly simple for the readership of a site like Pipedot, but I personally find them useful as refreshers, much as I do the excellent TechNyou video series "Critical Thinking" and "This Thing Called Science" which, though aimed at high-schoolers and below, are still a good foundational resource. Then again, I like bright colors and shiny objects.
Reply 5 comments

3 weeks old news... (Score: 0)

by Anonymous Coward on 2014-04-26 15:59 (#16F)

Please pipedot, we can do better than that.

Re: 3 weeks old news... (Score: 3, Interesting)

by bryan@pipedot.org on 2014-04-26 20:33 (#16G)

I know of a few people that should really have this poster hanging on their wall.

Besides, for those that haven't seen it yet: http://xkcd.com/1053/

not about being right or wrong? (Score: 2, Interesting)

by rocks@pipedot.org on 2014-04-27 05:05 (#16H)

Over the past decade or so, I have come to think that published bad science is shockingly common. I have tried to ruminate on the reasons for this and, while it is easy to identify many contributing factors in the way scientific communities and institutions work, I think the main reason is that science has two completely different functions in human life that work against one another. On the one had, science can be about challenging our own preferred, learned, or expert hypotheses (a la Feynman: "Science is the belief in the ignorance of experts" . On the other hand, science can be about presenting and defending our own preferred, learned, or expert hypotheses (a la Scientific American: Ask the Experts ). Both functions serve useful purposes in our lives at different times, but each can look bad from the perspective of the other. i.e., challenges to authoritative science can look bad and receive short thrift because we frequently have a lot of confidence in the authoritative accounts, but authoritative accounts can look bad when we stop prodding them for the ways they might be wrong or limited. Who gets to decide when it is appropriate to defend or challenge an authoritative account on some topic is a central issue. In my opinion, the commonness of published bad science is a consequence of spending too much attention focused on being right about nature and not enough attention focused on how we might be confirming our assumptions or biases.

Correlation, causation, and all that. (Score: 2, Insightful)

by danieldvorkin@pipedot.org on 2014-04-27 15:16 (#16R)

At this point the meme has so thoroughly permeated the culture that I really wish every mention of "correlation is not causation" (or "correlation does not imply causation," or any of its variants) came with a warning label explaining that in most cases, there's a whole lot more to the analysis than that.

Re: Correlation, causation, and all that. (Score: 2, Funny)

by danieldvorkin@pipedot.org on 2014-04-27 15:17 (#16S)

Or less gently, as my Slashdot .sig used to say, "The correlation between ignorance of statistics and using 'correlation is not causation' as an argument is close to 1."