I love a good open debate about evidence. And we got one this week – thanks to an open access journal and the platform that great science writers now have by blogging and tweeting.
The journal BMJ Open, self-described as “An open access, online-only general medical journal dedicated to publishing research from all disciplines and therapeutic areas,” published a paper this week entitled, “Oral contraceptive use is associated with prostate cancer: an ecological study.” Its conclusion: “A significant association between oral contraceptives and prostate cancer has been shown. It is hypothesised that the oral contraceptive effect may be mediated through environmental oestrogen levels; this novel concept is worth further investigation.”
That paper drew quite a response from some science bloggers, most notably British science writer Ed Yong (whose Not Exactly Rocket Science blog on the Discover magazine site is always a good read). On his Google+ page, Yong summarized his earlier Twitter rant (his words) about the paper. Excerpts:
You may have noticed that people who take the Pill don’t have prostates. Their suggestion is that Pill use -> higher oestrogen levels in the environment or drinking water -> higher prostate cancer risk. The problem is that both of those links are tenuous as hell and they haven’t made any efforts to substantiate them. Did they actually measure oestrogen levels in the environment to see if they correlated with either of those things? No. Because that would be too close to actual work. They also say that this mechanism is “plausible” without providing a reference. Which means they pulled it out of the air. Lots of studies produce correlations that aren’t necessarily causative, but if you don’t have the first clue about a mechanism that could plausible explain your result, perhaps it’s time to find one first?
They say study is “speculative” and “hypothesis-generating”. That’s a common refrain with these sorts of “ecological studies”. It is a fancy way of saying “so shallow as to be useless”. No, these studies can’t prove causation on their own, but the evidence they provide is so pathetically weak that they could equally well have generated some hypotheses by shouting them to the winds.
Rubbish like this is a complete insult to scientists in fields who actually have to do significant work before they can publish a paper. Just think of how much effort goes into your average neuroscience or molecular biology or behavioural ecology paper – months, probably years, of experiments, pernickety reviewers, and more. And these folks just clumsily mash together two data sets, found a correlation that they can’t explain, and publish it?? Thanks. Thanks a bunch.
But, almost predictably, because a provocative-sounding study was actually published in a journal of any type, more than 100 news stories were published about the study.
The Ministry of Truth blog also commented on some of that news coverage and on the underlying science.
On that site, BMJ Open editor in chief Trish Groves wrote:
“The paper and the press release both stated clearly that the paper wasn’t ascribing cause and effect.
Anyway, I hope you’ll find time to send an eletter to the paper at BMJ Open, adding a link to this critique.
Postpublication peer review can be just as important as the peer review process at the journal.”
On Twitter, she had responded to Yong:
“paper and press release both stated very clearly that @BMJ_Open study was ecological and only hypothesis promoting”
To all the journalists who wrote about a link between the Pill and prostate cancer without evaluating the evidence, all I can say is “Wow” and “Why?”