Health News Review

Just a few days ago, a paper in the journal PLoS One, “Media Coverage of Medical Journals: Do the Best Articles Make the News?” showed how journalists are more likely to report on observational studies than on randomized clinical trials.  The authors suggest this shows a systematic bias to report on weaker evidence.

And here’s one reason why that may happen.

This week the BMJ sent out a news release on a paper from the Annals of the Rheumatic Diseases, one of the specialist journals it publishes. And I’ve been waiting for days to address it – waiting for the journal’s embargo time to pass so that I’m not violating that sacred trust.

The headline of the news release:  HRT cuts risk of repeat knee/hip replacement surgery by 40%.

 

Nope.  Sorry, BMJ news release writers.  That’s an overstatement, to be kind.  An inaccuracy, to be accurate. That’s not what the study showed, because it wasn’t equipped to show that anything “cut risk.”  Proof of cutting risk would be proof of cause and effect.  And the observational study in question can’t do that.

Don’t blame the authors of the journal article. They didn’t use cause-and-effect language.  They concluded: “HRT is associated with an almost 40% reduction.”  (My emphasis added.) That’s the way you describe the results of an observational study. That’s what we try to teach journalists and the public with a primer that’s been on our site for years. Maybe the BMJ should have its news release writers read it.

The researchers went even further, for any writer – journal news release writers included – who bothered to read the study.  The research team wrote: “The main limitation of this study is its observational nature.”

To be clear, this was a large study with long followup.  This could be a head-turner in medical circles.

But it still is what it is – a study that can only show statistical association.

And association ≠ causation.

We’ve written about this problem with BMJ news releases in the past, and will continue to do so until they get it right.  Past examples:

As I wrote in one of these posts:  Journals could help lift all ships – or they can (and sometimes do) help us all drown in a daily tsunami of global miscommunication about health news.

——————-

Follow us on Twitter:

https://twitter.com/garyschwitzer

https://twitter.com/healthnewsrevu

and on Facebook.

 

 

Comments

Wendy Sue Swanson posted on January 22, 2014 at 6:05 pm

Thanks for writing this. Have you ever heard from BMJ staff after writing these posts?

It would behoove academic journals to require primary authors to submit a summary of their work (for the public) and takeaways to be used by the media and for use on social media. I’ve mentioned this to a couple of editors or large journals. Not there yet but perhaps we will be soon.

    Gary Schwitzer posted on January 22, 2014 at 6:40 pm

    Yes. Twice as I recall. Both times defensive, not admitting a mistake. Maybe it’ll be different this time.

    Shouldn’t we expect more from a journal?

James O'Callaghan, MD FAAP FHM posted on January 22, 2014 at 6:29 pm

Thanks for putting in clear language the limitations of observational studies, mainly association ≠ causation. Unfortunately, soundbites sell, so I suspect this problem will continue.

Stewart B. Leavitt, MA, PhD posted on January 22, 2014 at 9:04 pm

In fairness to publicists, it is often necessary to truncate the full implications of a story when creating a brief headline. You do not provide a link to the press release and, if they do imply causation in their explanation of the study, either they don’t understand research or they are misleading the public, or both. This also is not to excuse the way news writers often use sensationalism to garner a larger audience, but, with all the media outlets today, it is a very hard to attract the attention of readers.

    Gary Schwitzer posted on January 22, 2014 at 9:47 pm

    Stewart,

    Thanks for your note.

    Truncating is one thing.

    Accuracy is another, and is paramount.

    Using causal language, as this one did, to describe an observational study, is inaccurate.

      Stewart B. Leavitt, MA, PhD posted on January 22, 2014 at 10:17 pm

      Agreed. I’m not a publicist, so I really don’t want to defend them; and, typical news reporters just repeat the news releases without critical thinking, which is unforgivable. Just sayin’ there are sometimes reasons why the headlines themselves do not tell the story accurately. It would be helpful if you’d provide a link to the full BMJ press release.

      Gary Schwitzer posted on January 22, 2014 at 10:33 pm

      No news release posted online yet, so can’t provide any link.

      But I can say that there was no quibble with most of the body of the news release.

      But “the hook” — “cuts risk by 40%” — is all you need to sour the entire message because it’s wrong.

MacLochlainn posted on January 23, 2014 at 7:06 am

Wasn’t smokingl associated with lung cancer.

Bob Capers posted on January 23, 2014 at 7:54 am

this all might make more sense if you ever explained what BMJ is. those acronyms will kill you every time.

    Gary Schwitzer posted on January 23, 2014 at 8:33 am

    Bob,

    Thanks.

    A reminder that one should never take anything for granted.

    BMJ is the name of what used to be called the British Medical Journal.

Miriam Shuchman posted on January 23, 2014 at 7:57 am

Hi Gary-As always you are so clear in identifying the problems. Thanks. Have you approached the ICMJE http://www.icmje.org or the WAME (World Assoc of Medical Editors) or COPE (Committee on Publication Ethics) to work with journals to fixi this problem?

    Gary Schwitzer posted on January 23, 2014 at 8:35 am

    Miriam,

    No, I haven’t.

Trish Groves posted on January 23, 2014 at 12:15 pm

Hi Gary

Thanks for picking this up. With more than 50 journals, BMJ issues a lot of press releases every year (available here http://group.bmj.com/group/media/bmj-press-releases). If one slips below the usual standard we want to know about it, and we appreciate the work you and others do to keep our feet to the fire.

Just a few points of clarification:

* as you rightly said in your blog, the paper was not in The BMJ (http://www.bmj.com/) but in Annals of Rheumatic Diseases (http://ard.bmj.com/), one of more than 50 journals published by BMJ (the company http://company.bmj.com/)

* however, some of your audience – particularly on Twitter – have got the wrong end of the stick, thinking that the paper or press release were about The BMJ. Understandable, but worth correcting – I’ll do so on Twitter

* I check all press releases for research published in The BMJ and BMJ Open, the two journals I work for

* All press releases for all of BMJ’s journals are written by approved by the papers’ authors and are accompanied by the full paper, any linked editorial, and the authors’ contact details. See BMJ’s media release policy for details: http://www.bmj.com/about-bmj/resources-media

I’ll add that our press officers are very experienced and skilled, and that we all work hard to ensure good, accurate, transparent, and full reporting of research. And that I’ve responded here before to criticism: http://www.healthnewsreview.org/2012/03/bmj-news-releases-observational-studies-overstatement/
I hope that – as you did last time – you’ll read my response as a genuine acknowledgement of the constructive criticism, rather than defensive behaviour.

Trish

Dr Trish Groves
Head of Research, The BMJ
Editor-in-Chief, BMJ Open

Twitter/trished

    Gary Schwitzer posted on January 23, 2014 at 12:27 pm

    Trish,

    Thanks for your response and acknowledgement of our criticism.

    You are correct, I believe, that some folks on Twitter may have misinterpreted what I wrote, despite my attempt to be clear about the fact that the paper was not in The BMJ but in the Annals of Rheumatic Diseases – also published by BMJ (the company).

    It is perhaps inevitable that an occasional news release “slips below the usual standard.” But given the repeated instances I’ve highlighted on this topic of news releases about observational studies, I would hope that BMJ-group-wide training could reduce the chance and number of slips. And perhaps other steps could be taken to ensure that the news release is, indeed, a true reflection of the researchers’ work – including limitations.

Trish Groves posted on January 23, 2014 at 12:16 pm

Uh oh; I meant:

All press releases for all of BMJ’s journals are written by experienced press officers and then approved by the papers’ authors…

Sorry.

Trish

Trish Groves posted on January 23, 2014 at 3:32 pm

Here’s the article, with the press release posted alongside it:
http://ard.bmj.com/content/early/2014/01/07/annrheumdis-2013-204043.short?g=w_ard_ahead_tab

Trish

Terry Murray posted on February 4, 2014 at 12:31 am

Re comment by Wendy Sue Swanson that “it would behoove academic journals to require primary authors to submit a summary of their work (for the public).”

Possibly. But many of these authors are actually incapable of summarizing their work for a lay audience. I spent 10 years on the animal care committee of a major Toronto teaching hospital. We required researchers to provide a “lay summary” with their submission to the committee. In that time, I can count on the fingers of one hand the number of researchers who wrote lay summaries that included the species of animal they were were using, why they chose the species and clearly explained (in brief) what their research was about. I asked a researcher on the committee, who seemed to have particular difficulty with lay summaries, what he told his kids about his work. “It’s very hard,” he said.

Based on that and 35 years as a medical journalist trying to get researchers to speak in words of one syllable (figuratively), I don’t think the lay summary idea is going to work.