Health News Review

If you follow health care news in mainstream media, you’re going to be flooded with news from observational studies – research that is not a true experiment but, rather, what is seen by observing people doing different things over time. It’s valid and important research but one thing we can’t lose sight of:  such research CAN NOT PROVE CAUSE-AND-EFFECT.  It can only point to statistical associations, such as “It appears that people who do X are more likely to have Y happen.”  As a friend of mine wrote, the rooster crowing in the morning does not make the sun come up, even though a statistical association between the two coinciding is high.

Last week we saw stories about citrus fruits protecting women from stroke.  This week it was stories about “sleeping pills could kill 500,000.” This week we also had stories about “omega-3 fatty acids protecting the aging brain“….and about “Vitamin A may slash melanoma risk.” Sometimes it’s stories about lower risk (or protection), sometimes it’s stories about higher risk.

One thing is in common: almost all of the stories are simply wrong, using inaccurate language to describe the kinds of studies in question.

Week after week, year after year, for 6 years now, we have written about news stories that fail to explain the limitations of observational studies to readers. They use causal language – suggesting cause-and-effect findings – for studies that cannot prove cause-and-effect.

We don’t just criticize; we try to offer help.  In that spirit, we offer a primer on our site entitled:

Does the Language Fit The Evidence?  Association Versus Causation.” 

In it, we give examples of studies and examples of the words that are the only ways to accurately these studies.

The problems, the inaccuracies, the criticism would go away if journalists would simply read and act on this advice.

These are the kinds of stories that contribute to the background noise in the daily drumbeat of health care news stories – perhaps leading many consumers to be overwhelmed and to lose confidence in science and in journalism.

 

 

 

Comments

Merrill Goozner posted on March 2, 2012 at 10:17 am

Whooaaa, nellie. You write “such research CAN NOT PROVE CAUSE AND EFFECT” (YOUR EMPHASIS). I beg to differ. Very carefully controlled observational studies (doing everything possible to rule out confounding variables) has proven very useful over the years, whether in proving the harm caused by drugs (David Graham’s study on Vioxx) to the original studies that showed smoking causes cancer. To make your valid point that too many poorly done observational studies get reported in the press, you’ve (forgive the cliche) thrown the baby out with the bathwater.

    Gary Schwitzer posted on March 2, 2012 at 12:33 pm

    Gooz,

    Don’t want to split hairs with an old friend, but I must.

    I know the value of and history of observational studies.

    But, in reality, they cannot prove cause-and-effect. Yes, they can point to such a strong statistical association that little doubt is left.

    But, as you know, we try to improve the quality and accuracy of health care journalism, and journalists who use causal language about a study that can’t prove cause-and-effect are inaccurate and wrong.

    In making that point, we are not – by any means – throwing the baby out with the bathwater. Our respect for the role and importance of observational studies is intact.

    Over and over and over – over 1,700 stories reviewed in 6 years’ time – we have seen stories get this wrong. They – and the public they serve – need to understand that not all studies are equal. We’re trying to do what we can to lift all ships and improve critical thinking and evaluation of the hierarchy of evidence.

      Billy Rubin posted on March 4, 2012 at 8:23 am

      Gary’s right. While you can make major inferences with well-conducted observational studies, you can never -prove- cause-and-effect with them. There are a ton of good examples, but the best example in recent years is the saga of Hormone Replacement Therapy: there were dozens of well-designed trials showing HRT reduced cardiac deaths, overall mortality etc. and it was taken as gospel that HRT was beneficial. Only when they designed trials prospectively did they discover that no mortality benefits existed, and indeed there was (a little) harm.

Gregory D. Pawelski posted on March 2, 2012 at 2:13 pm

We tend to forget that medicine and most of its discoveries have been observational. Observational studies, which do not involve randomization but where available data are nonetheless analyzed to make treatment comparisons, have also been used to provide information on how well patients respond to treatment. Many investigators perform these types of studies by analyzing data from the Surveillance, Epidemiology and End Results (SEER) Registry.

Evidence based medicine, since the 1970′s, depended upon the randomized, controlled trial. It rests upon the assumption that evidence should be determined and applied as a basis for medical decision-making. Evidence is based upon quantities, similarities, populations and averages, rather than qualities, idiosyncrasies, individualization and specifics. It would be surprising if the most ardent supporter of evidence-based medicine would ever advocate a randomized trial for an intervention in which an observational study showed remarkable efficacy in preventing a near death situation.

Many major medical advances have never been subjected to a prospective randomized study before being introduced into routine management because their beneficial effects have been obvious. Recognizing the reliability of the evidence upon which clinical practice has increasingly come to depend, the time has come for physicians to reassess the value of direct observation and to trust more readily both the empirical and intuitive discoveries they make each day in their personal experience.

    Gary Schwitzer posted on March 2, 2012 at 7:27 pm

    Gregory,

    Thanks for your note, with which I agree.

    To reiterate, if necessary, I was commenting on the language of journalism, not the inherent value of observational studies.

David Egilman posted on March 5, 2012 at 6:44 am

if adopted by the press this misunderstanding of science would contribute to a disaster and would be a beloved construct for corporate America which has been spending billions to push this construct. Ong and Glantz | Peer Reviewed | Tobacco, Lawyers, and Public Health | 1749. November 2001, Vol 91, No. 11 | American Journal of Public Health. Ong and Glantz. Over a Barrel:Corporate Corruption of Science and Its Effects on Workers and the Environment DAVID S. EGILMAN, MD, MPH, SUSANNA RANKIN BOHME, AM IJOEM.com.

ALL environmental health research is observational. You do not do smoking or asbestos RCTs.

If you want to teach reporters about causation have them read Causation and Causal Inference in Epidemiology Kenneth J. Rothman, DrPH, Sander Greenland, MA, MS, DrPH, C Stat AJPH.and this http://aje.oxfordjournals.org/content/141/2/90.full.pdf+html (Causes by Rothman)

Some tips: The sum of all causes is INFINITY not 100. A rr less than 1 can show a causal relationship. Against Anti-health Epidemiology:Corporate Obstruction of Public Health via Manipulation of Epidemiology. DAVID EGILMAN MD, MPH, SAMANTHA HOWE IJOEH.com

    Gary Schwitzer posted on March 5, 2012 at 8:23 am

    David,

    Thanks for your note. I respect your opinion.

    I’m not talking about constructs. I’m talking about inaccurate language in news stories. Period. It’s no knock at the science.

    But since you raised “a disaster,” I would suggest we already have one: inaccuracies of the type we address contribute to the confusion of many Americans drowning in the tsunami of inaccurate health care news/information that floods them daily. We’ve not seen any disaster unfold when we’ve seen stories that accurately describe observational studies.

    For more than 6 years and across 1,700+ stories reviewed, we have worked daily to lift all ships by trying to improve the state of health care journalism and the flow of accurate, balanced and complete news/information to consumers.

    This isn’t a casual pursuit.