Columbia Journalism Review cover story on health journalism that “grossly mislead(s) the public”

The CJR cover story is entitled: ” ‘Survival of the wrongest’: How personal-health journalism ignores the fundamental pitfalls baked into all scientific research and serves up a daily diet of unreliable information.”

It’s written by David H. Freedman, a contributing editor at The Atlantic, and a consulting editor at Johns Hopkins Medicine International and at the McGill University Desautels Faculty of Management.

Highlights:

• He leads by slamming a New York Times piece by Tara-Parker Pope:  “In short, it’s a well-reported, well-written, highly readable, and convincing piece of personal-health-science journalism that is careful to pin its claims to published research. There’s really just one problem with Parker-Pope’s piece: Many, if not most, researchers and experts who work closely with the overweight and obese would pronounce its main thesis—that sustaining weight loss is nearly impossible—dead wrong, and misleading in a way that could seriously, if indirectly, damage the health of millions of people.”

• His broader view:
  • “In all areas of personal health, we see prominent media reports that directly oppose well-established knowledge in the field, or that make it sound as if scientifically unresolved questions have been resolved. The media, for instance, have variously supported and shot down the notion that vitamin D supplements can protect against cancer, and that taking daily and low doses of aspirin extends life by protecting against heart attacks. Some reports have argued that frequent consumption of even modest amounts of alcohol leads to serious health risks, while others have reported that daily moderate alcohol consumption can be a healthy substitute for exercise. Articles sang the praises of new drugs like Avastin and Avandia before other articles deemed them dangerous, ineffective, or both.What’s going on? The problem is not, as many would reflexively assume, the sloppiness of poorly trained science writers looking for sensational headlines, and ignoring scientific evidence in the process. Many of these articles were written by celebrated health-science journalists and published in respected magazines and newspapers; their arguments were backed up with what appears to be solid, balanced reporting and the careful citing of published scientific findings.But personal-health journalists have fallen into a trap. Even while following what are considered the guidelines of good science reporting, they still manage to write articles that grossly mislead the public, often in ways that can lead to poor health decisions with catastrophic consequences. Blame a combination of the special nature of health advice, serious challenges in medical research, and the failure of science journalism to scrutinize the research it covers.”

•  He then includes a nice plug for our work:
  • “Personal-health coverage began to move to the fore in the late 1980s, in line with the media’s growing emphasis on “news you can use.” That increased attention to personal health ate into coverage of not only other science, but also of broader healthcare issues. A 2009 survey of members of the Association of Health Care Journalists found that more than half say “there is too much coverage of consumer or lifestyle health,” and more than two-thirds say there isn’t enough coverage of health policy, healthcare quality, and health disparities. The author of a report based on that survey, Gary Schwitzer, a former University of Minnesota journalism researcher and now publisher of healthcare-journalism watchdog HealthNewsReview.org, also conducted a study in 2008 of 500 health-related stories published over a 22-month period in large newspapers. The results suggested that not only has personal-health coverage become invasively and inappropriately ubiquitous, it is of generally questionable quality, with about two-thirds of the articles found to have major flaws. The errors included exaggerating the prevalence and ravages of a disorder, ignoring potential side effects and other downsides to treatments, and failing to discuss alternative treatment options. In the survey, 44 percent of the 256 staff journalists who responded said that their organizations at times base stories almost entirely on press releases. Studies by other researchers have come to similar conclusions.”

One thing Freedman doesn’t do – and it’s something we strive for in our project – is to include any examples of journalism that gets it right.  We go out of our way to shine a home-page spotlight on top-rated 5-star stories.

While only 15% of the 1,846 stories we’ve reviewed in the past almost-seven years have received top 5-star scores from our reviewers, we can learn from those who work hard to attain excellence.   (The star scores are a reflection of how many of our 10 criteria are judged satisfactory.)

We also work hard to help journalists improve, by emailing them whenever one of their stories is reviewed, and by making presentations and leading workshops through the NIH Medicine in the Media series, the MIT Medical Evidence Boot Camps, the Association of Health Care Journalists annual conferences (and some local chapter workshops in the past), international journalist workshops with the National Cancer Institute, and more.  We offer a toolkit with primers for understanding studies.  We still anger some journalists who are not accustomed to having their work reviewed, or who think they are above criticism.  But those who want to learn and improve have done so by following our work.

Freedman’s piece is likely to upset many more journalists.  His central theme is sound.  We wish he’d been given, or taken, more space to reflect on journalists who can be good examples for others.

Addendum on January 9:  Paul Raeburn analyzed the Freedman CJR piece on the Knight Science Journalism Tracker.

Addendum on January 11:  The Respectful Insolence blog also criticized the Freedman CJR piece.

———————————

Follow us on Facebook,

and on Twitter:

https://twitter.com/garyschwitzer

https://twitter.com/healthnewsrevu

Comments

Daniel Pendick posted on January 4, 2013 at 7:39 am

I was disappointed and a little surprised by some of the naive and simplistic thinking in this article–starting with the idea that reporters should somehow figure out what most doctors think is good for patients and make sure our work remains in line with that. In that case, why bother? Why have journalists at all? All we would have is the Fox News of health, endlessly repeating a pre-prepared narrative, written by the AMA, of what most doctors think is good for us. Frankly, there is quite enough falsely upbeat cheerleading going on in consumer health writing already.

On the plus side, Freedman calls health health journalism out (correctly) for thoughtless “new study says” hyperbole. But when Freedman starts talking about “wrongness” in the medical literature…. aiy, Lucy! Do we really turn to medical journals to find out what is “right” and “wrong”? Is that how science works?

And in the critique of Parker-Pope, when he stoops to “most of us know people…who have lost weight and kept it off for years,” he is simply falling for the oldest (and wrongest) trick in the book: that the plural of anecdote is data.

It is quite possibly true that most people have a very, very hard time getting back to a healthy weight once they are obese. At its root, that’s what Pope’s story says. It is useful and HELPFUL to know that if you are obese. It means you are up against a tough job, so expect a struggle. Is it better to lie to and soothe readers? Is that our job? (Again, imagine the Fox News model of health reporting…)

Look, I’ve spent many hours on the phone with scientists discussing why a particular study in JAMA or NEJM has an important methodological issue that calls into question its findings and, more importantly, its relevance to clinical practice or to real patients. Not being willing or able to have that sort of conversation is at the root of the problem–not the “wrongness” of journals.

This is what Freedman missed: Many, many studies that are perfectly legitimate pieces of research do NOT reach the standard for something that can, does, or should alter clinical practice, or should alter consumer choices. Not making that distinction is where many reporters drop the ball. (I know, because I’ve done it plenty of times!) They should report those studies AS RESEARCH, not as health advice.

Second, not knowing an inflated relative risk from a hole in the ground is another serious problem. I would have preferred more specific criticism of these very real and relevant problems from Freedman.

Lastly, Gary Schwitzer is quite right: We should also hear about the health journalists who are doing a good job. This article offers a very simplistic and unbalanced view of health journalism. There are good plumbers, and there are bad plumbers. The same is true for us.