The CJR cover story is entitled: ” Survival of the wrongest’: How personal-health journalism ignores the fundamental pitfalls baked into all scientific research and serves up a daily diet of unreliable information.”
It’s written by David H. Freedman, a contributing editor at The Atlantic, and a consulting editor at Johns Hopkins Medicine International and at the McGill University Desautels Faculty of Management.
Highlights:
- He leads by slamming a New York Times piece by Tara-Parker Pope: “In short, it’s a well-reported, well-written, highly readable, and convincing piece of personal-health-science journalism that is careful to pin its claims to published research. There’s really just one problem with Parker-Pope’s piece: Many, if not most, researchers and experts who work closely with the overweight and obese would pronounce its main thesis—that sustaining weight loss is nearly impossible—dead wrong, and misleading in a way that could seriously, if indirectly, damage the health of millions of people.”
- His broader view:
- “In all areas of personal health, we see prominent media reports that directly oppose well-established knowledge in the field, or that make it sound as if scientifically unresolved questions have been resolved. The media, for instance, have variously supported and shot down the notion that vitamin D supplements can protect against cancer, and that taking daily and low doses of aspirin extends life by protecting against heart attacks. Some reports have argued that frequent consumption of even modest amounts of alcohol leads to serious health risks, while others have reported that daily moderate alcohol consumption can be a healthy substitute for exercise. Articles sang the praises of new drugs like Avastin and Avandia before other articles deemed them dangerous, ineffective, or both.What’s going on? The problem is not, as many would reflexively assume, the sloppiness of poorly trained science writers looking for sensational headlines, and ignoring scientific evidence in the process. Many of these articles were written by celebrated health-science journalists and published in respected magazines and newspapers; their arguments were backed up with what appears to be solid, balanced reporting and the careful citing of published scientific findings.But personal-health journalists have fallen into a trap. Even while following what are considered the guidelines of good science reporting, they still manage to write articles that grossly mislead the public, often in ways that can lead to poor health decisions with catastrophic consequences. Blame a combination of the special nature of health advice, serious challenges in medical research, and the failure of science journalism to scrutinize the research it covers.”
- He then includes a nice plug for our work:
- “Personal-health coverage began to move to the fore in the late 1980s, in line with the media’s growing emphasis on “news you can use.” That increased attention to personal health ate into coverage of not only other science, but also of broader healthcare issues. A 2009 survey of members of the Association of Health Care Journalists found that more than half say “there is too much coverage of consumer or lifestyle health,” and more than two-thirds say there isn’t enough coverage of health policy, healthcare quality, and health disparities. The author of a report based on that survey, Gary Schwitzer, a former University of Minnesota journalism researcher and now publisher of healthcare-journalism watchdog HealthNewsReview.org, also conducted a study in 2008 of 500 health-related stories published over a 22-month period in large newspapers. The results suggested that not only has personal-health coverage become invasively and inappropriately ubiquitous, it is of generally questionable quality, with about two-thirds of the articles found to have major flaws. The errors included exaggerating the prevalence and ravages of a disorder, ignoring potential side effects and other downsides to treatments, and failing to discuss alternative treatment options. In the survey, 44 percent of the 256 staff journalists who responded said that their organizations at times base stories almost entirely on press releases. Studies by other researchers have come to similar conclusions.”
One thing Freedman doesn’t do – and it’s something we strive for in our project – is to include any examples of journalism that gets it right. We go out of our way to shine a home-page spotlight on top-rated 5-star stories.
While only 15% of the 1,846 stories we’ve reviewed in the past almost-seven years have received top 5-star scores from our reviewers, we can learn from those who work hard to attain excellence. (The star scores are a reflection of how many of our 10 criteria are judged satisfactory.)
| 7 years of our work |
| 0 star |
68 |
| 1 star |
139 |
| 2 star |
410 |
| 3 star |
496 |
| 4 star |
461 |
| 5 star |
272 |
| Total |
1846 |
We also work hard to help journalists improve, by emailing them whenever one of their stories is reviewed, and by making presentations and leading workshops through the NIH Medicine in the Media series, the MIT Medical Evidence Boot Camps, the Association of Health Care Journalists annual conferences (and some local chapter workshops in the past), international journalist workshops with the National Cancer Institute, and more. We offer a toolkit with primers for understanding studies. We still anger some journalists who are not accustomed to having their work reviewed, or who think they are above criticism. But those who want to learn and improve have done so by following our work.
Freedman’s piece is likely to upset many more journalists. His central theme is sound. We wish he’d been given, or taken, more space to reflect on journalists who can be good examples for others.
Addendum on January 9: Paul Raeburn analyzed the Freedman CJR piece on the Knight Science Journalism Tracker.
Addendum on January 11: The Respectful Insolence blog also criticized the Freedman CJR piece.
———————————
Follow us on Facebook,
and on Twitter:
https://twitter.com/garyschwitzer
https://twitter.com/healthnewsrevu
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Paul Scott
January 2, 2013 at 10:45 pmWhile I see lots of problems in health writing, I found that article to be wrong and misguided in more ways to count. He seems to equate prevailing medical wisdom as the final word on whether an author is on the right track or not. He is flat out wrong in his understanding of nutrition and his commentary on what is wrong with RCTs reads like someone very very mew to the subject. Anyone who casually dismisses Gary Taubes doesn’t seem to have surveyed the landscape very carefully.
Daniel Pendick
January 4, 2013 at 7:39 amI was disappointed and a little surprised by some of the naive and simplistic thinking in this article–starting with the idea that reporters should somehow figure out what most doctors think is good for patients and make sure our work remains in line with that. In that case, why bother? Why have journalists at all? All we would have is the Fox News of health, endlessly repeating a pre-prepared narrative, written by the AMA, of what most doctors think is good for us. Frankly, there is quite enough falsely upbeat cheerleading going on in consumer health writing already.
On the plus side, Freedman calls health health journalism out (correctly) for thoughtless “new study says” hyperbole. But when Freedman starts talking about “wrongness” in the medical literature…. aiy, Lucy! Do we really turn to medical journals to find out what is “right” and “wrong”? Is that how science works?
And in the critique of Parker-Pope, when he stoops to “most of us know people…who have lost weight and kept it off for years,” he is simply falling for the oldest (and wrongest) trick in the book: that the plural of anecdote is data.
It is quite possibly true that most people have a very, very hard time getting back to a healthy weight once they are obese. At its root, that’s what Pope’s story says. It is useful and HELPFUL to know that if you are obese. It means you are up against a tough job, so expect a struggle. Is it better to lie to and soothe readers? Is that our job? (Again, imagine the Fox News model of health reporting…)
Look, I’ve spent many hours on the phone with scientists discussing why a particular study in JAMA or NEJM has an important methodological issue that calls into question its findings and, more importantly, its relevance to clinical practice or to real patients. Not being willing or able to have that sort of conversation is at the root of the problem–not the “wrongness” of journals.
This is what Freedman missed: Many, many studies that are perfectly legitimate pieces of research do NOT reach the standard for something that can, does, or should alter clinical practice, or should alter consumer choices. Not making that distinction is where many reporters drop the ball. (I know, because I’ve done it plenty of times!) They should report those studies AS RESEARCH, not as health advice.
Second, not knowing an inflated relative risk from a hole in the ground is another serious problem. I would have preferred more specific criticism of these very real and relevant problems from Freedman.
Lastly, Gary Schwitzer is quite right: We should also hear about the health journalists who are doing a good job. This article offers a very simplistic and unbalanced view of health journalism. There are good plumbers, and there are bad plumbers. The same is true for us.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like