As another full year of our analysis of health care news & PR wraps up soon, it is encouraging to see 3 articles in one week in the published literature address issues in health care journalism and PR – themes that we have addressed for 10 years. Each of the three papers delivers a different aspect of a report card to media messengers.
In another blog post today, Matt Shipman analyzed a PLoS One journal article, “Exaggerations and caveats in press releases and health-related science news.”
But two other papers published within the past week are also noteworthy:
The researchers found that only 1 in 6 news stories analyzed included comments from independent experts. Of the 104 comments from experts analyzed in the study, a financial conflict of interest was present in 32% of the comments, of which only 33% were reported in the story.
One of the 10 criteria that our project uses when we review health care news stories that include claims about interventions is double-barreled, looking for two things in order to get a satisfactory score: Does the story use independent sources and identify conflicts of interest? Of the 2,300 stories we’ve reviewed, only about half get a satisfactory score on this criterion. That’s a huge sample, probably the largest such sample of stories about health care interventions that anyone has analyzed – so you can judge whether a 50-50 score across the huge sample is a case of the glass half full or half empty. I see it as half empty because the message still isn’t hitting home with many journalists.
The authors of the CMAJ paper reference the Statement of Principles of the Association of Health Care Journalists (AHCJ), which I wrote. But a code of ethics or statement of principles isn’t worth the paper it’s printed on if it doesn’t become a fundamental part of the fabric of a newsroom or of an individual journalist’s modus operandi. And that clearly still isn’t happening enough. Many AHCJ members are the ones who write the stories that we review, giving them unsatisfactory scores. In our experience, when you combine journalists’ performance on COI/independent sources with how they’ve done in emphasizing/exaggerating benefits while minimizing/ignoring potential harms – and costs – you see why it’s a much bigger and worse report card. A recent breakdown:
To help journalists we provide a list of industry-independent experts who are willing to work with journalists. The list is currently being updated, revised and expanded. So, whereas there are about 100 names on the list now, we may double that in the next year.
Our message to journalists covering health care: if you don’t know the landscape and aren’t aware of the prevalence of conflicts of interest, or if you know this but fail to report on it, you don’t have any business reporting on these issues. Because there’s a good chance you’re misleading your audience and hurting more than helping.
U of Florida Professor Kim Walsh-Childers approached me about doing this analysis about 4 years ago when I was facing the very real prospect of shutting down this project due to a loss of funding. Subsequently, the Laura and John Arnold Foundation breathed new life into our work with a new grant.
This paper’s analysis covers the early years of our work from 2005 (beta development that year) until 2013 – an analysis of our first 1,889 news story reviews. They found:
On average, the stories reviewed during 2005–2010 successfully met just less than half of the criteria, but by 2010–2013, that average had improved to almost 70%. There were significant improvements over time in news organizations’ success in meeting six of HNR’s 10 criteria. In addition, there was a statistically significant decline in the percentage of stories rated satisfactory on establishing the true novelty of the intervention discussed in the story. There was no improvement in quantification of possible harms from medical interventions.
If you’re wondering why the grades in my little table above look so bad, in comparison with what the U of Florida analysis showed, let me clarify. My table reports on only the 5 criteria for which journalists/writers have performed the worst. The other 5 criteria get markedly better average grades. In my view, the 5 that I highlighted carry a far greater weight than the ones not represented in my table. And the analysis suggesting average across-the-board improvement treats all criteria as if they had equal weight.
One consistent theme through the years: stories and news releases tend to minimize or completely ignore the potential harms of health care interventions. The Florida research team concluded:
Health journalists and the organizations they represent can increase their public value—and hence, perhaps, regain some public relevance—by consistently providing the comprehensive information consumers need to make important decisions about their own and their families’ health.