11th birthday for the HealthNewsReview.org watchdog

5,200 articles – all of them trying to help people improve their critical thinking about health care interventions.  A team of about 50 contributors – almost all of them part-time.  An international reach with a growing audience.

There’s much to celebrate on the 11th anniversary of the launch of HealthNewsReview.org.  A screenshot of our home page on our very first day appears below. It is clear that from the beginning our mission – what we do and why it’s important – was not only to support excellence in health care journalism but to support consumers’ informed decision-making by helping them improve their critical thinking about health care claims. That’s what we’ve tried to do more than 5,000 times in 11 years.

Our bread-and-butter product is our systematic review and grading of news stories and news releases that include claims about interventions.  We apply 10 criteria to the review of such articles, but five of them are arguably the most important, and the ones for which writers get the poorest grades.  See the report card below.

Some users want to know which news organizations seem to do best, and vice versa.  We offer one way of looking at, and comparing performances of, news organizations only.  (No such tool yet for PR news release reviews.)  The interactive tool is available online.  It is imperfect and incomplete in some ways, but is, nonetheless, a glimpse of performance.  The grade that an article gets based on our 10 criteria is translated into a 0 – 5 star score, like a movie or book review.

In the “Sort by News Source” section of that page, from the pulldown menu, you can choose the news organizations whose performance you want to check.  Some that outperform the 3.09-star national average (in our sample) include:

• Vox.com with a 4.4-star average
• FiveThirtyEight.com with a 4.17 star average
• STAT news with a 4-star average

With a much larger sample over time, the Associated Press has an average grade of 3.66 stars, the New York Times and the Wall Street Journal are tied with average grades of 3.53 stars, the Los Angeles Times’ average grade is 3.45 stars, and NPR has an average grade over time of 3.43 stars.

Below the national average in our sample are:

• FoxNews.com with an average grade of 2.13 stars
• CBSNews.com with an average of 2.53 stars
• the Chicago Tribune with 2.76
• TIME magazine with 2.77

What doesn’t show up in the stats are the number of writers and news consumers (and health care consumers) who understand what we’re trying to do and get the message – and even change as a result.  This past year, writers who reacted to our constructive criticism wrote things such as:

• “I am disappointed with the rating – we are very conscious of also supporting high quality scientific reporting and it is something we actively check when we work with authors etc. It’s really useful for me to see the feedback and comments from external parties and we will use your 10 points to check our releases against in future.”

• “I corrected the error that you pointed out. Thanks for catching that. In a meeting with my editor today, we were debriefing about this article. She said, “I hope we get 5 stars from HealthNewsReview.org.” ;)  So thank you!  You made our week. The work you do is so important holding us to account, and these reviews really do matter to us.”

And a consumer-follower wrote:

“I deeply appreciate the efforts of those who invest themselves in shedding light on the untruths left out of mainstream media and even official studies. It’s easy to think about how little difference it may make to a society that is largely conditioned to accept what they’re being told. People like you who continue to work against the odds are why there is hope in the world. The concept of evil prospering when good people do nothing is why what you and others do, to shine light on truth and expose lies, matters so much. Best wishes in all your efforts to provide truth to those who are vulnerable to the misinformation so often found in medical news reports.”

We don’t need many messages like that to inspire us to keep going.  And we will keep going.  So stay tuned, and thanks for your continued interest and support. And special thanks to the Laura and John Arnold Foundation for their support since 2015.

Other recent articles about or by us:

Campaigning for a fact-based approach to health journalism. Bulletin of World Health Organization.

Pollution of health news: Time to drain the swamp.  Editorial in The BMJ. 

One Step Forward, One Step Back: Changes in News Coverage of Medical Interventions. Health Communication.