Health News Review

A WebMD news release last week announced that “WebMD is the First Choice for Consumers and Physicians Seeking Information on the Top Health and Wellness Topics.” The release went on to state that:

WebMD attracted the most visitors of any health information site in September and is again number one in page views in the health category.

That may be so, but our independent analysis of the quality of WebMD health news coverage does not put WebMD news at the top of the quality list. In fact, it’s almost at the bottom of our list.  Granted, there are other types of content that WebMD delivers, and we only evaluate news stories.  But the interactive chart below – available to anyone on our site – shows that WebMD is below the average of all the news sources we’ve reviewed in the past 6.5 years.

WebMD’s average score of 2.78 stars (out of a possible 5) is lower than the 6.5 year average across 1800+ stories of 3.06 stars.

 

Rankings for visits and page views are good to know if you’re an advertiser. It might not be a good measure of quality of news coverage.  We’re not picking on WebMD.  But if you’re going to make claims about rankings, you have to be ready to have those claims scrutinized.

Here is a quick update on some other average star score grades of other news organizations:

News source Average star score (out of possible 5)
Associated Press (n=247) 3.59
NPR.org (n=42) 3.57
Los Angeles Times (n=140) 3.51
New York Times (n=124) 3.47
USA Today (n=56) 3.43
Wall Street Journal (n=106) 3.4
Reuters (n=132) 3.26
Washington Post (n=43) 3.23
Chicago Tribune (n=26) 3.08
All 1,813 stories reviewed over 6.5 years 3.06 OVERALL AVERAGE
CNN.com (n=42) 2.98
HealthDay (n=178) 2.86
TIME (n=27) 2.85
WebMD (n=130) 2.78
MSNBC (n=13) 2.38

A few notes about these “scores” –

  • Our sampling is imperfect.  It’s the best we can do.  We don’t see everything.  We don’t review everything we see. Our funding has been cut back this year, reducing our volume even further.  And our funding is in jeopardy for the future.  Anyone with suggestions or deep-pocketed sources of industry-free donations – please contact us.
  • We review only what we can find online.  We don’t subscribe to the dead-tree printed versions of the newspapers above. Some news organizations make it very difficult to find their health news online. That’s why some leading news organizations don’t appear on the list.
  • We review these entities’ blogs as well as their other publications.  Why wouldn’t we?  We all find them the same way – undifferentiated online without caveats about “Oh, this is only a blog…don’t expect us to do critical analysis.”
  • Big overall reminder:  we only review stories that include claims about interventions.

Comments

Scott Hensley posted on October 23, 2012 at 1:22 pm

Thanks for doing this. We at NPR try to be mindful about the criteria your reviewers use, and it’s very helpful to be reminded about how we’re doing.

Mary posted on October 23, 2012 at 8:34 pm

This is hardly a fair comparison, as WebMD is not primarily a news outlet, although the other ranked organizations are. Medscape News may provide true news, but the rest of the sites (WebMD for consumers, Medscape Reference for clinicians) are more *information* providers, not news providers.

    Gary Schwitzer posted on October 23, 2012 at 8:51 pm

    Mary,

    With all due respect, I disagree.

    This is a news comparison, head-to-head. We didn’t compare their non-news items with other organizations’ news items. It was their news – what they labeled as news – versus what other people labeled as news. The comparison couldn’t be more fair. It’s real journalists paid real money posting real stories at WebMD.

    By your argument, perhaps NPR shouldn’t be put in the mix, because much of its programming is non-news (Ask Me Another, Car Talk, Radiolab, music, special series, etc.) Yet NPR ranks second – very close to first.

    We believe that if you hang out your shingle saying you’re doing news, you better be ready to be judged alongside all the other folks hanging out similar shingles.