October 23, 2012
Rating page views versus rating news quality: 2 entirely different metrics
A WebMD news release last week announced that “WebMD is the First Choice for Consumers and Physicians Seeking Information on the Top Health and Wellness Topics.” The release went on to state that:
WebMD attracted the most visitors of any health information site in September and is again number one in page views in the health category.
That may be so, but our independent analysis of the quality of WebMD health news coverage does not put WebMD news at the top of the quality list. In fact, it’s almost at the bottom of our list. Granted, there are other types of content that WebMD delivers, and we only evaluate news stories. But the interactive chart below – available to anyone on our site – shows that WebMD is below the average of all the news sources we’ve reviewed in the past 6.5 years.
WebMD’s average score of 2.78 stars (out of a possible 5) is lower than the 6.5 year average across 1800+ stories of 3.06 stars.
Rankings for visits and page views are good to know if you’re an advertiser. It might not be a good measure of quality of news coverage. We’re not picking on WebMD. But if you’re going to make claims about rankings, you have to be ready to have those claims scrutinized.
Here is a quick update on some other average star score grades of other news organizations:
| News source |
Average star score (out of possible 5) |
| Associated Press (n=247) |
3.59 |
| NPR.org (n=42) |
3.57 |
| Los Angeles Times (n=140) |
3.51 |
| New York Times (n=124) |
3.47 |
| USA Today (n=56) |
3.43 |
| Wall Street Journal (n=106) |
3.4 |
| Reuters (n=132) |
3.26 |
| Washington Post (n=43) |
3.23 |
| Chicago Tribune (n=26) |
3.08 |
| All 1,813 stories reviewed over 6.5 years |
3.06 OVERALL AVERAGE |
| CNN.com (n=42) |
2.98 |
| HealthDay (n=178) |
2.86 |
| TIME (n=27) |
2.85 |
| WebMD (n=130) |
2.78 |
| MSNBC (n=13) |
2.38 |
A few notes about these “scores” —
- Our sampling is imperfect. It’s the best we can do. We don’t see everything. We don’t review everything we see. Our funding has been cut back this year, reducing our volume even further. And our funding is in jeopardy for the future. Anyone with suggestions or deep-pocketed sources of industry-free donations – please contact us.
- We review only what we can find online. We don’t subscribe to the dead-tree printed versions of the newspapers above. Some news organizations make it very difficult to find their health news online. That’s why some leading news organizations don’t appear on the list.
- We review these entities’ blogs as well as their other publications. Why wouldn’t we? We all find them the same way – undifferentiated online without caveats about “Oh, this is only a blog…don’t expect us to do critical analysis.”
- Big overall reminder: we only review stories that include claims about interventions.
Comments (6)
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Scott Hensley
October 23, 2012 at 1:22 pmThanks for doing this. We at NPR try to be mindful about the criteria your reviewers use, and it’s very helpful to be reminded about how we’re doing.
Mary
October 23, 2012 at 8:34 pmThis is hardly a fair comparison, as WebMD is not primarily a news outlet, although the other ranked organizations are. Medscape News may provide true news, but the rest of the sites (WebMD for consumers, Medscape Reference for clinicians) are more *information* providers, not news providers.
Gary Schwitzer
October 23, 2012 at 8:51 pmMary,
With all due respect, I disagree.
This is a news comparison, head-to-head. We didn’t compare their non-news items with other organizations’ news items. It was their news – what they labeled as news – versus what other people labeled as news. The comparison couldn’t be more fair. It’s real journalists paid real money posting real stories at WebMD.
By your argument, perhaps NPR shouldn’t be put in the mix, because much of its programming is non-news (Ask Me Another, Car Talk, Radiolab, music, special series, etc.) Yet NPR ranks second – very close to first.
We believe that if you hang out your shingle saying you’re doing news, you better be ready to be judged alongside all the other folks hanging out similar shingles.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like