Please note: We stopped doing systematic criteria-driven reviews of news stories and PR news releases in December, 2018 when we lost all external funding. But all past reviews remain accessible on the site.
Between early 2006 and the end of 2018, we systematically reviewed 2,616 news stories. Between the start of 2015 until the end of 2018, we also systematically reviewed 606 public relations news releases as well. Click here for how those criteria slightly differ.
Our review criteria consist of 10 different elements that we think all health care news stories and all health care news (press) releases about interventions should include.
We think these criteria address the basic issues that consumers need to know in order to develop informed opinions about these interventions–and how/whether they matter in their lives. In some cases, it may be impossible or unreasonable for a story/release to address some of our criteria, in which case we’ll say so in our comments. We may rate certain such criteria as not applicable in some cases.
News Story Review Criteria
Our longterm experience with thousands of stories shows that the cost criterion gets the most unsatisfactory scores. News stories don’t help patients and consumers if they don’t tackle costs.
New healthcare treatments should work well. When reading a story, people want to know: How effective is the intervention? Do the numbers back it up?
In the news, if you’re only hearing about the potential benefits of a test or treatment, run for the hills. What are the harms, and how often do they occur?
Ideally, healthcare interventions are subjected to rigorous testing to prove they work. When reading about a new intervention, device or diagnostic tool, people should come away with a sense of how rigorous the evidence is for the intervention.
Sometimes new conditions or diseases actually may just be opportunities to medicalize (and monetize) normal states of health. Readers aren’t well-served by stories that exaggerate or oversell conditions.
Health care is rife with conflicts of interest. Readers deserve to know if the sources in the story have any conflicts of interest. They also deserve to hear the opinion of objective, independent third-party experts, who are vital to backstopping exaggerated claims.
Just because something is “new” in health care doesn’t mean it’s better. When reading about a new drug, test, surgery, etc., readers should come away informed that there are other options on the table.
New medical treatments often get lots of attention before they’re ever made available to the public. After reading a story about a new intervention, a person should be able to answer: Is it actually available to me?
Often something that’s lauded as new or novel really isn’t. Or it may be new, but it’s not better. So, when reading about a “new” medical intervention, people deserve to know: What is truly new and different about this idea?
Sometimes news stories rely solely or largely on a PR news release–and never disclose that the information came from a news release. Readers deserve to know if a story includes no original reporting.
News Release Review Criteria
The 10 criteria used to review news releases are almost identical to those above.
Criterion #6 is slightly different for news release reviews. It gives this guidance:
Does the news release identify funding sources & disclose conflicts of interest?
Potential conflicts of interests should be disclosed in an editor’s note included in the release. The funding for the project must also be disclosed along with any relationship that the investigators may have with that funder.
Another significant difference for #6 regards the use of independent sources: Unlike news stories, news releases are not expected to feature comment from an objective expert.
The other key difference is Criterion #10. That final criterion for news release reviews is this:
Does the news release include unjustifiable, sensational language, including in the quotes of researchers?
Red flags rise up in our eyes when we see terms such as:
- could become the new standard of care
- Holy Grail
- simple blood test
- this might/may/could lead to (which means, of course, that “might/may/could not” applies as well)
Sources: Our criteria are at least partially influenced by, and appear in, the following seminal publications:
- Coverage by the News Media of the Benefits and Risks of Medications. Ray Moynihan, B.A., Lisa Bero, Ph.D., Dennis Ross-Degnan, Sc.D., David Henry, M.B., Ch.B., Kirby Lee, M.A., Judy Watkins, B.A., Connie Mah, B.A., and Stephen B. Soumerai, Sc.D.N Engl J Med 2000; 342:1645-1650 June 1, 2000 DOI: 10.1056/NEJM200006013422206