Our review criteria consist of 10 different elements that we think all health care news stories about interventions should include. We think these criteria address the basic issues that consumers need to know in order to develop informed opinions about these interventions – and how/whether they matter in their lives. In some cases, it may be impossible or unreasonable for a story to address some of our criteria, in which case we’ll say so in our comments. We may rate certain such criteria as not applicable in some cases.
Each of the criteria are at least partially influenced by, and appear in, the following seminal publications:
About 70% of the stories we review fail to satisfy this criterion.
We expect stories to give readers some sense – a quantitative estimate – of the scope of the potential benefit. Many stories:
Many stories emphasize or exaggerate potential benefits while minimizing or completely ignoring potential harms. They may:
We expect journalists to critically evaluate the evidence, not merely to take published or presented research as gospel. Many stories:
This criterion assesses whether a story exaggerates or over-sells a condition. There are different forms of “mongering” – turning risk factors into diseases (e.g., low bone mineral density becomes osteoporosis); misrepresentation of the natural history and/or severity of a disease (e.g., early-stage low-grade prostate cancer); medicalization of minor or transient variations in function (e.g. temporary erectile dysfunction or female ‘sexual dysfunction’); medicalization of normal states (baldness, wrinkles, shyness, menopause); or exaggeration of the prevalence of a disorder (e.g., using rating scales to ‘diagnose’ erectile dysfunction).
In order to get a satisfactory score on this criterion, two things must occur: There must be an independent expert source quoted (someone not directly connected with the research) and there must be some attempt to let the public know about potential conflicts of interest.
We understand that news wants to cover what’s new. But we think there’s an obligation to put the new into the context of existing alternatives that have a longer, more proven track record. Unsatisfactory stories may:
Does the story make a drug sound like it’s available at the corner drug store (or may be soon) when it’s really only in an early-phase trial? Does it feature one doctor promoting a technology without providing a sense of how many trained doctors there are, or what the learning curve is? Did it allow promoters to make unfounded predictions of FDA approval or widespread availability? There needs to be some sense of history – of the thousands of previous flashes in the pan in medical research that were equally exciting but never panned out.
Many “new” treatments, tests, products or procedures are not really novel. The agent may be another member of a well-established therapeutic class of drug; even if it represents a new class, it may offer no more than the drugs that are widely available. In the press release for a new drug this essential information may be lost in the hype and the drug is portrayed as “novel” in order to increase initial sales. We believe that journalists should be able to make these distinctions by doing their research or by asking independent sources.
It is simply not sound journalistic practice to lift material directly from a news release without stating that this is the case. This criterion will rarely be employed. If, on the front end, we know that a news release was sent out and if we can get a copy of that release, we will review it along with the story.
There are multiple ways to search our reviews. You may search by keyword, news source or review rating.