supporting excellence in health & medical news reporting

Health News Review

Things you should know about research stories

Absolute vs. Relative Risk

Radio commentator Paul Harvey has a feature called "The Rest of the Story." Researchers, clinicians or journalists who only report on relative differences in making claims about a new idea should tell the rest of the story. It is absolute differences that probably matter most to most people trying to make sense out of such claims.

Consider the risk for blindness in a patient with diabetes over a 5-year period. If the risk for blindness is 2 in 100 (2%) in a group of patients treated conventionally and 1 in 100 (1%) in patients treated with a new drug, the absolute difference is derived by simply subtracting the two risks: 2% - 1% = 1%.

Expressed as an absolute difference, the new drug reduces the 5-year risk for blindness by 1%.

The relative difference is the ratio of the two risks. Given the data above, the relative difference is:

1% ÷ 2% = 50%

Expressed as a relative difference, the new drug reduces the risk for blindness by half.

Each is accurate. But if your job is marketing manager for the new drug, you are likely to only use the relative risk reduction. If your job is journalist, you would serve your readers and viewers better by citing the raw data and pointing out the absolute risk reduction. That's the "rest of the story" often missing in news releases and direct-to-consumer prescription drug ads.

Number Needed to Treat

The number needed to treat, or NNT, is the number of patients who need to be treated to prevent one additional bad outcome, calculated as 1/Absolute Risk Reduction.

So, let's look at our hypothetical diabetes blindness drug example.

Let's say the risk for blindness in a patient with diabetes over a 5-year period is 2 in 100 (2%) in a group of patients treated conventionally and 1 in 100 (1%) in patients treated with a new drug. So the absolute difference is derived by simply subtracting the two risks: 2% - 1% = 1%.

The number needed to treat would be 1 / 1% (or .01) = 100.

So you would need to treat 100 people with diabetes for 5 years in order to prevent one case of blindness. You can see that this is an important way to look at new claims about new drugs.

This is a brief introduction to the concept of NNTs. More information, including different ways to calculate NNTs, is available here.

Single Source Stories

The Statement of the Principles of the Association of Health Care Journalists states that journalists should: "Recognize that most stories involve a degree of nuance and complexity that no single source could provide. Journalists have a responsibility to present diverse viewpoints in context. In addition, anyone with knowledge of the health care industry, of medicine, and of the scientific community knows that many vested interests reside among government health spokespersons, researchers, universities, drug companies, device manufacturers, providers, insurers and so on. To reflect only one perspective of only one source is not wise. Most one-source stories lack depth and meaning. Avoid single-source stories."

Consumers beware: if you read a single-source story, it's healthy to be skeptical about any claims made therein.

Commercialism

The U.S. spends more money per capita on health care than any other country. So it is not surprising that there are many commercial interests in the health care industry looking for good publicity from journalists.

The Statement of the Principles of the Association of Health Care Journalists (http://www.healthjournalism.org/files/AHCJ_principles.pdf ) states that journalists should:

Be vigilant in selecting sources, asking about, weighing and disclosing relevant financial, advocacy, personal or other interests of those we interview as a routine part of story research and interviews.
Investigate and report the possible links between sources of information (studies or experts) and those (such as the manufacturers) who promote a new idea or therapy. Investigate and report the possible links between researchers and private companies, researchers and public institutions, patient advocacy groups and their sponsors, celebrity spokespersons and their sponsors, non-profit health and professional organizations and their sponsors.

Here are several articles on commercialism in news coverage of drugs:

"Bitter Pill," by Trudy Lieberman, is available at: http://www.cjr.org/issues/2005/4/lieberman.asp.

"Celebrity Selling," by Ray Moynihan (2 parts), available at: http://bmj.bmjjournals.com/cgi/content/full/324/7349/1342 and http://bmj.bmjjournals.com/cgi/content/full/325/7358/286

"Commercialism in TV Health News," by Gary Schwitzer, is available at: http://www.poynter.org/content/content_view.asp?id=85652

FDA Approval Not Automatic

Many stories about drugs that are still in clinical trials include some estimate or projection of when the drug will be submitted to the Food and Drug Administration for approval, when the FDA might approve the drug, or when the drug might be available on the market. Many times, such projections are just shots in the dark. Until the trials are completed, until the data are submitted to the FDA for review, and until the FDA has made a decision, many of these predictions may be empty promises and may be proven wrong.

(See "How the media left the evidence out in the cold," at: http://bmj.bmjjournals.com/cgi/content/full/326/7403/1403)

If consumers hear or see stories that say a new drug "may be approved" or "could/should be approved" or "could be on the market" in some near time frame, don't put too much stock in that prediction. The drug may never be approved and may never be available.

Phases of Drug Trials

Journalists who report on drugs while they are still in clinical trials need to understand the distinction between Phases I, II, and III of drug trials. It is misleading to report bold or conclusive statements about how well a drug works when it is only in Phase I trials, since the primary goal of Phase I trials is to evaluate how safe a drug is, not how well it works. (A simple guide to clinical trials is available at: http://www.cancer.gov/clinicaltrials/understanding/what-is-a-clinical-trial) But many times journalists report on early phase drug trials as if all the evidence is in hand. (See "How the media left the evidence out in the cold," at: http://bmj.bmjjournals.com/cgi/content/full/326/7403/1403) The Association of Health Care Journalists advises its members to "give accurate portrayals of the status of investigational drugs, devices and procedures, including significant caveats and explanations of hurdles, unknowns and potential problems."

If consumers see or hear stories that don't carry such caveats, they should have doubts about the accuracy and balance of the story.

Devices

In an article, "Covering Medical Technology" in the Columbia Journalism Review, (http://archives.cjr.org/year/01/5/lieberman.asp) journalist Trudy Lieberman wrote:

"In the name of news and the desire to build audience, the media are stimulating demand for medical tests and treatments that are unproven and untested, and may even be harmful. The lure of stories about medical breakthroughs and miracles is so strong that the press rushes to report on them even if there is little or no evidence that they are safe and effective. … Journalists often fall victim to powerful public relations machines representing some very big money. Reporting on a product or technology not yet proven clinically effective generates sales for manufacturers and stimulates a momentum that is hard to reverse. …Too many journalists take a formulaic approach to supposed medical breakthroughs. They start with the premise that a technology works or is effective, so the formula almost always dictates a positive spin and produces a predictable story. Too often, stories omit contrary information or do not acknowledge the uncertainty that often surrounds new tests and treatments."

Consumers should know that medical devices and procedures don't undergo the same type of scrutiny for safety and effectiveness as that used for prescription drugs. As a result, patients (and news consumers) may not be given much information on possible benefits and harms of new devices and procedures. But that puts even more responsibility on journalists to recognize, and report on, what is and isn't known about such new devices and procedures.

Animal & Lab Studies

Stories about research in animals or about research in the laboratory but not yet in humans (sometimes called pre-clinical or in-vitro studies) should include warnings about how this research may not pan out in people. Stories that fail to include such information may paint a brighter picture for possible application in humans than is actually the case.

Nonetheless, preliminary research stories continue to be reported. In the future, we may follow up on some of these stories to see how many panned out in people. Here are some examples: