Note to our followers: Our nearly 13-year run of daily publication of new content on HealthNewsReview.org comes to a close at the end of 2018. Publisher Gary Schwitzer and other contributors may post new articles periodically. But all of the 6,000+ articles we have published contain lessons to help you improve your critical thinking about health care interventions. And those will be still be alive on the site for a couple of years.

When ‘fact-checked’ health news doesn’t tell the whole story

Joy Victory is Deputy Managing Editor of HealthNewsReview.org. She tweets at @thejoyvictory.

On the surface, this headline from Healthline.com looks like a good thing:

'fact-checked' health news

It’s news about an experimental drug that shows promise in treating a severe form of MS. And the information has been fact-checked to boot. Both good things, right?

But the story didn’t include key details like side effects, and it used a quote lifted directly from the drug company news release, among other problems. These red flags raise an important question: Who is controlling the “facts” on this story? Healthline’s fact-checkers or the drug company that funded the study?

Poor reporting of the statistics

Specifically, readers may come away from this story thinking that in just three months, the drug–called siponimod–reduced MS symptoms by 21 percent. Here’s how the story describes the benefits:

After three months of use, siponimod was able to slow down the disease’s effects by 21 percent and reduced the risk of six-month disease progression by 26 percent.

However, the reality is somewhat different. Siponimod reduced the risk of disease progression by 21 percent. This means that 21% fewer patients saw their disease get worse during a very brief 3-month window — NOT that patients taking the drug averaged a 21% reduction in their symptoms.

And the impact is far less impressive when you look at the raw, absolute numbers. At three months, 288 of 1,096 patients (26%) receiving the drug got worse, whereas 173 of 545 (32%) on placebo got worse. That’s an absolute risk reduction of 6% (32% – 26%).

More on why this matters: Reporting the findings: Absolute vs relative risk

In simplest terms, slightly fewer people in the siponimod group got worse than those on placebo. For many people, the drug won’t do much of anything.

No critical commentary

That marginal absolute benefit is why an expert-authored editorial that ran in the same issue of The Lancet concluded the study findings were “disappointing results and do not suggest that siponimod is an effective treatment for SPMS.”

The editorial paints a very different picture than the news release issued by Novartis, the drug maker. The release’s headline was “Phase III data in The Lancet show Novartis siponimod significantly improves outcomes in patients with secondary progressive MS.”

A dissenting viewpoint like those in the Lancet editorial didn’t make it into Healthline’s coverage, which only had quotes from the news release, and from a researcher at the National MS Society (which is partially funded by Novartis).

“That’s a problem. With an editorial, it’s right there, that’s low-hanging fruit,” said journalist and fact-checker Brooke Borel, who wrote The Chicago Guide to Fact-Checking. “You didn’t even have to dig for that.”

‘Feels overly promotional’

Clicking Healthline’s “fact-checked” button opens a pop-up window with their policy:

All Healthline content is medically reviewed or fact checked to ensure as much factual accuracy as possible.

We have strict sourcing guidelines and only link to reputable media sites, academic research institutions and, whenever possible, medically peer reviewed studies.

If you feel that any of our content is inaccurate, out-of-date, or otherwise questionable, please contact us through the feedback form on this page.

Despite these guidelines, Borel noted the story “doesn’t have all the context you’d want a reported piece to have. It concerns me to see [fact-checking] buttons like this because I do think it’s misleading and there’s not much information for the readers on the actual process they’re using and the quality of the fact checking.”

Preeti Malani, MD

Preeti Malani, MD

Preeti Malani, MD, a professor of medicine at the University of Michigan, agreed.

“‘Fact-checked’ is an unusual term to use in this setting. While the term has a very specific meaning in journalism, it doesn’t appear to be used in that manner here,” said Malani, who is also a journalist, and a HealthNewsReview.org contributor. “The entire piece feels overly promotional (the lede uses the term ‘breakthrough’). The use of relative risk is also deceptive. No mention of cost, adverse effects or limitations.”

Healthline didn’t respond to a request for comment.

To its credit, the story does mention that the drug didn’t improve the walking ability of MS patients when compared to placebo, and at the very end of the piece, it refers to the findings as “somewhat modest.”

These details also were included in another story we looked at, from the US edition of The Guardian. Their story is an interesting contrast to the HealthLine story, because it feels far more “fact-checked:” It includes absolute numbers to convey the benefits, quotes the independent commentary from The Lancet editorial, and discusses limitations of the study. (Yet their story also didn’t discuss adverse effects.)

Health News Review Criteria

Click to see our criteria.

These are all basics we ask journalists to include in their news reporting, as part of our 10 review criteria.

“The HealthLine piece definitely doesn’t meet best standards for health journalism. It’s important to keep the approach more objective for many reasons but especially for a condition like MS that still doesn’t have highly effective therapy,” Malani said. “Patients and their families could easily be misled by this piece.  While this new drug has a role in treatment, it is not a ‘breakthrough.'”

You might also like

Comments

We Welcome Comments. But please note: We will delete comments left by anyone who doesn’t leave an actual first and last name and an actual email address.

We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, or profanity. We will also end any thread of repetitive comments. Comments should primarily discuss the quality (or lack thereof) in journalism or other media messages about health and medicine. This is not intended to be a forum for definitive discussions about medicine or science. Nor is it a forum to share your personal story about a disease or treatment -- your comment must relate to media messages about health care. If your comment doesn't adhere to these policies, we won't post it. Questions? Please see more on our comments policy.

Comments are closed.