Big news orgs can’t resist reporting on small trials; hype doesn’t help readers, though

BUY THE HYPEThis week, a Wall Street Journal reader left this comment online in response to a story on bunnies & broccoli: “Uh. This study was conducted with 15 rabbits. For 4 weeks. And it included a drug not tested on humans. Why are you reporting on this?” Read our review of the story in question.

Maybe news organizations need to wise up about how turned-off many readers might be about not-ready-for-prime-time news.

ACL surgery study makes headlines, but should it have?

This week, the Wall Street Journal also delivered this: A Potential Breakthrough in ACL Surgery. It was based on preliminary 3-month results on 10 people.  Personal note:  As one of the thousands of weekend warriors and others each year who have torn their ACLs, I wouldn’t go to the bank with anything based on 3 months followup. Also, not all torn ACLs are alike. So the selection of subjects for this trial might be crucial in determining its results. The WSJ did include this caveat:

“…not every patient with a torn ACL will be a candidate for the procedure. The tear needs to leave a 6-8 millimeter stub on the tibia side of the ligament.”

OK, thanks, but is that useful? How many of us have any idea what our “stub” size was?

The New York Times didn’t include any such caveat in its story, but showed a bit more restraint than the WSJ, with this:

Experts were intrigued but cautious.

“This is definitely an advance,” said Dr. Jo Hannafin, a senior attending orthopedic surgeon at the Hospital for Special Surgery in New York, who was not involved in the experiment.

But, she added, “I don’t think we will know for three to five years whether this technique is really effective or not.”

Word to watch out for: Revolutionize

Nonetheless, the NYT joined the WSJ and the Boston Globe in reporting on a short-term, 10 person experiment. None of the three mentioned any data being published in a peer-reviewed journal. All three profiled the same star trial participant–a 26-year old man. The Globe went so far as to project:

“…the procedure could be available for widespread use in three to four years and revolutionize how orthopedic surgeons treat ACL injuries.”

Let me be clear:  I think this is fascinating research.  I hope it pans out.  But it sure is too early to be talking about revolutionary breakthroughs.

NYT can’t resist chocolate story, despite tiny sample size

The New York Times actually gave us two examples this week of giving significant attention to very early, very small trials.

Studies involving chocolate are like catnip for journalists, a friend of mine recently suggested.

Well, the cats were going crazy again this week as we saw on the Well blog of the New York Times, which blasted this headline:

Chocolate Can Boost Your Workout. Really.

You can read the details if you wish.  But I suggest the most important details didn’t appear until 390 words deep in an 841 word story–akin to drinking a half pint of sludge before you realize it’s not the award-winning microbrew you expected. If you got that far, you finally learned:

“they found eight male recreational cyclists who agreed, in the interests of science, to swallow a little dark chocolate every day.”

First, that’s wrong. The researchers recruited 9 subjects, not 8, following them for 14 days.  That’s really all you need to know to see that the NYT headline was way over the top. Handful of subjects, short-term experiment.  That doesn’t support the bold, causal statement that “chocolate can boost your workout” – even without the “Really” appendage.

If you read the journal article upon which the NYT blog post is based, you’ll see it ends with this:

However, future double-blinded studies will need to confirm this effect.

Finally, this journal manuscript was published more than 3 months ago.  Why is this tiny, short-term study news now?

You might also like

Comments (1)

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

sharon begley

March 28, 2016 at 7:50 am

Well put, Gary. Here in the trenches we are all waiting for the day when news organizations have evidence of ” how turned-off many readers might be about not-ready-for-prime-time [medical and health] news,’ as well as data that those readers outnumber the ones who click on the most sensationalized, poorly-supported stories. I am sorry to say that that day seems far in the future.