This week, a Wall Street Journal reader left this comment online in response to a story on bunnies & broccoli: “Uh. This study was conducted with 15 rabbits. For 4 weeks. And it included a drug not tested on humans. Why are you reporting on this?” Read our review of the story in question.
Maybe news organizations need to wise up about how turned-off many readers might be about not-ready-for-prime-time news.
This week, the Wall Street Journal also delivered this: A Potential Breakthrough in ACL Surgery. It was based on preliminary 3-month results on 10 people. Personal note: As one of the thousands of weekend warriors and others each year who have torn their ACLs, I wouldn’t go to the bank with anything based on 3 months followup. Also, not all torn ACLs are alike. So the selection of subjects for this trial might be crucial in determining its results. The WSJ did include this caveat:
“…not every patient with a torn ACL will be a candidate for the procedure. The tear needs to leave a 6-8 millimeter stub on the tibia side of the ligament.”
OK, thanks, but is that useful? How many of us have any idea what our “stub” size was?
The New York Times didn’t include any such caveat in its story, but showed a bit more restraint than the WSJ, with this:
Experts were intrigued but cautious.
“This is definitely an advance,” said Dr. Jo Hannafin, a senior attending orthopedic surgeon at the Hospital for Special Surgery in New York, who was not involved in the experiment.
But, she added, “I don’t think we will know for three to five years whether this technique is really effective or not.”
Nonetheless, the NYT joined the WSJ and the Boston Globe in reporting on a short-term, 10 person experiment. None of the three mentioned any data being published in a peer-reviewed journal. All three profiled the same star trial participant–a 26-year old man. The Globe went so far as to project:
“…the procedure could be available for widespread use in three to four years and revolutionize how orthopedic surgeons treat ACL injuries.”
Let me be clear: I think this is fascinating research. I hope it pans out. But it sure is too early to be talking about revolutionary breakthroughs.
The New York Times actually gave us two examples this week of giving significant attention to very early, very small trials.
Studies involving chocolate are like catnip for journalists, a friend of mine recently suggested.
Well, the cats were going crazy again this week as we saw on the Well blog of the New York Times, which blasted this headline:
You can read the details if you wish. But I suggest the most important details didn’t appear until 390 words deep in an 841 word story–akin to drinking a half pint of sludge before you realize it’s not the award-winning microbrew you expected. If you got that far, you finally learned:
“they found eight male recreational cyclists who agreed, in the interests of science, to swallow a little dark chocolate every day.”
First, that’s wrong. The researchers recruited 9 subjects, not 8, following them for 14 days. That’s really all you need to know to see that the NYT headline was way over the top. Handful of subjects, short-term experiment. That doesn’t support the bold, causal statement that “chocolate can boost your workout” – even without the “Really” appendage.
If you read the journal article upon which the NYT blog post is based, you’ll see it ends with this:
However, future double-blinded studies will need to confirm this effect.
Finally, this journal manuscript was published more than 3 months ago. Why is this tiny, short-term study news now?