A journalist just asked us this week, “Is it possible to address your criteria in <200 words?” Our answer was a clear “yes.” But it didn’t happen in this 211-word story.
This story covered the bare bones of the study. There was no critical analysis. No independent perspectives. The headline reads, “Options in Treating Incontinence,” but no other options are given, besides the two treatments studied in the trial. In addition, the article uses language, such as “significantly,” without backing up the claim with hard data.
In addition, the headline (and/or first few lines) should make clear that the study involved women only. Incontinence is also a big problem in men.
The story ends with the lead researcher saying that the study “gives patients the information to make an informed decision.” Unfortunately, this story about the study comes nowhere close to doing that.
The story does not compare the prices between a six-month supply of anticholinergic drugs and a single botox injection.
The story briefly mentions that the two treatments “vary in side effects and long-term effectiveness,” but these are never quantified. Instead, the article mentions the certain side effects are “much more common,” such as dry mouth for the anticholinergic drugs, but actual figures would have helped to put expressions like “much more” into perspective.
The article twice uses the word “significantly.” For example, the story states : “Scientists… found that both treatments significantly reduced incontinence episodes,” and also, “Significantly, those who took Botox were more likely to have their symptoms completely disappear.” And here, actual figures are a must, since “significantly” has a very different meaning to the general population than to researchers, who usually mean “statistical significance.” For example, 27 percent of the botox group saw their urinary symptoms disappear, versus 13 percent of the anticholinergic group. The inclusion of these figures put the story’s “significantly” into perspective.
Journalists can help readers understand the difference between statistical significance and clinical significance.
The story does detail the side effects of dry mouth, urinary infections and incomplete emptying of the bladder, but these are never quantified. But language like “much more common” or “more likely” is unhelpful in its lack of specificity. The paper itself revealed:
Dry mouth occurred in significantly more participants in the anticholinergic group than in the onabotulinumtoxinA group (46% vs. 31%, P=0.02). Intermittent catheterization was recommended according to protocol criteria at scheduled visits in the onabotulinumtoxinA group only (in 5% of the participants at 2 months, 3% at 4 months, and 1% at 6 months). However, additional women in both groups performed catheterization off-protocol . More women in the onabotulinumtoxinA group than in the anticholinergic group had a urinary tract infection (33% vs. 13%, P<0.001).
Maybe 15 percentage point differences or 20 percentage point differences are things we should tell readers about and let them judge.
One of the things that invariably gets left out in 211-word blog posts is an evaluation of the strength or limitations of the evidence. In this study, for example, the researchers combined two active treatments in this trial, as participants took either a daily medicine and a placebo injection or a Botox injection and a daily placebo pill. Usually, patients would not be receiving both treatments at the same time. The researchers acknowledge this limitation by writing : “…we cannot determine to what extent the observed improvements reflect a placebo effect,” but this comment failed to make it into the Times story.
The researchers also detailed what they considered to be the strengths of their approach and the generalizability of the study findings. This wasn’t addressed in the story either.
The story does not engage in disease mongering.
There is no independent source quoted in this article. The only comment is from the lead author of the story.
The story does not detail other available alternatives, such as muscle strengthening exercises, surgery, or other medications.
The article mentions that anticholinergic medicine is the “most common treatment” for urinary incontinence. It is also well-known that botox injections are readily available.
There wasn’t any claim of novelty in the story. But it could have commented on this claim by the authors:
“This is the first study to compare the effectiveness of Botox treatments to oral medication,” said study senior author Susan F. Meikle, M.D., M.S.P.H., of the Contraception and Reproductive Health Branch of the NIH’s Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) and Program Director of the Pelvic Floor Disorders Network (PFDN). “Previously, Botox was reserved for women who had tried oral medications but found them ineffective. Because we included some women who had not been treated with oral medication before, these results suggest that Botox could be discussed as an option for first line treatment.”
The story does not appear to rely on a press release.