Whenever we see a verb like “slashes” in a headline, our skepticism meter starts to buzz. In this case, the headline does not do the story justice. The story appropriately brings to light the current state of evidence around medical use of marijuana for treating epilepsy, restrictions on its use, the small size of this particular trial, and the high rate of adverse side effects. It makes great use of independent sources, too. It’s unfortunate in our click-crazed media moment that so many people may read nothing more than this headline in a tweet and feel sufficiently informed.
Overall, we were impressed by the quality of this story and level of detail it included. The story would have been even stronger if it had discuss the potential costs of the treatment.
Interestingly, the news release from NYU also earned high marks from us–showing how a strong news release can beget strong news coverage.
Quantitative reporting of evidence, unaffiliated expert commentary, and context setting–these are the details that help accurately explain the implications of new research and truly inform readers.
There was not a discussion of costs for this intervention–this was the weak spot in the story.
The story says in the first paragraph: “Among children taking cannabidiol, the decrease in the frequency of convulsive seizures — which involve a loss of consciousness, stiffened muscles and jerking movements — was 23 percentage points greater than the decrease in seizures among children taking a placebo.”
What exactly does this mean? That, for example, the children taking a placebo experienced seizures on 73 out of every 100 days during the trial and that those taking the drug experienced seizures on 50 out of every 100 days? We would need to know the actual number of children who did and did not experience seizures and then know more about the frequency of those seizures to understand this reduction.
Later in the story, there is a better explanation of the comparison of frequency of seizures, and the story does something that most stories don’t ever do. It explains the range of experiences during the trial, not just the average. It says, “Within the study, individual participants experienced convulsive seizures at a rate ranging from four per month, on average, to 1,717 per month. During the 14-week study, frequency of convulsive seizures decreased from an average of 12.4 to 5.9 per month in the cannabidiol group, compared with 14.9 to 14.1 in the placebo group. On average, the change in seizure frequency amounted to a 39% decrease for the cannabidiol group patients, compared with a roughly 13% decrease among the placebo group.”
That’s quite a range, and it would have been good for the story to include a sentence explaining that the sheer variety in the patient population would make it difficult to adequately assess the efficacy of the drug in a trial this small in size. We’re unclear on the math here, though. It looks to us that there was a 26 percentage point difference between the reduction in the drug group and the placebo group.
Still, this is a lot of data-driven detail we don’t normally see in news stories, so we appreciated this.
The story nicely refers to the harms up high and then returns to them in greater detail later in the piece. It also does something we see too rarely, which is mention how many patients dropped out of the study.
The story gives a fairly good description of the trial. It also provides appropriate context and caveats. It says, for example, “The study was a randomized, double-blind, placebo-controlled human trial, which is considered the gold standard test for any new medicine.” And it also includes a good quote saying, “No one study decides an issue; the sample size is still relatively small (because this is a rare syndrome and so hard to study large numbers of cases) and the duration of treatment so far has been relatively short.”
There is no disease mongering in the piece and it goes into a lot of detail to explain the specific condition.
The story quotes four independent sources, a very high number for a story about a study like this. It also explains that the study was partially funded by a drug company trying to commercialize a cannabis-based drug.
There is not a deep comparison of alternatives in this piece, but we give it credit for quoting an independent expert saying, “Interestingly, it looks similar to our other seizure medications in terms of efficacy and tolerability. … So, sadly, not a home run for most patients but another tool in our treatment regimen.”
There is a significant section on availability of marijuana and marijuana-derived drugs. A reader can fairly easily infer that this intervention is not yet available.
But, as we noted in our news release review, “The FDA does, however, consider cannabidiol an investigational product for treating children with epilepsy and has “expanded access provisions” that allow its use.”
This would have been useful information to include in the story.
The story establishes that these are not novel findings but rather an incremental addition to the body of knowledge around marijuana-derived therapies.
The story does not rely on a news release.