Whenever we see a verb like “slashes” in a headline, our skepticism meter starts to buzz. In this case, the headline does not do the story justice. The story appropriately brings to light the current state of evidence around medical use of marijuana for treating epilepsy, restrictions on its use, the small size of this particular trial, and the high rate of adverse side effects. It makes great use of independent sources, too. It’s unfortunate in our click-crazed media moment that so many people may read nothing more than this headline in a tweet and feel sufficiently informed.
Overall, we were impressed by the quality of this story and level of detail it included. The story would have been even stronger if it had discuss the potential costs of the treatment.
Interestingly, the news release from NYU also earned high marks from us–showing how a strong news release can beget strong news coverage.
Quantitative reporting of evidence, unaffiliated expert commentary, and context setting–these are the details that help accurately explain the implications of new research and truly inform readers.
There was not a discussion of costs for this intervention–this was the weak spot in the story.
The story says in the first paragraph: “Among children taking cannabidiol, the decrease in the frequency of convulsive seizures — which involve a loss of consciousness, stiffened muscles and jerking movements — was 23 percentage points greater than the decrease in seizures among children taking a placebo.”
What exactly does this mean? That, for example, the children taking a placebo experienced seizures on 73 out of every 100 days during the trial and that those taking the drug experienced seizures on 50 out of every 100 days? We would need to know the actual number of children who did and did not experience seizures and then know more about the frequency of those seizures to understand this reduction.
Later in the story, there is a better explanation of the comparison of frequency of seizures, and the story does something that most stories don’t ever do. It explains the range of experiences during the trial, not just the average. It says, “Within the study, individual participants experienced convulsive seizures at a rate ranging from four per month, on average, to 1,717 per month. During the 14-week study, frequency of convulsive seizures decreased from an average of 12.4 to 5.9 per month in the cannabidiol group, compared with 14.9 to 14.1 in the placebo group. On average, the change in seizure frequency amounted to a 39% decrease for the cannabidiol group patients, compared with a roughly 13% decrease among the placebo group.”
That’s quite a range, and it would have been good for the story to include a sentence explaining that the sheer variety in the patient population would make it difficult to adequately assess the efficacy of the drug in a trial this small in size. We’re unclear on the math here, though. It looks to us that there was a 26 percentage point difference between the reduction in the drug group and the placebo group.
Still, this is a lot of data-driven detail we don’t normally see in news stories, so we appreciated this.
The story nicely refers to the harms up high and then returns to them in greater detail later in the piece. It also does something we see too rarely, which is mention how many patients dropped out of the study.
The story gives a fairly good description of the trial. It also provides appropriate context and caveats. It says, for example, “The study was a randomized, double-blind, placebo-controlled human trial, which is considered the gold standard test for any new medicine.” And it also includes a good quote saying, “No one study decides an issue; the sample size is still relatively small (because this is a rare syndrome and so hard to study large numbers of cases) and the duration of treatment so far has been relatively short.”
There is no disease mongering in the piece and it goes into a lot of detail to explain the specific condition.
The story quotes four independent sources, a very high number for a story about a study like this. It also explains that the study was partially funded by a drug company trying to commercialize a cannabis-based drug.
There is not a deep comparison of alternatives in this piece, but we give it credit for quoting an independent expert saying, “Interestingly, it looks similar to our other seizure medications in terms of efficacy and tolerability. … So, sadly, not a home run for most patients but another tool in our treatment regimen.”
There is a significant section on availability of marijuana and marijuana-derived drugs. A reader can fairly easily infer that this intervention is not yet available.
But, as we noted in our news release review, “The FDA does, however, consider cannabidiol an investigational product for treating children with epilepsy and has “expanded access provisions” that allow its use.”
This would have been useful information to include in the story.
The story establishes that these are not novel findings but rather an incremental addition to the body of knowledge around marijuana-derived therapies.
The story does not rely on a news release.
Systematic, Criteria-Driven
Following a shift in journalism’s business model, a diverse ecosystem of publications, many covering science, has sprung up online. This has made it more difficult to sort out what qualifies as journalism, what doesn’t, and whether the public even cares.
https://t.co/qO2a27FiSe
Researchers and journalists do the public a grave disservice when they spin health study results to make them appear safer or more effective than warranted, says @HealthNewsRevu founder @garyschwitzer https://t.co/QIKsRbP6Ed
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like