NOTE TO READERS: When this project lost substantial funding at the end of 2018, I lost the ability to continue publishing criteria-driven news story reviews and PR news release reviews - once the bread-and-butter of the site going back to 2006. The 3,200 archived reviews, while still educational, are getting old and difficult for me to technically maintain on the back end of the website. So I am announcing that I plan to remove these reviews from the site by April 1, 2021. The blog and the toolkit - two of the most popular features on the site - will remain. If you wish to peruse the reviews before they disappear, please do so by the end of March 2021. After that date you may still be able to access them via the Internet Archive Wayback Machine - https://archive.org/web/.
Read Original Story

Small study suggests electrical stimulation of the brain can ease migraine pain

Rating

1 Star

Small study suggests electrical stimulation of the brain can ease migraine pain

Our Review Summary

We like the LA Times’ Booster Shots blog. We really do.

But if the paper insists on being brief in its blog posts, at least it could provide links to provide more context and to plug holes in what is reported briefly on the blog.

So, for example, it would require only a few words to address these flaws:

  • No mention of costs
  • A description of migraine that is too vague, thus committing disease-mongering
  • No evaluation of the quality of the evidence
  • An inadequate discussion of the scope of the benefits and harms (Why not give absolute numbers?  Why not clearly explain the limitations of a study that tested a device in just 33 people?
  • Why not a link to the published study? (After all, the journal Cephalalgia isn’t on most readers’ shelves!)

Again, we appreciate the existence of the Booster Shots blog.  But this was a bad day.

 

Why This Matters

Migraine is not a topic to fool around with.  People who have this problem may gravitate toward any news of progress.  We think they deserved more scrutiny of claims and more context than they got in this piece.

Criteria

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

No mention of costs – and none of these devices is cheap.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story explains that 39% of patients who received the real device had at least a 50% decrease in the number of headaches per month.

Why not say that this was 10 patients?  And why not give us the precise absolute number – not the relative risk reduction – of how many headaches? 

More importantly, no mention is made of any difference between the rate of response in subjects given the real treatment and those provided a sham treatment. Did the sham treatment result in a significant improvement as well?

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

Also not good enough.

The story explained that "the most common side effect was migration of the leads" but it didn’t explain how common that was.  Was it 100% of the time?  50%  What does common mean?

And it didn’t explain how bothersome that was to people.  Was it like a hiccup?  Did it hurt?  

Finally, the editorial to which the story refers points out that even if this technology works (and we don’t know that it does), the "complication rates would need to be reduced" if it is to be acceptable. We don’t know why that didn’t make it into the story.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

Not good enough.

Yes, the story said this was "a small pilot study" and yes it said "if the results can be confirmed…."

But why not explicitly state how limited are the conclusions that can be drawn from a study of only 33 people who got the device?

And, given that the funding for the study came from the manufacturer, readers should be told what safeguards were in place against bias.  How do they know this study has any validity? 

Does the story commit disease-mongering?

Not Satisfactory

The description of migraine given in the story is too broad: "characterized by severe pain accompanied by nausea and sensitivity to light…" Any severe headache can be accompanied by nausea and photophobia. The readers do not know from this description whether they are fellow sufferers or not. And perhaps more important: a careful reading of the editorial that accompanies the study demonstrates that the study participants were a highly rarified subgroup of more general migraine sufferers. These subjects had to have migraine in the same area that the occipital nerve travels and had to respond to some tests of the occipital nerve. Therefore, the story does the reader a disservice by neglecting to qualify to whom this study might apply.

Does the story use independent sources and identify conflicts of interest?

Satisfactory

OK.  Medtronic’s funding of the study is disclosed.  Minimal independent input was achieved by citing what an editorial writer wrote.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

Inadequate in comparing this device with the range of alternatives the 14% of uncontrolled migraine sufferers pursue.  This device was described as "one such approach."  What are the others?  How do they compare?

Does the story establish the availability of the treatment/test/product/procedure?

Not Satisfactory

The reader would not be able to figure out from the story whether this is an experimental device or something already on the market.  And, if on the market, how widespread is its use? Yes, the first paragraph says "If the results can be confirmed, the approach could provide another tool…" and perhaps savvy readers would read into this that the approach is not yet available.  But there is no clear, explicit statement on availability.  It would only require a few words.

Does the story establish the true novelty of the approach?

Not Satisfactory

Inadequate in explaining the range of alternatives the 14% of uncontrolled migraine sufferers pursue.  This device was described as "one such approach."  What are the others?  How do they compare? How truly novel is this, if at all?

Does the story appear to rely solely or largely on a news release?

Not Applicable

Not applicable because we can’t be sure of the extent to which the story relied on a news release.  There is no quote from an interview.  The lead author isn’t quoted at all.  An accompanying editorial is cited. 

Total Score: 1 of 9 Satisfactory

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.