A critique of the study limitations, cost and a meaningful description of benefits are all markedly absent in this NBC.com story on the FDA approval of a hand-held electronic device to treat people with cluster headaches, a rare form of extremely painful and debilitating headaches. The gammaCore device is designed to stimulate the vagus nerve through the skin of the neck.
The article draws substantially from a news release issued by the device’s maker, including four quotes. The only physician-researcher quotes come from the person who tested the device and co-authored a study published about the test results in 2016. We wish the story would have included some indication of the device’s cost since it’s set to hit the U.S. market later this year.
The article aptly notes that its unclear how the vagus nerve stimulator works, but it neglects to explain that the study cited found no significant differences in full or short-term pain relief between 62 people using the device, compared to 73 treated with a sham device. Only when a “subset analysis” was performed on those with occasional cluster headaches was there evidence of benefit to about 25 percent of those treated, compared to 15 percent getting the sham treatment.
The NBC.com story could have highlighted these limitations of the study and noted that many people with cluster headaches should not use the device (anyone with an implanted medical device, with high blood pressure, or with heart rhythm disturbances); and that overall the stimulator — while deemed safe and modestly effective for some — may have limited usefulness.
Despite its omissions, the article provided much more information than did a Reuters brief which disappoints with its lack of detail.
You need to read the study in order to get to the details – and this story whiffed on some of the key details. As pointed out in an editorial accompanying the manuscript about the study in a journal:
“The primary endpoint was not achieved in this study – ie, differences in reduction in pain severity between active and sham groups were not significant…..conclusions drawn in this way are notorious for failing to replicate.”
Even in the abstract of the journal manuscript itself, the conclusion stated:
“the response rate was not significantly different (vs sham) for the total population”
Reliance on a company statement or news release isn’t usually going to give you that kind of information. And for most readers, that may have been the most significant piece of news.
One of the first questions a cluster headache sufferer will have about the device is: how much does it cost? The article doesn’t offer any information about the cost of the device, or comparison to the costs of other treatment. A company release for British audiences described the device in terms of its savings compared to medications (“25% reduction in costs associated with acute treatments such as zolmitriptan, sumatriptan and inhaled oxygen”) but a search on the actual cost of the device and the cost of replacing the pre-loaded doses came up empty-handed.
The article uses percentages to vaguely describe the benefits (“appeared to help”) in the treatment vs sham treatment groups, but it does not mention the absolute number of patients in the two arms of the study, or what “help” meant. It relies mostly on complimentary statements by one of the investigators who conducted a study of the device, and by the company. But the major lapse in full disclosure comes with the absence of information about the failure of the clinical trial to reach its statistical goal. The apparent benefit of the device over the sham device was only seen after a pre-specified subgroup analysis. As noted in the accompanying editorial: “The primary endpoint was not achieved in this study – ie, differences in reduction in pain severity between active and sham groups were not significant…..conclusions drawn in this way are notorious for failing to replicate.”
The side effects were considered to be temporary and relatively minor, but should have been mentioned.
Dr. Silberstein noted, “It does not have the side effects or dose limitations of commonly prescribed treatments…” This is perhaps true but a closer examination of the research results reveals that 11 percent of the participants treated with the gammaCore device saw lip or facial dropping/pulling or twitching during the sham controlled trial and 7 percent experienced these side effects during the open label part of the trial (when both the researchers and volunteers knew which treatment was being administered).
The study and the editorial accompanying its publication offer substantial detail about the considerable efforts made by the investigators to test a device and account for the real difficulties in “blinding” a study involving a rapid means of pain relief or prevention of pain. The article could have been greatly strengthened by offering information about the quality of the research and the weaknesses and limitations that exist in the results.
There was no disease mongering about cluster headaches. The article provides context about the problem and how rare it is.
The story doesn’t quote any independent sources – not even from the editorial in the journal that offered a critique about the study’s limitations and conclusions.
The article did mention that there are “commonly prescribed treatments,” and it named some of them.
The article notes that the product is available in the EU and that the company will begin selling it in the U.S. later in 2017.
The article focused on the FDA’s approval of the device as the news hook. It did not repeat the manufacturer’s claim that the device is the first of its kind.
The story relied significantly on the company’s news release and would have been strengthened by either directly vetting the claims or seeking an independent expert source to help with that.
The only quotes in the story came from a company statement.
A critique of the study limitations, cost and a meaningful description of benefits are all markedly absent in this NBC.com story on the FDA approval of a hand-held electronic device to treat people with cluster headaches, a rare form of extremely painful and debilitating headaches. The gammaCore device is designed to stimulate the vagus nerve through the skin of the neck.
The article draws substantially from a news release issued by the device’s maker, including four quotes. The only physician-researcher quotes come from the person who tested the device and co-authored a study published about the test results in 2016. We wish the story would have included some indication of the device’s cost since it’s set to hit the U.S. market later this year.
The article aptly notes that its unclear how the vagus nerve stimulator works, but it neglects to explain that the study cited found no significant differences in full or short-term pain relief between 62 people using the device, compared to 73 treated with a sham device. Only when a “subset analysis” was performed on those with occasional cluster headaches was there evidence of benefit to about 25 percent of those treated, compared to 15 percent getting the sham treatment.
The NBC.com story could have highlighted these limitations of the study and noted that many people with cluster headaches should not use the device (anyone with an implanted medical device, with high blood pressure, or with heart rhythm disturbances); and that overall the stimulator — while deemed safe and modestly effective for some — may have limited usefulness.
Despite its omissions, the article provided much more information than did a Reuters brief which disappoints with its lack of detail.
You need to read the study in order to get to the details – and this story whiffed on some of the key details. As pointed out in an editorial accompanying the manuscript about the study in a journal:
“The primary endpoint was not achieved in this study – ie, differences in reduction in pain severity between active and sham groups were not significant…..conclusions drawn in this way are notorious for failing to replicate.”
Even in the abstract of the journal manuscript itself, the conclusion stated:
“the response rate was not significantly different (vs sham) for the total population”
Reliance on a company statement or news release isn’t usually going to give you that kind of information. And for most readers, that may have been the most significant piece of news.
One of the first questions a cluster headache sufferer will have about the device is: how much does it cost? The article doesn’t offer any information about the cost of the device, or comparison to the costs of other treatment. A company release for British audiences described the device in terms of its savings compared to medications (“25% reduction in costs associated with acute treatments such as zolmitriptan, sumatriptan and inhaled oxygen”) but a search on the actual cost of the device and the cost of replacing the pre-loaded doses came up empty-handed.
The article uses percentages to vaguely describe the benefits (“appeared to help”) in the treatment vs sham treatment groups, but it does not mention the absolute number of patients in the two arms of the study, or what “help” meant. It relies mostly on complimentary statements by one of the investigators who conducted a study of the device, and by the company. But the major lapse in full disclosure comes with the absence of information about the failure of the clinical trial to reach its statistical goal. The apparent benefit of the device over the sham device was only seen after a pre-specified subgroup analysis. As noted in the accompanying editorial: “The primary endpoint was not achieved in this study – ie, differences in reduction in pain severity between active and sham groups were not significant…..conclusions drawn in this way are notorious for failing to replicate.”
The side effects were considered to be temporary and relatively minor, but should have been mentioned.
Dr. Silberstein noted, “It does not have the side effects or dose limitations of commonly prescribed treatments…” This is perhaps true but a closer examination of the research results reveals that 11 percent of the participants treated with the gammaCore device saw lip or facial dropping/pulling or twitching during the sham controlled trial and 7 percent experienced these side effects during the open label part of the trial (when both the researchers and volunteers knew which treatment was being administered).
The study and the editorial accompanying its publication offer substantial detail about the considerable efforts made by the investigators to test a device and account for the real difficulties in “blinding” a study involving a rapid means of pain relief or prevention of pain. The article could have been greatly strengthened by offering information about the quality of the research and the weaknesses and limitations that exist in the results.
There was no disease mongering about cluster headaches. The article provides context about the problem and how rare it is.
The story doesn’t quote any independent sources – not even from the editorial in the journal that offered a critique about the study’s limitations and conclusions.
The article did mention that there are “commonly prescribed treatments,” and it named some of them.
The article notes that the product is available in the EU and that the company will begin selling it in the U.S. later in 2017.
The article focused on the FDA’s approval of the device as the news hook. It did not repeat the manufacturer’s claim that the device is the first of its kind.
The story relied significantly on the company’s news release and would have been strengthened by either directly vetting the claims or seeking an independent expert source to help with that.
The only quotes in the story came from a company statement.
Systematic, Criteria-Driven
Excelente conversación entre @garyschwitzer de @HealthNewsRevu y @marynmck sobre exageración y #COVID19 en el excelente MOOC de @utknightcenter. Vale la pena verlo. https://www.youtube.com/watch?v=SkgXiPojtg4&feature=youtu.be
Words of real wisdom, as usual @garyschwitzer
And: “There were no details, charts, graphs, numbers, nothing published in a journal."
#Pharma chairman:“If you don’t put out data people will say why are you withholding the data?”
Well, yes, that is exactly what people are asking. https://twitter.com/garyschwitzer/status/1264572399523180548
Comments (1)
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Peter H. Smith
February 13, 2018 at 3:34 pmThis device is a form of highway robbery. Be sure you know what you are getting before you pay for it. When you use it the first time, it starts a 31 day timer. It disables itself after that. Get a copy of the instructions and run it past your lawyer before deciding to buy (only a lawyer can make sense out of this.) It sounds like one “treatment” takes 6 “doses.” The dose limit is as restrictive as the calendar day limit.
I had great hope for this treatment. I thought it would make it possible to board commercial flights with some chance of arriving unscathed despite the ban on high-flow oxygen aboard the plane. Instead, I have an outrageously expensive device that will become a paperweight 31 days after I use it.
My hatred of our medical/insurance/device racket – er, system – has reached a new high. Move over, epinephrine syringes!
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like