A critique of the study limitations, cost and a meaningful description of benefits are all markedly absent in this NBC.com story on the FDA approval of a hand-held electronic device to treat people with cluster headaches, a rare form of extremely painful and debilitating headaches. The gammaCore device is designed to stimulate the vagus nerve through the skin of the neck.
The article draws substantially from a news release issued by the device’s maker, including four quotes. The only physician-researcher quotes come from the person who tested the device and co-authored a study published about the test results in 2016. We wish the story would have included some indication of the device’s cost since it’s set to hit the U.S. market later this year.
The article aptly notes that its unclear how the vagus nerve stimulator works, but it neglects to explain that the study cited found no significant differences in full or short-term pain relief between 62 people using the device, compared to 73 treated with a sham device. Only when a “subset analysis” was performed on those with occasional cluster headaches was there evidence of benefit to about 25 percent of those treated, compared to 15 percent getting the sham treatment.
The NBC.com story could have highlighted these limitations of the study and noted that many people with cluster headaches should not use the device (anyone with an implanted medical device, with high blood pressure, or with heart rhythm disturbances); and that overall the stimulator — while deemed safe and modestly effective for some — may have limited usefulness.
Despite its omissions, the article provided much more information than did a Reuters brief which disappoints with its lack of detail.
You need to read the study in order to get to the details – and this story whiffed on some of the key details. As pointed out in an editorial accompanying the manuscript about the study in a journal:
“The primary endpoint was not achieved in this study – ie, differences in reduction in pain severity between active and sham groups were not significant…..conclusions drawn in this way are notorious for failing to replicate.”
Even in the abstract of the journal manuscript itself, the conclusion stated:
“the response rate was not significantly different (vs sham) for the total population”
Reliance on a company statement or news release isn’t usually going to give you that kind of information. And for most readers, that may have been the most significant piece of news.
One of the first questions a cluster headache sufferer will have about the device is: how much does it cost? The article doesn’t offer any information about the cost of the device, or comparison to the costs of other treatment. A company release for British audiences described the device in terms of its savings compared to medications (“25% reduction in costs associated with acute treatments such as zolmitriptan, sumatriptan and inhaled oxygen”) but a search on the actual cost of the device and the cost of replacing the pre-loaded doses came up empty-handed.
The article uses percentages to vaguely describe the benefits (“appeared to help”) in the treatment vs sham treatment groups, but it does not mention the absolute number of patients in the two arms of the study, or what “help” meant. It relies mostly on complimentary statements by one of the investigators who conducted a study of the device, and by the company. But the major lapse in full disclosure comes with the absence of information about the failure of the clinical trial to reach its statistical goal. The apparent benefit of the device over the sham device was only seen after a pre-specified subgroup analysis. As noted in the accompanying editorial: “The primary endpoint was not achieved in this study – ie, differences in reduction in pain severity between active and sham groups were not significant…..conclusions drawn in this way are notorious for failing to replicate.”
The side effects were considered to be temporary and relatively minor, but should have been mentioned.
Dr. Silberstein noted, “It does not have the side effects or dose limitations of commonly prescribed treatments…” This is perhaps true but a closer examination of the research results reveals that 11 percent of the participants treated with the gammaCore device saw lip or facial dropping/pulling or twitching during the sham controlled trial and 7 percent experienced these side effects during the open label part of the trial (when both the researchers and volunteers knew which treatment was being administered).
The study and the editorial accompanying its publication offer substantial detail about the considerable efforts made by the investigators to test a device and account for the real difficulties in “blinding” a study involving a rapid means of pain relief or prevention of pain. The article could have been greatly strengthened by offering information about the quality of the research and the weaknesses and limitations that exist in the results.
There was no disease mongering about cluster headaches. The article provides context about the problem and how rare it is.
The story doesn’t quote any independent sources – not even from the editorial in the journal that offered a critique about the study’s limitations and conclusions.
The article did mention that there are “commonly prescribed treatments,” and it named some of them.
The article notes that the product is available in the EU and that the company will begin selling it in the U.S. later in 2017.
The article focused on the FDA’s approval of the device as the news hook. It did not repeat the manufacturer’s claim that the device is the first of its kind.
The story relied significantly on the company’s news release and would have been strengthened by either directly vetting the claims or seeking an independent expert source to help with that.
The only quotes in the story came from a company statement.