April 21, 2017 Read Original Story

NBC’s single source report on device for treating cluster headaches lacks balance

Reviewed By

Rating

FDA Approves Vagus Nerve Stimulator to Treat Cluster Headaches

Our Review Summary

A critique of the study limitations, cost and a meaningful description of benefits are all markedly absent in this NBC.com story on the FDA approval of a hand-held electronic device to treat people with cluster headaches, a rare form of extremely painful and debilitating headaches. The gammaCore device is designed to stimulate the vagus nerve through the skin of the neck.

The article draws substantially from a news release issued by the device’s maker, including four quotes. The only physician-researcher quotes come from the person who tested the device and co-authored a study published about the test results in 2016. We wish the story would have included some indication of the device’s cost since it’s set to hit the U.S. market later this year.

The article aptly notes that its unclear how the vagus nerve stimulator works, but it neglects to explain that the study cited found no significant differences in full or short-term pain relief between 62 people using the device, compared to 73 treated with a sham device. Only when a “subset analysis” was performed on those with occasional cluster headaches was there evidence of benefit to about 25 percent of those treated, compared to 15 percent getting the sham treatment.

The NBC.com story could have highlighted these limitations of the study and noted that many people with cluster headaches should not use the device (anyone with an implanted medical device, with high blood pressure, or with heart rhythm disturbances); and that overall the stimulator — while deemed safe and modestly effective for some — may have limited usefulness.

Despite its omissions, the article provided much more information than did a Reuters brief which disappoints with its lack of detail.

Why This Matters

You need to read the study in order to get to the details – and this story whiffed on some of the key details. As pointed out in an editorial accompanying the manuscript about the study in a journal:

“The primary endpoint was not achieved in this study – ie, differences in reduction in pain severity between active and sham groups were not significant…..conclusions drawn in this way are notorious for failing to replicate.”

Even in the abstract of the journal manuscript itself, the conclusion stated:

“the response rate was not significantly different (vs sham) for the total population”

Reliance on a company statement or news release isn’t usually going to give you that kind of information. And for most readers, that may have been the most significant piece of news.

Criteria

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

One of the first questions a cluster headache sufferer will have about the device is: how much does it cost? The article doesn’t offer any information about the cost of the device, or comparison to the costs of other treatment. A company release for British audiences described the device in terms of its savings compared to medications (“25% reduction in costs associated with acute treatments such as zolmitriptan, sumatriptan and inhaled oxygen”) but a search on the actual cost of the device and the cost of replacing the pre-loaded doses came up empty-handed.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The article uses percentages to vaguely describe the benefits (“appeared to help”) in the treatment vs sham treatment groups, but it does not mention the absolute number of patients in the two arms of the study, or what “help” meant. It relies mostly on complimentary statements by one of the investigators who conducted a study of the device, and by the company. But the major lapse in full disclosure comes with the absence of information about the failure of the clinical trial to reach its statistical goal. The apparent benefit of the device over the sham device was only seen after a pre-specified subgroup analysis. As noted in the accompanying editorial: “The primary endpoint was not achieved in this study – ie, differences in reduction in pain severity between active and sham groups were not significant…..conclusions drawn in this way are notorious for failing to replicate.”

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

The side effects were considered to be temporary and relatively minor, but should have been mentioned.

Dr. Silberstein noted, “It does not have the side effects or dose limitations of commonly prescribed treatments…” This is perhaps true but a closer examination of the research results reveals that 11 percent of the participants treated with the gammaCore device saw lip or facial dropping/pulling or twitching during the sham controlled trial and 7 percent experienced these side effects during the open label part of the trial (when both the researchers and volunteers knew which treatment was being administered).

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The study and the editorial accompanying its publication offer substantial detail about the considerable efforts made by the investigators to test a device and account for the real difficulties in “blinding” a study involving a rapid means of pain relief or prevention of pain. The article could have been greatly strengthened by offering information about the quality of the research and the weaknesses and limitations that exist in the results.

Does the story commit disease-mongering?

Satisfactory

There was no disease mongering about cluster headaches. The article provides context about the problem and how rare it is.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The story doesn’t quote any independent sources – not even from the editorial in the journal that offered a critique about the study’s limitations and conclusions.

Does the story compare the new approach with existing alternatives?

Satisfactory

The article did mention that there are “commonly prescribed treatments,” and it named some of them.

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

The article notes that the product is available in the EU and that the company will begin selling it in the U.S. later in 2017.

Does the story establish the true novelty of the approach?

Satisfactory

The article focused on the FDA’s approval of the device as the news hook. It did not repeat the manufacturer’s claim that the device is the first of its kind.

Does the story appear to rely solely or largely on a news release?

Not Satisfactory

The story relied significantly on the company’s news release and would have been strengthened by either directly vetting the claims or seeking an independent expert source to help with that.

The only quotes in the story came from a company statement.

Total Score: 4 of 10 Satisfactory

Comments

We Welcome Comments. But please note: We will delete comments left by anyone who doesn’t leave an actual first and last name and an actual email address.

We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, or profanity. We will also end any thread of repetitive comments. Comments should primarily discuss the quality (or lack thereof) in journalism or other media messages about health and medicine. This is not intended to be a forum for definitive discussions about medicine or science. Nor is it a forum to share your personal story about a disease or treatment -- your comment must relate to media messages about health care. If your comment doesn't adhere to these policies, we won't post it. Questions? Please see more on our comments policy.

COMMENTS button We welcome comments, which users can leave at the end of any of our systematic story reviews or at the end of any of our blog posts.

But before leaving a comment, please review these notes about our policy.

You are responsible for any comments you leave on this site.

This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.

We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.

We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.

There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”

The authors of the Retraction Watch comments policy urge commenters:

“Shed light, not just heat. Facts, challenges, disagreements, corrections — those are all fine. Attacking the person, instead of the idea or the interpretation, is neither acceptable nor helpful.”

We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.

And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.

The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.