Summary of study on migraine surgery: ‘dramatic’ results but what about the alternatives?

Does the news release adequately discuss the costs of the intervention?

Not Satisfactory

The news release did not include the cost of this migraine surgery — nor did it really say what a migraine surgery entails, except for briefly comparing it to a cosmetic forehead-lift. According to the website linked in the news release, a cosmetic forehead-lift (also called a “brow lift) costs an average of $3,403. However, a look at anecdotal articles online reveals that the cost can often be much more if not covered by insurance — up to $30,000 in some cases.

Does the news release adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The description of benefits of this research is a bit cloudy, since it seemed to have two endpoints: first, the study attempted to quantify the extent of disability in migraine patients by using the Pain Self-Efficacy Questionnaire (PSEQ). Second, it looked to see if PSEQ numbers improved following migraine surgery. The release says that the study found that after the migraine surgery, patients improved an average of 112% over their baseline PSEQ score. However, as we have pointed out on numerous occasions, these types of statistics often don’t tell us much. We don’t know if this means that migraine patients’ self-efficacy went from “bad” to “great,” or just from “terrible” to “less terrible.” Did this improvement mean that patients were able to go back to work, or that they were able to get out of bed? Without concrete numbers, it is hard to tell.

Does the news release adequately explain/quantify the harms of the intervention?

Not Satisfactory

This is perhaps one of the most egregious omissions of the news release: not mentioning any potential harms of a surgery that directly impacts the nerves and muscles of the face. While the method of this surgery is not included in the news release, a look at the Mass General web page where the lead author works describes the surgery as a “release” of nerves in the eye, forehead, back of head, and nasal regions. Depending on the extent of the surgery, a patient may be required to undergo anesthesia in an operating room. The risks are not mentioned on this page either, but a look at the risks of the similar brow-lift procedure shows that complications may include facial nerve damage or paralysis, infection, and scarring.

Does the news release seem to grasp the quality of the evidence?

Not Satisfactory

This news release scores poorly on the quality of evidence for several reasons. First, neither the news release nor the study itself described any limitations of the study (though it is safe to assume there are several, including the small sample size). Second, and most important, the study was based entirely on results from patients’ self-reported questionnaires. Self-reported data is notoriously susceptible to bias–patients may have different interpretations of the questions, may want to consciously or unconsciously please the researchers, or may be dishonest because they are embarrassed they don’t meet what they think is the “desired” outcome. That the news release did not mention this obvious limitation is a serious flaw. Finally, it is important to note that the Pain Self Efficacy Questionnaire (PSEQ) only measures the patients’ feelings of self-efficacy and daily functioning, not whether or not the severity of their pain has actually improved. Someone could be feeling the same amount of pain after the surgery, but if they felt that they were functioning better in their daily life than their questionnaire score would increase.

Does the news release identify funding sources & disclose conflicts of interest?

Not Satisfactory

The news release does not mention the funding sources, and that earns a Not Satisfactory rating here. The study does disclose that none of the researchers have conflicts of interest.

Does the news release compare the new approach with existing alternatives?

Not Satisfactory

This is another important criterion which the news release did not meet. And even though the published study did not address alternatives either, we still encourage news release writers to include this background for interested readers — both patients and journalists.

Presumably, patients who turn to surgery have been resistant to the usual interventions and if so, this should be clarified in the news release.

There are many alternatives to migraine treatment, from prophylactic daily medications to Botox injections. In fact, Botox injections operate on the same mechanism that the migraine surgery does: the botulinum toxin is injected into facial nerves to paralyze them and release pressure on the nerves surrounding the face. Botox injections for migraines have been well-studied and proven to be effective; the procedure was approved by the FDA in 2010. Unlike the surgery however, the Botox injections are not permanent — though they also appear to carry fewer side effects. It would make the most sense to compare the surgery to Botox injections, but neither the researchers nor the news release did this. Nor did they compare the migraine surgery to any other migraine treatments. Instead, they compared results from the patient questionnaire to patients that suffered other chronic pain ailments, such as lower back pain. They said that migraine patients reported greater improvement than lower back pain patients, but this tells us very little because the two conditions are so different.

Does the news release establish the availability of the treatment/test/product/procedure?

Not Satisfactory

The news release made no mention of how or where patients could obtain migraine surgery. Online it seems that these surgeries are offered at many boutique plastic surgery centers around the country–though interestingly it is only plastic surgeons who seem to offer this treatment, not neurologists (the doctors that most often treat migraine sufferers). The news release and the study also did not mention that not all migraine patients will be eligible for the procedure. Although we know little about migraines, it is known that they are triggered in different people through different mechanisms. A woman who has hormonal migraines, for example, might not benefit from nerve decompression.

Does the news release include unjustifiable, sensational language, including in the quotes of researchers?

Not Satisfactory

The headline of the news release was taken from a quote from the lead author, who said that migraine surgery can lead to “dramatic improvements” in the daily functioning of migraine patients. Yet, as mentioned above, based on the quality of the evidence and benefits we have no idea just how big these improvements are. Is a patient that was formerly severely disabled now able to work and maintain a normal quality of life? All we know is that the patients improved in one self-reported measurement, but we don’t know how much improvement should be considered “dramatic.”