Health News Review

Comparing different stories on the same topic helps to point out relative strengths and weaknesses. The Reuters story was edged out by the competing AP coverage. Lack of context regarding previous research was the main shortcoming here.

Our Review Summary

While nicely written and generally on the mark with its key messages, this Reuters story could have done a better job explaining how this new study — a systematic review – differed from the dozens of reviews that preceded it and why it is newsworthy. The key difference is that the study used a more robust methodology than previous reviews, and the findings suggest that acupuncture delivered by experts was slightly more effective than a “sham” acupuncture procedure (i.e. the effects of acupuncture are more than just a placebo). As such, there is reason believe that the new study provides a stronger assessment of the benefits of acupuncture than previous studies that have reported inconsistent findings. And clinicians may see this as justification for referring more patients to acupuncture therapy.


Why This Matters

There have been a mind-numbing number of reviews of the acupuncture research in recent years, many of which have reported conflicting conclusions about the effectiveness of this approach. To understand why a new review merits our attention, stories should try to articulate how its methods differ from previous research. Otherwise, there’s no way to tell if the conclusions are more believable than what we’ve already heard.

Interestingly, the two stories we reviewed demonstrated “the glass half full/half empty” phenomenon, at least in the story headlines.  The Reuters story noting “limited benefit” and USA Today noting “Acupuncture works.”  Clearly the study provided a very mixed message


Criteria

Satisfactory

Does the story adequately discuss the costs of the intervention?

Satisfactory

“A typical acupuncture session runs for about $100 and is often not covered by health insurance,” according to the story.

Satisfactory

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Satisfactory

The story puts the benefits in terms readers can understand and that accurately reflects the magnitude of the benefit: “…about 50 percent of patients had their symptoms cut in half with acupuncture, compared to almost 43 percent of those treated with sham acupuncture and 30 percent with no acupuncture-like therapy.”

Not Satisfactory

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

The story says that acupuncture is safe when performed by a qualified and licensed acupuncturist, which is true. The story didn’t report on the flip side of this statement, which was addressed by another recent review of the evidence — i.e. that when acupuncture is delivered by unqualified practitioners, there are dozens of published cases of severe adverse reactions including death. Tough call here, but we think the story didn’t quite meet our standard.

Not Satisfactory

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story doesn’t clearly explain what sets this meta-analysis apart from the many other evidence reviews that have been conducted in recent years. One of the main differences is that it omitted studies where participants might have been able to tell whether they were receiving a real or sham acupuncture treatment. In addition, the study was an “individual patient data” meta-analysis, which the authors note is a type of study that differs from previous “summary” meta-analyses. They are considered superior to summary meta-analyses “because they enhance data quality, enable different forms of outcome to be combined, and allow use of statistical techniques of increased precision.”

There was also no mention of any possible limitations of studies on acupuncture. Importantly, there’s no way to do a study where the acupuncturists themselves don’t know what kind of treatment they’re giving (real or sham), and this knowledge could introduce bias. Such bias might help account for the small difference in benefit shown between real and fake acupuncture in the review.

 

Satisfactory

Does the story commit disease-mongering?

Satisfactory

There was no disease-mongering here.

Not Satisfactory

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The story consulted one of the study authors, as well as the head of the department of Asian Medicine & Acupuncture Research at the University of North Carolina at Chapel Hill School of Medicine. Both of these sources are biased either by their affiliation with the study or their general support and belief in the effectiveness of acupuncture.  (In an admirable display of transparency, the Asian medicine expert acknowledges that he is a “biased clinician.”) The story would have been improved with the inclusion of comments from an expert in pain treatment who is not invested in the study or in acupuncture as a treatment. Such an expert might well be considered biased against acupuncture, but at least readers would be getting both sides of the argument.

Satisfactory

Does the story compare the new approach with existing alternatives?

Satisfactory

The story mentions medications, physical, and talk therapy as potential options for chronic pain.  A complete story however might also have included some information about the standard treatments and how they compare to placebo.  When you look at physical therapy for chronic low back and NSAIDs the results are not all that impressive

Satisfactory

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

While acupuncture is mainstream enough to be available in metropolitan areas, those living in rural places will often have a tough time finding a licensed and qualified practitioner. Although the story could have been more emphatic on this point, it does mention the “inconvenience of getting to a clinic” as one of the main downsides of therapy. We’ll give the benefit of the doubt.

Not Satisfactory

Does the story establish the true novelty of the approach?

Not Satisfactory

There have been plenty of systematic reviews of acupuncture that have reported inconsistent findings. (The review referenced above found 57 systematic reviews of the acupuncture evidence.) The story does not mention this previous research.

Satisfactory

Does the story appear to rely solely or largely on a news release?

Satisfactory

The story has enough independent reporting for us to be confident it wasn’t based on a press release.

Total Score: 6 of 10 Satisfactory


We Welcome Comments

But please note: We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, profanity or any from anyone who doesn't list what appears to be an actual email address. We will also end any thread of repetitive comments. We don't give medical advice so we won't respond to questions asking for it. Please see more on our comments policy.