While nicely written and generally on the mark with its key messages, this Reuters story could have done a better job explaining how this new study — a systematic review — differed from the dozens of reviews that preceded it and why it is newsworthy. The key difference is that the study used a more robust methodology than previous reviews, and the findings suggest that acupuncture delivered by experts was slightly more effective than a “sham” acupuncture procedure (i.e. the effects of acupuncture are more than just a placebo). As such, there is reason believe that the new study provides a stronger assessment of the benefits of acupuncture than previous studies that have reported inconsistent findings. And clinicians may see this as justification for referring more patients to acupuncture therapy.
There have been a mind-numbing number of reviews of the acupuncture research in recent years, many of which have reported conflicting conclusions about the effectiveness of this approach. To understand why a new review merits our attention, stories should try to articulate how its methods differ from previous research. Otherwise, there’s no way to tell if the conclusions are more believable than what we’ve already heard.
Interestingly, the two stories we reviewed demonstrated “the glass half full/half empty” phenomenon, at least in the story headlines. The Reuters story noting “limited benefit” and USA Today noting “Acupuncture works.” Clearly the study provided a very mixed message
“A typical acupuncture session runs for about $100 and is often not covered by health insurance,” according to the story.
The story puts the benefits in terms readers can understand and that accurately reflects the magnitude of the benefit: “…about 50 percent of patients had their symptoms cut in half with acupuncture, compared to almost 43 percent of those treated with sham acupuncture and 30 percent with no acupuncture-like therapy.”
The story says that acupuncture is safe when performed by a qualified and licensed acupuncturist, which is true. The story didn’t report on the flip side of this statement, which was addressed by another recent review of the evidence — i.e. that when acupuncture is delivered by unqualified practitioners, there are dozens of published cases of severe adverse reactions including death. Tough call here, but we think the story didn’t quite meet our standard.
The story doesn’t clearly explain what sets this meta-analysis apart from the many other evidence reviews that have been conducted in recent years. One of the main differences is that it omitted studies where participants might have been able to tell whether they were receiving a real or sham acupuncture treatment. In addition, the study was an “individual patient data” meta-analysis, which the authors note is a type of study that differs from previous “summary” meta-analyses. They are considered superior to summary meta-analyses “because they enhance data quality, enable different forms of outcome to be combined, and allow use of statistical techniques of increased precision.”
There was also no mention of any possible limitations of studies on acupuncture. Importantly, there’s no way to do a study where the acupuncturists themselves don’t know what kind of treatment they’re giving (real or sham), and this knowledge could introduce bias. Such bias might help account for the small difference in benefit shown between real and fake acupuncture in the review.
The story consulted one of the study authors, as well as the head of the department of Asian Medicine & Acupuncture Research at the University of North Carolina at Chapel Hill School of Medicine. Both of these sources are biased either by their affiliation with the study or their general support and belief in the effectiveness of acupuncture. (In an admirable display of transparency, the Asian medicine expert acknowledges that he is a “biased clinician.”) The story would have been improved with the inclusion of comments from an expert in pain treatment who is not invested in the study or in acupuncture as a treatment. Such an expert might well be considered biased against acupuncture, but at least readers would be getting both sides of the argument.
The story mentions medications, physical, and talk therapy as potential options for chronic pain. A complete story however might also have included some information about the standard treatments and how they compare to placebo. When you look at physical therapy for chronic low back and NSAIDs the results are not all that impressive
While acupuncture is mainstream enough to be available in metropolitan areas, those living in rural places will often have a tough time finding a licensed and qualified practitioner. Although the story could have been more emphatic on this point, it does mention the “inconvenience of getting to a clinic” as one of the main downsides of therapy. We’ll give the benefit of the doubt.
There have been plenty of systematic reviews of acupuncture that have reported inconsistent findings. (The review referenced above found 57 systematic reviews of the acupuncture evidence.) The story does not mention this previous research.
The story has enough independent reporting for us to be confident it wasn’t based on a press release.
Systematic, Criteria-Driven
Monday at #ADA2021, an infectious disease epidemiologist, a journalist, and a researcher shared perspectives on communication of risk in the era of COVID-19: http://ow.ly/ROfR30rMeE7 @AstleyCM @garyschwitzer @berthahidalgo @ADA_DiabetesPro
Shoutouts to @susan_bewley @drkevinknopf @MichaelBaum11 in this call for more than fawning coverage from press releases for these kinds of stories.
Also, linked history inside goes back 5 years just on Grail apparently trying to become Holy Grail. https://twitter.com/garyschwitzer/status/1408849696416841731
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like