Why does psychiatry so often get a free pass on standards of evidence?

Rob-Wipond.0561smallaThe following guest post is by Rob Wipond, a freelance investigative journalist. He contacted us with concerns regarding our review of a highly rated Philadelphia Inquirer story. We subsequently had him develop these comments into a blog post/critique of our review. Although Wipond raises some legitimate issues, we think some of his criticisms represent a misunderstanding of our mission and goals at HealthNewsReview.org, and so we’re taking this opportunity to clarify our objectives. We’ve appended a rebuttal and response to the bottom of the post which we hope you’ll also take a few moments to read. 


A HealthNewsReview.org team gave a five-star, 9/10 glowing rating to a Philadelphia Inquirer article about an electro-mechanical device that ostensibly helps people avert experiences of panic. I would give this review by HealthNewsReview.org a failing grade. And though psychiatry has certainly produced more dangerous interventions than this breath-training device seems to be, the HealthNewsReview.org review nevertheless illustrates some of the common ways in which hyperbolic psychiatric and psychological claims frequently get free passes from otherwise thoughtful medical critics. I believe these deeper problems need to be more widely examined and discussed, so I’ve written a review of the HealthNewsReview.org review.

1) Is there disease-mongering with dubious statistics occurring?

freespira device

The Freespira device.

The HealthNewsReview.org review begins, “As the [Inquirer] article states, several million people in the U.S. have panic disorder, and millions more experience panic attacks…” And the original Philadelphia Inquirer article reports, “Six million people in the United States have panic disorder, and 27 million have episodes of panic.”

So both the Inquirer and HealthNewsReview.org are implying that some 33 million people — 10% of the U.S. population — have a condition that makes them potential users of this medical device. The sheer extent of “disorder” or “disease” implied by that number should make us immediately alert to the likelihood that we could be dealing with “disease-mongering” – the creation or exaggeration of a malady for the purpose of increasing sales of a medical treatment. So we must ask, where did those numbers come from? Neither the Inquirer nor HealthNewsReview.org cite a source for those astounding numbers; they are simply uttered as if they are established fact.

That people sometimes have experiences of panicky feelings, and some more frequently and intensely than others, is certainly a given. But “panic disorder” can in no way be described as an actual medical disorder or disease. As described in the Diagnostic and Statistical Manual of Mental Disorders, a “panic attack” is nothing more than a surge of fear or discomfort which leads to at least four of thirteen symptoms such as sweating, discomfort, tingling, accelerated heart rate, or light-headedness. A person then supposedly has “panic disorder” if the “panic attack” is bothersome enough to make the person concerned about the experience potentially recurring. To wit: This “symptomology” outlines what pretty much anyone can and often does experience when feeling really anxious.

Perhaps not surprisingly then, there exists not one scientifically validated, medically reliable method to diagnose this “panic disorder” that HealthNewsReview.org states millions of Americans “have.” All methods of diagnosing it more closely resemble psychiatric screening tests, which are notorious for producing extremely high rates of false positives – even after we grant the questionable presumption that their “true positives” are in fact accurate. Indeed, the source of the majority of these kinds of population prevalence rates for “mental illnesses” is simply mental health screening tests administered through telephone surveys, like those used in the US Substance Abuse and Mental Health Services Administration’s national behavioral health survey.

That doesn’t mean people don’t experience things like panic – it just means it’s grossly misleading to imply to the public that we have any reliable way of scientifically, medically distinguishing between a person who has “panic disorder” and a person who sometimes has completely normal, common experiences of panic. How is any of this different than the kind of disease-mongering that labels natural “low testosterone” in aging men as a widespread “disease” requiring “treatment”?

2) Is there treatment-mongering with vague claims occurring?

Again in part repeating claims from the Inquirer article, the HealthNewsReview.org review states, “[panic disorder] is very treatable, although treatment can be time consuming (meeting 2-3 times/week for 3-4 months with a therapist) and fairly expensive. It usually includes the use of drugs as well entailing visits to a psychiatrist.”

Why is HealthNewsReview.org even using a vague term like “treatable”? Treatable is defined as “able to be treated.” Anything and everything can technically be “treated” – that’s why the term is a favorite of disease-mongering press releases and news articles. The medically important question is, whether or not that treatment is effective at curing a diagnosable disease and, if so, how effective that treatment is. And as HealthNewsReview.org frequently states in its reviews, it’s vital in these kinds of articles to convey the precise details about efficacy as clearly and accurately as possible to readers. Because neither the HealthNewsReview.org review nor the Inquirer article present any specific evidence or statistical details to clarify their suggestive claims that “panic disorder” can be scientifically diagnosed and effectively cured, their statements can only be regarded as misleading “treatment-mongering” —  the suggestive exaggeration of the efficacy of treatments.

3) Is important context for exorbitant costs omitted?

The Inquirer story notes that many people, when they become very anxious and start to experience symptoms such as sweating, discomfort, tingling, accelerated heart rate, or light-headedness, often start either unconsciously hyperventilating or deliberately trying to breathe more deeply to calm themselves. However, the Inquirer notes, hyperventilating and deep breathing can themselves produce physical and mental sensations like sweating, discomfort, tingling, accelerated heart rate, or light-headedness, which can in turn cause people to feel even more panicky about what is happening to them. Hence, this new medical device helps train people to breathe more shallowly — at a cost of $500 monthly.

The HealthNewsReview.org review states that the story should have explained that “if people can train themselves at home to use this device, it would save visits (time and money) to psychologists and psychiatrists.”

That’s certainly true. But the Inquirer and HealthNewsReview.org could perhaps have instead better mentioned that this $500/month assistance likely achieves about the same effects of reducing oxygen intake as does many physicians’ long-standing advice to treat hyperventilation syndrome by just breathing into a paper bag.

It would also be worth mentioning that, for a typical one-time $15 drop-in fee, virtually any yoga center in any town offers training in “pranayama” or breathing techniques that usually specifically highlights how breathing rapidly and/or deeply can quickly induce intensification of inner experiences, and how breathing more shallowly again usually reduces those effects. That certainly beats spending $500-a-month on a breath-training device or throwing money at psychologists and psychiatrists for therapies and drugs.

4) Is there promotion of unverified claims and no discussion of harms?

This is in some ways the strangest aspect of the HealthNewsReview.org review: It correctly identifies many problems in the Inquirer article, but then suggests that they aren’t really of any substantive concern.

HealthNewsReview.org notes that the Inquirer story does not mention if any of the studies cited were “randomized, controlled, peer-reviewed, or even published,” and does not identify which studies were sponsored by the manufacturer. The Inquirer story also hypes the device’s FDA clearance, HealthNewsReview.org points out, but does not mention the “exceedingly low standard” the FDA uses which “really says nothing about the product’s effectiveness or safety”. The Inquirer also makes a treatment effectiveness claim that HealthNewsReview.org says is “a stretch” because there is no evidence whatsoever to support it.

However, HealthNewsReview.org writes a number of times that it’s giving “a pass” and giving “the benefit of the doubt” on these critical issues, and then concludes by giving the Inquirer a stellar 9 out of 10 rating for fulsome scientific coverage. But why? If it was a press release about a new drug for hypertension, would HealthNewsReview.org give the same “benefit of the doubt” to the pharmaceutical company pushing it? Is there something about psychiatry that makes it not need to be held to the same standards of science as other medical specialties?

HealthNewsReview.org also complains that there is “no explanation of the harms” of the device. This leads to the only “unsatisfactory” mark in the whole review – yet even then HealthNewsReview.org seems more concerned about the possible harms caused to people with a “real mental health need” who dropped out of the device trials. But what other harms might there be that could be important to mention? How about the very obvious potential harms to anxiety-ridden people training themselves to breathe shallowly and reduce oxygen intake all the time!

 5) Is a press release used as primary source?

The HealthNewsReview.org review lauds that the Inquirer story “did not rely on a news release.” But is that correct? The manufacturer’s press release states: “In the United States, over 6 million people annually suffer from panic disorder. Panic episodes which affect a total of 27 million adults in the U.S. every year are also a common component of other anxiety disorders.” No source is cited. But now we know where the dubious statistics that serve as a foundation for the Inquirer article came from.

All together, then, this HealthNewsReview.org review gets a failing grade, because it seems to show that HealthNewsReview.org is not holding psychiatric claims to the same standard of evidence to which it holds other medical claims. That needs to change because, as this whole situation shows, in some ways news coverage of psychiatry and psychology needs thoughtful critics even more than medicine does.

_____________________________________________________________________

headshotcroppedtightHealthNewsReview.org Rebuttal

Honey vs. bile: Which works better for improving health care journalism?

By Kevin Lomangino, Managing Editor

As our readership has grown since the January 2015 relaunch of HealthNewsReview.org, we’ve increasingly heard from readers who wish to push us in different directions. One camp, whose views are loosely reflected by Rob Wipond’s critique, feel that we aren’t hitting hard enough on stories that fail to expose key problems with whatever research is being reported on. For example, Wipond is unhappy that we repeatedly “gave the benefit of the doubt” to the Inquirer story despite some obvious deficiencies in the coverage. He suggests that our standards are too lax specifically when it comes to stories about psychiatry, and that we wouldn’t have been so generous with pharma-themed stories about a new hypertension drug.

But on the other end of the spectrum are critics, often journalists, who complain that we are unduly harsh in our reviews. They say that our criteria are unrealistic in this age of reduced newsroom staffs and budgets. They say we’re too demanding at a time when journalists are continually being squeezed to pump out more copy in less time and with fewer resources.

Faced with the pull of these competing forces, we always revert back to our guiding principles and consider about how we can best advance them. Our over-arching goal is “to improve the public dialogue about health care,” and a major part of that objective is encouraging journalists to write more complete, balanced stories that provide an accurate portrait of the benefits, harms, and costs of medical interventions.

And having reviewed some 2,100 stories about medical interventions and corresponded with dozens of journalists about those reviews, we think there’s no question that honey draws more flies than bile. We’ve found that if you want to engage journalists in a conversation about how to improve health news, you simply can’t keep using a hammer. Bludgeoning journalists with negative comments is not an effective strategy for engaging with them or encouraging them to write more thoughtfully.

I’m not saying that HealthNewsReview.org is in the business of putting out puff pieces that ignore significant problems in news coverage. Rather, the approach we’ve settled on is to recognize any reasonable attempt to address our criteria, and to record a Satisfactory rating when there is such an attempt. However, we always point to missing information and provide constructive suggestions for improvement in the comments for each criterion. This way, journalists rightfully receive credit for effort toward meeting our standards, and yet hopefully remain open to considering our constructive criticism for their next story. We think that’s a much better outcome than if we grade with an unforgiving eye, and as a result get written off by journalists as out of touch and unrealistic.

In the case of the Inquirer story, we thought there was enough balance to justify Satisfactory ratings in areas where Wipond would’ve given a failing grade. With respect to evidence quality, for example, the story calls the research on the device “a small early study,” and it quotes an independent expert who compares the device unfavorably to exposure therapy, a more proven option. The story also makes it clear that the device wouldn’t be considered “ready for prime time” under normal circumstances, which is why the insurer is testing it. This was a close call, but we thought those caveats gave readers a sense that the evidence on the device remains shaky. Of course, we would’ve liked to see many of the additional details that we mentioned in the review regarding the study design and limitations. But by “giving the benefit of the doubt” on the rating, we feel the reporter will be much more likely to consider those missing elements for her next story.

We also think it’s incorrect and unfair to assume, as Wipond does, that the Inquirer story relied on a company news release for its statistics on panic disorder. The statistic that 6 million Americans suffer from panic disorder is widely cited by a number of organizations including the Anxiety and Depression Association of America. Moreover, there’s evidence of considerable original reporting in this story that goes way beyond any news release. So while it’s certainly possible that the story did rely on that news release for its prevalence statistics, we think it would be inappropriately presumptuous of us to ding the story on that basis. Again, we think it’s important to give the benefit of the doubt in these situations.

That being said, I think Wipond was right to call us out for repeating the Inquirer’s unsourced statistics on panic disorder. We’ve often dinged stories for passing along such statistics without establishing where they came from, and we shouldn’t ourselves have done what we’d criticize a news story for doing. We also could’ve been more precise, as Wipond points out, when discussing the “treatability” of panic disorder and what that means in terms that matter to people.

But I reject the idea that our review represents any sort of special treatment or reduced standard for psychiatric claims. The benefit of the doubt is something that we extend to all news stories and the journalists who write them, regardless of the subject matter. We think it’s the best way for our project to improve the public dialogue about healthcare.

You might also like

Comments (4)

We Welcome Comments. But please note: We will delete comments left by anyone who doesn’t leave an actual first and last name and an actual email address.

We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, or profanity. We will also end any thread of repetitive comments. Comments should primarily discuss the quality (or lack thereof) in journalism or other media messages about health and medicine. This is not intended to be a forum for definitive discussions about medicine or science. Nor is it a forum to share your personal story about a disease or treatment -- your comment must relate to media messages about health care. If your comment doesn't adhere to these policies, we won't post it. Questions? Please see more on our comments policy.

Edward Dantes

December 23, 2015 at 9:07 pm

Wow, that’s a pretty poor rebuttal. Wipond raised good points, such as about the lack of clear reliability and validity for so-called panic disorder, and Health News Review was unable to respond except by criticizing Wipond for spewing “bile.” Health News Review also used the classic defense of “appealing to authority” (saying that the APA says that 6 million people have anxiety disorder); but that is not actual evidence.

    Kevin Lomangino

    December 24, 2015 at 12:56 pm

    Edward,

    Thanks for your comment. I did not intend to suggest that Wipond was “spewing bile” and indeed acknowledged the legitimacy of some of his points. I was referring more generally to the idea that it’s important to acknowledge a story’s efforts to meet our criteria with positive feedback, rather than dwell exclusively on the things that could/should have been done better (in other words, HealthNewsReview needs to offer more to journalists than bile). And I think you misunderstood the point of my citing APA’s 6 million figure. I was not appealing to their authority, but showing that there are any number of places that the story could have gotten this figure other than the company news release. So it would have been problematic for us to accuse the story of relying excessively on that news release based on that number.

    Kevin Lomangino
    Managing Editor

J. Mathers

December 24, 2015 at 4:31 pm

Personal attacks (a few of which were overt), whining, and rhetoric aside, the Managing Editor’s art-of-the-state definition of the “benefit of the doubt” seems demonstrably similar to that of “willful blindness” under a literary version of the ticking time bomb scenario necessitating compromise of journalistic principles and standards. Sadly, aside from slippery synonyms what is clear from this limping rebuttal and strikingly inadvertent expose is that readers asking “who benefits?” from such persistent strains of doubt or willful blindness will rarely find, occupying the answer, themselves.
Thank you Mr. Wipond for persistently being moved in a spirit of civility to strive for truth in fairness, evidence and facts as well as having the strength and courage to stand for your chosen profession: investigative journalism whilst inviting as well as encouraging, others to do the same.

Paul

December 25, 2015 at 2:51 pm

Edward Dantes nailed it: the response was poor.
“Bile?” Wipond gave an analysis. Deal with his anlaysis.
With regard to the APA, might look critically at what comes from the APA as it has its hand in the drug companies cookie jar.