Note to our followers: Our nearly 13-year run of daily publication of new content on HealthNewsReview.org came to a close at the end of 2018. Publisher Gary Schwitzer and other contributors may post new articles periodically. But all of the 6,000+ articles we have published contain lessons to help you improve your critical thinking about health care interventions. And those will be still be alive on the site for a couple of years.
Read Original Story

A first-person account of insomnia therapy focusing on “data, not pills”

The Evidence Points to a Better Way to Fight Insomnia

Our Review Summary

man in bed with eyes opened suffering insomnia and sleep disorderThis story describes the step-by-step path of an individual patient/author seeking information to compare drugs to cognitive behavioral therapy (CBT) in treating his own insomnia. It isn’t a news story about research, although it draws from a recent Annals of Internal Medicine study and other studies. The writer mixes his own personal experience with a thorough discussion of the data. The result is a very helpful and readable overview of the evidence on insomnia treatments.

 

Why This Matters

Insomnia is very common complaint to primary care physicians.  Many patients specifically request a sleeping pill, believing them to be more effective than non-pharmacologic approaches. This story provides value by emphasizing the effectiveness of these non-drug approaches.

 

Criteria

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

We wish this story comparing effectiveness of different therapies for insomnia had compared costs as well. How often is cognitive behavioral therapy for insomnia covered by insurance? Is there a co-pay per visit? What is a typical number of treatments, and what might it cost? How does that compare to taking some of the common drugs?

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Satisfactory

The story carefully lists several benefits from the Annals study: patients treated with CBT fell asleep almost 20 minutes faster and were awake in the night almost half an hour less and their total time “sleeping in bed” was increased by 10 percent.

We have only two minor quibbles. One is that the writer compares himself to the study participants, and he may not meet the same criteria as they did. “For me, and many patients, C.B.T. works. And as studies show, it works better than drugs,” he wrote. But the Annals study had “narrow inclusion criteria” that perhaps deserved some comment.

The second is that total sleep time gain is described in relative terms with a figure of 10%. The absolute increase in sleep time was 7 minutes, according to the study.

Does the story adequately explain/quantify the harms of the intervention?

Not Applicable

There is no known harm to CBT for insomnia. There are myriad harms to commonly used sleeping medications. Although the story would have been strengthened by explaining those harms (further dissuading people from medication, which is what physicians recommend), the story’s failure to mention harms of medications when the article itself was about CBT seems overly punitive. We’ll rate it Not Applicable.

Does the story seem to grasp the quality of the evidence?

Satisfactory

The writer-as-patient tells us that “Data, not pills, was my path to relief.” He provides high-quality evidence to support his path to his own personal relief. He doesn’t quote others or balance his own interpretation with the comments of experts, which might have made the story stronger.

Does the story commit disease-mongering?

Satisfactory

There is no exaggeration of the impact of insomnia.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

While the story is a personal one, we think it could have been strengthened with the opinion of a primary care provider who commonly treats insomnia. Any story that makes health claims should have their facts/interpretation run by an expert.

Does the story compare the new approach with existing alternatives?

Satisfactory

The story does a good job of explaining the alternatives of CBT, or behavioral vs. drug therapy for insomnia. It might have been interesting to compare “no treatment” at all to the others.

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

We are not given a lot of detail, but consider both CBT and medications to be widely available. This is particularly true because the writer makes it clear that an online program was helpful (rather than requiring a specialized clinic or provider).

Does the story establish the true novelty of the approach?

Satisfactory

The story carefully gives us context for the research described, which is not novel.

Does the story appear to rely solely or largely on a news release?

Satisfactory

The story does not rely on a news release.

Total Score: 7 of 9 Satisfactory

Comments (4)

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Keith

June 12, 2015 at 10:50 am

I don’t see why “an opinion of a primary care provider who commonly treats insomnia” would have strengthened the story. PCPs’ opinions vary, and it would be easy to find some PCP somewhere to say something consistent with the story. While it is true that “[a]ny story that makes health claims should have their facts/interpretation run by an expert,” a quick look at Austin Frakt’s record would indicate that he is indeed a well-regarded expert on health care and the interpretation of health research. In fact, given the widespread evidence indicating that physicians have difficulty interpreting statistical evidence,* an uninformed opinion by a PCP may have been deleterious.

*See e.g. http://www.ncbi.nlm.nih.gov/pubmed/24756486

Reply

Kevin Lomangino

June 12, 2015 at 11:11 am

The author of this piece, Austin Frakt, PhD, emailed us with the following feedback which he’s given me the go-ahead to post here. I will respond below.

Kevin Lomangino
Managing Editor

Thanks. Very thought provoking. I always welcome feedback, and I think yours is entirely reasonable, but not without its own limitations. I gave consideration to each of your concerns in the process of writing the piece, but had my own concerns about including cost information and expert opinion quotes, as indicated below.

1) Your review didn’t mention that I’m an expert at evidence review and comparative effectiveness research, by profession. Am I equivalent to any other journalist? My credentials should be clear from my bio, to which my piece links. (You might say that I should have just mentioned my credentials in my piece, which is fair, though I doubt that would have gotten through the editors, given they’re in the bio.)

Along these lines, I made some comments on Twitter about using expert opinion quotes. I’m not so sure they’re as helpful a guide as some may think. Here’s one such tweet, which has others in reply. For reasons explained there, in areas in which I’m able to read and assess the research literature, I often don’t go to experts, apart from having my piece peer reviewed.

2) One should always be on guard against using a media piece for marketing purposes, or appearances thereof. This relates to cost, because I worried about over-promoting CBT in the piece by seeming to sell it on efficacy AND cost. I had hoped that readers would find it rather obvious that an online course (to which they could click through and evaluate the cost themselves) would probably be cheaper to them than drugs. But, really, this varies by insurance so I couldn’t possibly know for a specific individual. I followed up with a comment on Twitter about cost of CBT: ~$40 online. (It looks like my recollection is about right.)

I suppose a final point is that your review is really not of the author, but of the piece, which is influence by editors as well. Did you forward your review to the Upshot editors? Do you routinely forward your reviews to editors? I’m just curious. It’d be a good idea if you don’t.

Frakt followed up with an email stating the following:

I’ll just emphasize to you that I’m presenting an alternative point of view for discussion purposes. I’m not 100% sure I’m right, but I think there are at least legitimate points of view on both sides.

Also, your review will serve the purpose of reinforcing the practice I already have: getting expert input when I think I need it. It won’t cause me to seek quotes from other experts for all my pieces. That’s just not what I do as a matter of routine, and given my credentials, I think that’s defensible and fine.

Not sure about cost either, as a matter of routine, but I will think about it more than I would have otherwise. That’s some progress! :)

Reply

    Kevin Lomangino

    June 12, 2015 at 1:01 pm

    Austin,

    Thanks so much for taking the time to write to us with this feedback and for allowing us to post it as a comment.

    First off, I will be the first to agree that our criteria have limitations. When viewed in aggregate, the criteria will almost always, in my view, give you a good overall sense of the quality of the coverage. But in some cases, a story may rate “Not Satisfactory” on certain criteria without substantially affecting the quality of the coverage. And certain criteria may not be applicable for certain stories. When reviewing stories, we often discuss these concerns among the review team and do our best to reflect these judgement calls in our review comments for each criterion.

    With that as context, I’ll respond to your discussion points:

    1. I agree, our review should have mentioned your expertise as a health economist rather than describing you simply as a patient/author. One of our reviewers commented about it to me in an email, but I didn’t add it to the review. That does impact one’s assessment of how to interpret the story and would have been useful to point out. By the same token, as you say, it would have been useful to have that context in the original piece. How much info to include in the text itself and when to link out to that info is always a tough call. (Who knows many readers in the general audience will click on your bio to find out more about you?)

    2. Regarding independent sources, we had a disagreement on this point within the review team. One of the reviewers wanted to rate your piece “Not Applicable” on that point, given your background and the first-person style of writing that reads more like an opinion piece. To be consistent with other reviews, I thought we should rate this Not Satisfactory and hold you to the same standard as other journalists. Why? First, it’s on the New York Times UpShot blog, where journalists’ work appears. Second, although you may be an expert in comparative effectiveness research, I would argue that you don’t bring the same skills and perspective as someone who treats people with insomnia on a daily basis. Could that perspective have added to the piece? Possibly. I might say probably. Hard to say without talking to them.

    On that point, I would take issue with some of the comments on your Twitter stream about “gaming” the quotes you might get from such a source. Of course, as journalists we know that it’s easy to spin a story one way or another using quotes. But I think you have to go into such discussions with an eye toward learning the truth and reporting it — not just supporting your own thesis. Isn’t it possible that you could learn something useful from such a source, and that it would’ve added to the coverage? That’s what I’d be looking to get out of such a conversation. But it’s a point worth discussing and I hope others will jump in on that point.

    With that being said, the fact that you peer review your pieces before publication is very interesting, and it alleviates much of the concern I would have about not having an independent source quoted in the story. But readers don’t know you do that, and we don’t know who does the peer review. We look for journalists to name who provided independent perspectives and to disclose potential conflicts of interest in those sources. For me at least, an important part of talking with outside experts is to test my assumptions/thinking against their expertise. Peer review may accomplish that, but again it’s not reflected to the end reader. And maybe you interviewed people but decided the quotes weren’t worth publishing. Again, the criteria aren’t perfect….

    3. Regarding cost, I think it’s possible to present cost information in a way that is not overly promotional and gives people a sense of what the options are. Speaking personally, the fact that your program cost only $40 is surprising and I think would be to many readers. That’s cheap. I think it was worth including in the coverage along with what a course of in-person therapy might cost. But again, other people may disagree as to where the dividing line is between useful info and promotion. The fact that you are thinking about these issues in your coverage speaks to the quality of thought going into it.

    4. Lastly, we know from comments we’ve received from many journalists they share our reviews with editors. And Publisher Gary Schwitzer routinely presents to audiences of journalists and editors at conferences and trainings. We agree with your observation that our reviews are “really not of the author, but of the piece, which is influence[d] by editors as well.” That’s one reason we never name the journalist in the review. We know there are many forces at play in production of the final piece. Many criticisms that we may make could reflect an editor’s cuts, an overly enthusiastic headline writer, etc. However, while we notify every journalist when we’ve posted a review, we do not routinely notify editors unless we can’t find a way to contact the writer. We don’t want to be seen as undermining journalists in front of their bosses, especially in the case of a low-scoring review. Someone who welcomes constructive criticism might not appreciate us sending that criticism to their boss. At least that’s how I think of it.

    Again, thanks for raising these issues and for engaging us. I welcome additional feedback.

    Kevin

    Reply

Larry Husten

June 12, 2015 at 2:34 pm

So I guess this is an anecdote (which should never be confused with data) about one individual’s attempt to assess and apply the data. So it’s a bit like the latest version of the Jane Brody school of health journalism, which is to write about one’s own attempts to incorporate the latest research. I was suspicious then and I’m suspicious now. How come we almost never read accounts of the writer’s failure to confirm the data? And this is related to the ubiquitous use of patient examples in new drug/intervention stories, which almost always contain a desperate person attaining a happy ending.

Reply