Why you should be suspicious of subgroups: A new addition to our toolkit

Kevin Lomangino is the managing editor of HealthNewsReview.org. He tweets as @KLomangino.

Our toolkit is chock full of resources designed to help journalists and the public think smarter about health care.

And we just added a brand new primer: 3 reasons why you should be suspicious of study ‘subgroup’ results.

Why do we need to know about subgroups?

Because they’re often touted in news stories and PR releases as evidence that a treatment is effective. But experts warn that these analyses — which involve looking at a small group within a larger study — are subject to serious, sometimes fatal limitations.

And media messages about subgroups rarely explain why these results may be misleading.

Excerpts from the primer:

  • “Because they’re smaller than the overall study, subgroups are more likely to harbor subtle imbalances that can skew the findings.”
  • “Subgroups have a lousy track record of identifying legitimate health benefits. In a recent analysis of clinical trials that included a claim from a subgroup, researchers found that only 5 out of the 46 results they looked at were ever tested in a follow-up study. None of the 46 subgroup claims held up in the subsequent research.”
  • “Another problem with subgroups is the higher likelihood of false-positive findings. Simply put, the more subgroups you look at, the greater the risk that you’ll find a positive result that’s attributable to random chance rather than the intervention.”

Researchers who study this issue say, “Subgroup analyses have historically misinformed as much as they have informed.”

I hope you’ll read the entire primer and find out why.

You might also like

Comments

We Welcome Comments. But please note: We will delete comments left by anyone who doesn’t leave an actual first and last name and an actual email address.

We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, or profanity. We will also end any thread of repetitive comments. Comments should primarily discuss the quality (or lack thereof) in journalism or other media messages about health and medicine. This is not intended to be a forum for definitive discussions about medicine or science. Nor is it a forum to share your personal story about a disease or treatment -- your comment must relate to media messages about health care. If your comment doesn't adhere to these policies, we won't post it. Questions? Please see more on our comments policy.

Comments are closed.