NOTE TO READERS: When this project lost substantial funding at the end of 2018, I lost the ability to continue publishing criteria-driven news story reviews and PR news release reviews - once the bread-and-butter of the site going back to 2006. The 3,200 archived reviews, while still educational, are getting old and difficult for me to technically maintain on the back end of the website. So I am announcing that I plan to remove these reviews from the site by April 1, 2021. The blog and the toolkit - two of the most popular features on the site - will remain. If you wish to peruse the reviews before they disappear, please do so by the end of March 2021. After that date you may still be able to access them via the Internet Archive Wayback Machine - https://archive.org/web/.
Read Original Story

How a New Treadmill Test Predicts ‘Survival of the Fittest’

Rating

2 Star

Categories

Tags

ABCNews.com

How a New Treadmill Test Predicts ‘Survival of the Fittest’

Our Review Summary

Encouraging people to exercise more (and eat a more healthful diet) makes a lot of sense, but this story oversimplified the implications of this study. It would have benefited from a fuller discussion of the risks of unnecessary screening tests, an explanation of the costs and the expertise required to correctly do this test, a true comparison of alternatives, and a better analysis of the quality of the evidence.

 

Why This Matters

This study proposes a new way to quantify health risks using information currently available from treadmill stress tests. The proposed “FIT treadmill score” was derived using previously collected data on 58,000 patients who underwent stress tests for a variety of reasons. The authors then assessed whether the patients died over an average of 10 years of follow-up. The key findings are that a person’s fitness — their maximum exertion level, and the ability to achieve maximum predicted heart rate — along with age and sex were the best predictors of future death. However, several important caveats aren’t clear from the two stories we reviewed about this study. First, the study had a number of weaknesses and limitations that we discuss below in the review; additional research will be needed to see if these results hold up with a more rigorous study design. It’s also not clear whether this test will actually lead individuals to make changes that will reduce their risk of premature death. It’s even conceivable that the test could cause harm — another issue that we tackle below in the review. Like many other seemingly “simple” tests, the FIT treadmill score gets a lot more complicated when you start asking tough questions — something we encourage all journalists to do when covering stories like this.

Criteria

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The study’s authors promoted this new way of analyzing treadmill results as a low-cost approach, saying in a press release, “The FIT Treadmill Score is easy to calculate and costs nothing beyond the cost of the treadmill test itself.” But this story should have explained how much it costs for a patient to make an appointment to get a treadmill test.

Another cost angle here is that many people may undergo the test simply to “know their score.” Is that a good idea? We would argue a vigorous “no.” For someone who exercises regularly without symptoms, such information really wouldn’t add much to what we already know. So cost matters in the sense that people may now ask for this test simply to get their score, which for many will be a waste of money.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story says that “the closer someone came to reaching their maximum heart rate as they ran, the more likely they were to die within ten years, compared to someone who only reached about 85 percent of their maximum heart rate during the test.” That’s the only quantification in the story, and it’s not accurate. In fact, the closer participants came to reaching their maximum predicted heart rate, the less likely they were to die in ten years. [This error appears to have been corrected in an updated version of the story.]  That error aside, the key here is how much more or less likely were study subjects to die with various levels of fitness?  How meaningful were the differences? Better quantification of benefits can be found in a competing story in the LA Times.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

There were many stories about this study, and we could not find one that mentioned harms. This is unfortunate, because screening tests do produce false positives and people do make choices based on those results. As a clinician in a New York Times piece about the relative weakness of treadmill tests as a predictor of health explained, “In my own practice I’ve seen people who thought they shouldn’t be exercising anymore because someone put them on a treadmill and got an abnormal test result when in fact there was nothing wrong with them.” The hypothesis here is that someone with a poor result will be encouraged to take action and improve their level of fitness. But what if that’s not the case? What if it increases their level of hopelessness and reinforces unhealthful behaviors? One may presume that information can never be harmful, but that isn’t always clear.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The study is complicated, and even the press release struggles to explain it in clear language. But the story did not do readers any favors in the way it explained how the study was conducted. It said merely that the study “looked at standard treadmill stress test results for more than 58,000 subjects, ages 18 to 96.” That could lead people to believe that 58,000 people were enrolled in a study and followed over time. But in fact, one of the biggest limitations of this study is that it was retrospective, meaning that researchers are looking back in time at health records to draw these conclusions, not measuring patients taking the treadmill test in real time and following them into the future. Such retrospective studies are more susceptible to bias. The test might not perform as well when measured prospectively, meaning getting the score first and then seeing what happens to the study subjects. The study does not mention this or any other possible limitation of this study

The story also promotes the idea that this test can “change your destiny.”  Nothing in this study suggests that acting on this treadmill information will or won’t change anything. That would require a whole different study to see how people respond to this information and what happens to them.

Does the story commit disease-mongering?

Satisfactory

This story did hype the treadmill test as a way to “change your destiny,” but it did not engage in disease mongering.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The only expert quoted by the story is Dr. Jennifer Ashton, an ABC News medical contributor. And we have a problem when TV networks use doctors simultaneously as contributors and as expert sources on their stories. It’s a conflict of interest when doctors are paid to be journalists talking about what they’re paid to do in their day jobs. We’ve written previously that the contributor role “should carry with it an emphasis on independent vetting of claims and evidence, rather than offering personal or professional opinion.” But that’s not what happens here. Ashton’s quote doesn’t really increase our knowledge or understanding of the study — she mainly exhorts readers to use the test as motivation to increase their fitness levels: “This is in your power to change and improve your numbers. You can actually change your own destiny,” she says. So despite the fact that this story quotes a doctor who’s not affiliated with the study, we don’t think it lives up to our standard for this criterion. We’re rating it Not Satisfactory.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

This story would have benefited from a full comparison of this new way of reading treadmill scores against existing tests for heart health. And the fact is, nobody needs a fancy exercise test to know that they’re out of shape, overweight, and in need of exercise. For some, the cost of the test if paid for out of pocket could be a gym membership for a year.

Does the story establish the availability of the treatment/test/product/procedure?

Not Satisfactory

Treadmill stress tests are pretty widely available, but readers might not know that they need to be referred to a doctor to obtain the test discussed here. A competing LA Times story pointed out that one cannot simply rely on the scores provided by a standard treadmill in a gym and would need to have an expert conduct a clinical treadmill test.

Does the story establish the true novelty of the approach?

Satisfactory

The press release for the study called the new formula for reading treadmill tests “novel,” and the story could have addressed that claim more thoroughly. Jennifer Ashton, an ABC News medical correspondent, alludes to novelty when she says, “What’s new here is that there is now a fancy equation doctors can use to compare the chances of survival for one 50-year-old woman against another 50-year-old woman.” That’s a good start, but is this way of making comparisons between people any different or better than previous efforts, like the Framingham Risk Score? We’ll give the story the benefit of the doubt here, but more analysis from outside experts would have helped establish what’s truly new about the study.

Does the story appear to rely solely or largely on a news release?

Satisfactory

The story did not rely on a press release.

Total Score: 3 of 10 Satisfactory

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.