Encouraging people to exercise more (and eat a more healthful diet) makes a lot of sense, but this story oversimplified the implications of this study. It would have benefited from a fuller discussion of the risks of unnecessary screening tests, an explanation of the costs and the expertise required to correctly do this test, a true comparison of alternatives, and a better analysis of the quality of the evidence.
This study proposes a new way to quantify health risks using information currently available from treadmill stress tests. The proposed “FIT treadmill score” was derived using previously collected data on 58,000 patients who underwent stress tests for a variety of reasons. The authors then assessed whether the patients died over an average of 10 years of follow-up. The key findings are that a person’s fitness — their maximum exertion level, and the ability to achieve maximum predicted heart rate — along with age and sex were the best predictors of future death. However, several important caveats aren’t clear from the two stories we reviewed about this study. First, the study had a number of weaknesses and limitations that we discuss below in the review; additional research will be needed to see if these results hold up with a more rigorous study design. It’s also not clear whether this test will actually lead individuals to make changes that will reduce their risk of premature death. It’s even conceivable that the test could cause harm — another issue that we tackle below in the review. Like many other seemingly “simple” tests, the FIT treadmill score gets a lot more complicated when you start asking tough questions — something we encourage all journalists to do when covering stories like this.
The study’s authors promoted this new way of analyzing treadmill results as a low-cost approach, saying in a press release, “The FIT Treadmill Score is easy to calculate and costs nothing beyond the cost of the treadmill test itself.” But this story should have explained how much it costs for a patient to make an appointment to get a treadmill test.
Another cost angle here is that many people may undergo the test simply to “know their score.” Is that a good idea? We would argue a vigorous “no.” For someone who exercises regularly without symptoms, such information really wouldn’t add much to what we already know. So cost matters in the sense that people may now ask for this test simply to get their score, which for many will be a waste of money.
The story says that “the closer someone came to reaching their maximum heart rate as they ran, the more likely they were to die within ten years, compared to someone who only reached about 85 percent of their maximum heart rate during the test.” That’s the only quantification in the story, and it’s not accurate. In fact, the closer participants came to reaching their maximum predicted heart rate, the less likely they were to die in ten years. [This error appears to have been corrected in an updated version of the story.] That error aside, the key here is how much more or less likely were study subjects to die with various levels of fitness? How meaningful were the differences? Better quantification of benefits can be found in a competing story in the LA Times.
There were many stories about this study, and we could not find one that mentioned harms. This is unfortunate, because screening tests do produce false positives and people do make choices based on those results. As a clinician in a New York Times piece about the relative weakness of treadmill tests as a predictor of health explained, “In my own practice I’ve seen people who thought they shouldn’t be exercising anymore because someone put them on a treadmill and got an abnormal test result when in fact there was nothing wrong with them.” The hypothesis here is that someone with a poor result will be encouraged to take action and improve their level of fitness. But what if that’s not the case? What if it increases their level of hopelessness and reinforces unhealthful behaviors? One may presume that information can never be harmful, but that isn’t always clear.
The study is complicated, and even the press release struggles to explain it in clear language. But the story did not do readers any favors in the way it explained how the study was conducted. It said merely that the study “looked at standard treadmill stress test results for more than 58,000 subjects, ages 18 to 96.” That could lead people to believe that 58,000 people were enrolled in a study and followed over time. But in fact, one of the biggest limitations of this study is that it was retrospective, meaning that researchers are looking back in time at health records to draw these conclusions, not measuring patients taking the treadmill test in real time and following them into the future. Such retrospective studies are more susceptible to bias. The test might not perform as well when measured prospectively, meaning getting the score first and then seeing what happens to the study subjects. The study does not mention this or any other possible limitation of this study
The story also promotes the idea that this test can “change your destiny.” Nothing in this study suggests that acting on this treadmill information will or won’t change anything. That would require a whole different study to see how people respond to this information and what happens to them.
This story did hype the treadmill test as a way to “change your destiny,” but it did not engage in disease mongering.
The only expert quoted by the story is Dr. Jennifer Ashton, an ABC News medical contributor. And we have a problem when TV networks use doctors simultaneously as contributors and as expert sources on their stories. It’s a conflict of interest when doctors are paid to be journalists talking about what they’re paid to do in their day jobs. We’ve written previously that the contributor role “should carry with it an emphasis on independent vetting of claims and evidence, rather than offering personal or professional opinion.” But that’s not what happens here. Ashton’s quote doesn’t really increase our knowledge or understanding of the study — she mainly exhorts readers to use the test as motivation to increase their fitness levels: “This is in your power to change and improve your numbers. You can actually change your own destiny,” she says. So despite the fact that this story quotes a doctor who’s not affiliated with the study, we don’t think it lives up to our standard for this criterion. We’re rating it Not Satisfactory.
This story would have benefited from a full comparison of this new way of reading treadmill scores against existing tests for heart health. And the fact is, nobody needs a fancy exercise test to know that they’re out of shape, overweight, and in need of exercise. For some, the cost of the test if paid for out of pocket could be a gym membership for a year.
Treadmill stress tests are pretty widely available, but readers might not know that they need to be referred to a doctor to obtain the test discussed here. A competing LA Times story pointed out that one cannot simply rely on the scores provided by a standard treadmill in a gym and would need to have an expert conduct a clinical treadmill test.
The press release for the study called the new formula for reading treadmill tests “novel,” and the story could have addressed that claim more thoroughly. Jennifer Ashton, an ABC News medical correspondent, alludes to novelty when she says, “What’s new here is that there is now a fancy equation doctors can use to compare the chances of survival for one 50-year-old woman against another 50-year-old woman.” That’s a good start, but is this way of making comparisons between people any different or better than previous efforts, like the Framingham Risk Score? We’ll give the story the benefit of the doubt here, but more analysis from outside experts would have helped establish what’s truly new about the study.
The story did not rely on a press release.