CT Scans Deemed Best for Checking Heart Arteries

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The significant costs of both MRI and CT angiography are not discussed. Their adoption as screening tests could have a substantial impact on overall health care costs, and the omission of a discussion of costs is a major oversight. See this New York Times article for why the costs of CT angiography are relevant.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story does provide some important details on the meta-analysis, including the number of studies and patients, as well as sensitivity results. However, we have two critiques:

1) It notes that the sensitivity results, 97% and 87%, were for the studies of people with suspected coronary artery disease. We believe that’s incorrect. According to the published study, these figures are for the overall population in the meta-analysis, which includes those with suspected OR known disease, including those with heart attacks. For the analysis just of patients with suspected coronary artery disease, the results were different: 98% and 89%. But that’s a minor problem.  A bigger issue,  to us, is the following.
 
2) We would have liked to have seen the specificity results, too. We’re unsure why they choose to report just sensitivity and not specificity. False positives are important, too, besides just false negatives.

3) The study had clear limitations that were ignored in the story.  The authors of the study concluded, "Randomized studies are clearly needed to address the potential of coronary CT angiography for use in triage as a means of positively altering management and outcomes in patients with suspected CAD (coronary artery disease)."  The take home message for readers of this story may be that CT scans are "Best for Checking Heart Arteries"  – as the headline states – a conclusion that has yet to be clearly demonstrated.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

Although the story highlights the presumed advantages of CT scans, there isn’t a single word about potential harms. That includes the radiation exposure leading to risks for cancer, which are acknowledges in the introduction to the published study. (According to the most recent estimates, the risks may be in the neighborhood of one case of cancer for every 800 individuals scanned.) The story also leaves out the specificity results of this research, ignoring the rates of false positives and their potential harms.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story is clear about the type of study, that it was a meta-analysis or "review of studies." It also presents sensitivity rates and quotes outside sources noting that the conclusions were already known in general. 

Now for the bad news. First, there is the misleading context, or lack of context, for this evidence, as we’ve described. We feel that the story, starting with its headline, is worded in such a general way that many readers will think the evidence supports the use of noninvasive over invasive imaging. Second, the story fails to report on specificity results, the percentage of healthy people who are correctly identified by the tests as not having the condition. See the Benefits criterion.

The story also does not note any of the limitations of the evidence acknowledged in the published study. These include:

• The review included very few studies of MRI (only 20 of the 109 studies)
• It included very few studies (only 5 of 109) that directly compared CT and MRI.
• There were some other methodological limitations.

We also see some other potential limitations to the study conclusions. Most (19/20) of the MRI studies in the analysis used older, less powerful technology, while a small minority (3/89) of the CT studies used older studies; therefore, the CT scans had a technological leg up in this analysis. We also question the similarity of patients across the studies used in this meta-analysis. Finally, the analysis acknowledges that the studies were of poor to moderate quality, and the MRI studies were of poorer quality than the CT studies.

Another part of evaluating the evidence is discussing what the review doesn’t show, what we still don’t know, and what future research is needed. The final sentence in the published study sums it up: "Randomized studies are clearly needed to address the potential of coronary CT angiography for use in triage as a means of positively altering management and outcomes in patients with suspected CAD [coronary artery disease]."

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The story does cite two independent sources who appeared unrelated to the study.

The story did not identify any potential conflicts of interest. While the study publication identifies no primary funding sources, the senior author, Marc Dewey, reports in his conflict of interest statement financial relationships with Toshiba Medical Devices, Bayer-Scherling, and Guerbet, companies that manufacture imaging devices or contrast media used with them. These relationships were disclosed by the medical journal when they published the study – yet the story makes no menton of them.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

The story notes that CT and MRI are each used for different reasons, but it implies erroneously that either is an all-around alternative to catheterization, and thus the "best" way to detect coronary artery disease.