Read Original Story

Strong report questions effectiveness of breast cancer screening — but needed a discussion of study limitations

Here's new evidence that widespread breast cancer screening isn't effective

Our Review Summary

breast cancer diagnosis graphThis story reports on a new study looking at breast cancer screening rates across hundreds of counties in the United States. It found that while higher breast cancer screening rates were associated with increased rates of breast cancer detection, more screening was not associated with reduced breast cancer mortality. The explanation for this finding, according to both the study and the story, is that higher levels of screening lead to overdiagnosis of breast cancer — that is, finding small. early-stage breast cancers that probably would never need treatment.

The story overall is solid and notably features some very helpful visuals — including a figure from the study that demonstrates the main study outcomes (click on the image for a closer look), as well as a risk communication tool that outlines the benefits and harms of mammography. But the story fell short when it came to discussing the many limitations of this study design and their possible impact on the results. Those limitations were discussed in the study itself, or perhaps would have come to light through inclusion of an additional expert perspective.


Why This Matters

A growing body of evidence suggests that screening mammography is not as beneficial as once thought. But information questioning the benefits of screening mammography usually receives widespread criticism from screening advocates, which adds to confusion for many women on whether mammography is beneficial and necessary. Stories that clearly explain the findings of new research, and place the results in context with other research and expert guidelines, are therefore valuable to women who are contemplating whether to be screened and at what age.


Does the story adequately discuss the costs of the intervention?

Not Satisfactory

There was no discussion of costs in this story or of insurance coverage for mammography. While many readers may understand that screening mammography is generally covered by insurance, it would have been helpful to state explicitly that mammograms in the U.S. are covered by the Affordable Care Act. More broadly, given that this study questions the effectiveness of mass breast screening, it’s worth at least asking what such screening is costing the health care system.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?


The story satisfactorily reports the main results of the study and describes the benefits of mammography as seen across a large body of research. It explains that in the study, “The number of diagnosed breast cancers rose with more screening, yet the number of deaths from breast cancer 10 years later remained stubbornly stable.” It doesn’t quantify the increase in breast cancer detection in the text (the study reported a 16% increase in breast cancer diagnoses for every 10 percentage point increase in screening), but it does include a figure from the study which clearly demonstrates that while more cancers were found, mortality from breast cancer remained stable.

The story also includes a risk communication illustration, based on a systematic review of available evidence, that presents the absolute mortality benefits (as well as harms) for women over 50 who participate in screening for at least 10 years.

The story mentions that screening may be more effective in women with a family history of breast cancer. It does not discuss whether the study found any significant results among demographic subgroups for whom screening mammography may be more effective, but it doesn’t appear that the authors were capable of making such distinctions based on the data available (see Evidence section below).

Does the story adequately explain/quantify the harms of the intervention?


The story is mainly about harms and the fact that increased breast cancer screening was not associated with lower mortality rates in the study — suggesting that there is a high rate of overdiagnosis of breast cancer. “The trouble with overdiagnosis is that while the cancers doctors find wouldn’t have harmed their patients, the treatment and stress that result from the diagnosis probably will,” the story says. The story adds that estimates of the scope of the problem are “frustratingly broad — from less than 10 percent to 50 percent or more of women who get screened. In other words, we don’t actually know how bad the problem is.”

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story does not discuss the limitations to the study design, which were substantial and were explained in some detail in the paper that’s the basis for the story. The research study used ecological observations of large groups (not individuals) that are subject to inherent biases. In addition, the study authors explained that 10 years (the length of follow up in the study) might not be long enough to demonstrate mortality differences, especially for early stage breast cancer, and that detailed information such as tumor sub-type and information regarding treatment with adjuvant therapy are not available for review and consideration. There’s potential for confounding as well, as some of the factors associated with an increased risk of developing breast cancer such as income and (late) age at first birth are also factors associated with a higher likelihood of undergoing screening mammography. Lastly, the study authors noted that there is no way to tell if the women who died of breast cancer during the 10 years were actually the ones diagnosed during that time period.

Does the story commit disease-mongering?


The article did not engage in disease mongering.

Does the story use independent sources and identify conflicts of interest?


The story cites independent commentary from an accompanying editorial that provides useful context. Inclusion of a second independent voice might have drawn attention to some of the study’s limitations as discussed above. The story doesn’t discuss potential conflicts of interest among the study authors, but the only conflict disclosed with the manuscript was related to a company that makes thermometers. Overall, this seems Satisfactory to us.

Does the story compare the new approach with existing alternatives?


The alternative to mammography is not getting screened. The story suggests this is an option, especially for women who are under the age of 50. “Women with no risk factors for breast cancer should also know that they can wait until age 50 for their first screening,” the story says.

Does the story establish the availability of the treatment/test/product/procedure?


It’s obvious from the story that mammography is widely available.

Does the story establish the true novelty of the approach?


The story notes that “this isn’t the first study to call into question the benefits of mammography and whether women are too often the victims of overdiagnosis — that is, being diagnosed with cancers that would not have been fatal or even harmful.”

Does the story appear to rely solely or largely on a news release?


The story brings in context from an accompanying editorial, risk communication tools, independent guidelines, and other studies of overdiagnosis. It clearly went beyond any news release.

Total Score: 8 of 10 Satisfactory

Comments (1)

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Dan Mayer

August 6, 2015 at 9:50 pm

I’m not sure what this means, but when I look at those graphs, I see the number of outlier values getting less as the incidence of mammography increases. Anyone have any explanation for this? It suggests that in some situations there is lower cancer mortality while in some there is more (although the outliers on the high side are farther from the line than the low ones).