We simply don’t know why more news organizations can’t do an adequate job of explaining the limitations of observational studies – most notably, that they can’t prove cause and effect.
Yes, they can show strong associations. But they can’t prove cause and effect.
NBC Nightly News, as one example last night, inadequately explained the latest suggestion that coffee consumption can lower the risk of prostate cancer. In the anchor lead, Brian Williams framed this as another case of flip-flopping science, lightheartedly talking about what they say about “all those medical studies…if you don’t like the findings, wait for the next study.”
The story seemed puzzled at how the same “lab” 30 years ago reported that coffee was linked to an increase in pancreatic cancer. NBC said the researchers later said they got it wrong. This time – with the prostate cancer link – they say they got it right.
There was not one word in the story about the limitations of observational studies – and that it’s not a black-and-white issue of “wrong” or “right.” In fact, it’s often a matter of how researchers and journalists communicate and translate the findings.
To worsen the problem, NBC’s science correspondent asked about the new finding, “If not caffeine, what is the cause?” But that implies that a causal link – or the absence of a causal link – was established in this study.
All in all, it was not a good job. It ended with unwarranted and unsupported advice:
“At the very least men can enjoy a daily cup or more of coffee.”
[2017 Update: This video is no longer available but the transcript can be found HERE].
Meantime, a Health.com story on CNN.com easily and clearly reported:
“As with other questionnaire-based studies of coffee consumption and disease, the results do not prove that coffee directly prevents aggressive prostate cancer. The study shows only an association, although it is a relatively strong one.”
Shiuan Chen, Ph.D., director of tumor cell biology at the City of Hope Cancer Center in Duarte, California, says the evidence isn’t compelling enough for doctors to recommend that middle-aged men up their coffee intake. “I don’t think it’s any reason for changing habits in the immediate moment,” he says.
That’s markedly different than what NBC reported.
A definite cause-and-effect link is still far from proven, experts say, and just how coffee might help thwart prostate malignancy isn’t clear.
“It’s probably too early to tell someone that [he or she] should go out and start drinking coffee just because of this study.”
Any consumer or any journalist could learn from some of the tips we give in our primer on the difference between causation and association.
And, by the way, it is possible to do a better job on such studies even on TV. CBS’ Jennifer Ashton emphasized association – not causation – throughout this segment.