Health News Review

We simply don’t know why more news organizations can’t do an adequate job of explaining the limitations of observational studies – most notably, that they can’t prove cause and effect.

Yes, they can show strong associations. But they can’t prove cause and effect.

NBC Nightly News, as one example last night, inadequately explained the latest suggestion that coffee consumption can lower the risk of prostate cancer. In the anchor lead, Brian Williams framed this as another case of flip-flopping science, lightheartedly talking about what they say about “all those medical studies…if you don’t like the findings, wait for the next study.”

The story seemed puzzled at how the same “lab” 30 years ago reported that coffee was linked to an increase in pancreatic cancer. NBC said the researchers later said they got it wrong. This time – with the prostate cancer link – they say they got it right.

There was not one word in the story about the limitations of observational studies – and that it’s not a black-and-white issue of “wrong” or “right.” In fact, it’s often a matter of how researchers and journalists communicate and translate the findings.

To worsen the problem, NBC’s science correspondent asked about the new finding, “If not caffeine, what is the cause?” But that implies that a causal link – or the absence of a causal link – was established in this study.

All in all, it was not a good job. It ended with unwarranted and unsupported advice:

“At the very least men can enjoy a daily cup or more of coffee.”

Visit msnbc.com for breaking news, world news, and news about the economy

Meantime, a Health.com story on CNN.com easily and clearly reported:

“As with other questionnaire-based studies of coffee consumption and disease, the results do not prove that coffee directly prevents aggressive prostate cancer. The study shows only an association, although it is a relatively strong one.”

Shiuan Chen, Ph.D., director of tumor cell biology at the City of Hope Cancer Center in Duarte, California, says the evidence isn’t compelling enough for doctors to recommend that middle-aged men up their coffee intake. “I don’t think it’s any reason for changing habits in the immediate moment,” he says.

That’s markedly different than what NBC reported.

HealthDay did a good job in explaining:

A definite cause-and-effect link is still far from proven, experts say, and just how coffee might help thwart prostate malignancy isn’t clear.

“It’s probably too early to tell someone that [he or she] should go out and start drinking coffee just because of this study.”

Any consumer or any journalist could learn from some of the tips we give in our primer on the difference between causation and association.

And, by the way, it is possible to do a better job on such studies even on TV. CBS’ Jennifer Ashton emphasized association – not causation – throughout this segment.

Comments

Andrew Holtz posted on May 18, 2011 at 11:51 am

Even if there is some sort of causal link here (and not just a mutual association with some third, unmeasured factor), association doesn’t tell you which way the cause>effect arrow points. Based just on this evidence it is just as reasonable to flip the arrow the reverse direction. Guess how many stories led with, “Study suggests that having a lower risk of aggressive prostate cancer may boost coffee drinking”. That headline seems absurd, but it has just as much support (from this particular study) as suggestions that coffee intake lowers cancer risk.
Of course, there are other reasons to suspect that it is more likely that diet influences cancer risk than the other way around, but interpreting (and reporting on) scientific evidence requires discipline… it requires reining in our beliefs and hopes. That standard includes being strict about not seeing a cause>effect arrow when all the evidence really shows is that two things were found near each other.

Ken Leebow posted on May 18, 2011 at 12:28 pm

Thanks for the review. BTW, ABC was similar to NBC’s analysis.
A week before this study came out, I put on my website “Life without Lipitor” this brief blurb for my readers:
I start my day with about three cups of coffee. I always look forward to my morning java. I won’t mention coffee consumption every day. Just assume, I drink three cups a day. Yes, I am addicted. And no, they have not identified negative health impacts from coffee consumption. And yes, I’m sure someone will read this and refer to a study that coffee has some negative health impacts.
I am not a fan of the “study/research of the day”. In a Tweet, I just read this – Coffee May Protect Against Aggressive Breast Cancer: http://bit.ly/lLWYNn – Take note: First it says “may”. Well, one day, I may be as rich as Bill Gates. While the latter statement is true, um, I do not believe it will ever happen. Second, who did this study, for how long, how large was the group being studied, and was it an observational or clinical study? I don’t know the answer, because, candidly, I will not waste my valuable time researching “the study of the day”. My recommendation: Ignore the study of the day.

keith cass posted on May 19, 2011 at 2:05 am

Drammatic headlines are no use to the men who have PCa (prostate cancer) the headlines just add to the confusion that already exists. If we put drinking coffee claim alongside those of Abiraterone and Provenge both of which are truly ground breaking treatments then coffee drinking pales into insignifcance. Men like myself with stage 4 aggressive PCa want well researched,factual phase III trial results.
The only benefit to the man/woman in the street is that it brings PCa to the attention of some people that otherwise may never had thought about this massive killer of men. What we need is more joined up thinking from research establishment to help find better biomarkers for PCa and then just maybe we wouldnt have to spend so much of our health service money on treating men such as myself with late stage PCa and we might actually save lives in the process.
20 years on with the PSA test and here in the UK it is still locked away in the GPs filing cabinets waiting for a better test!!!

Abraham Reback posted on May 23, 2011 at 2:11 pm

What about the observations and trials that are funded by those who have a vested interest in the outcome. The scientists know that if the results are not satisfactory they can kiss future funding goodbye.