Why you should beware of those aiming to “separate fact from fiction” in nutrition research

The following post is by Alan Cassels, a pharmaceutical policy researcher at the University of Victoria, British Columbia, a journalist, and author of the The Cochrane Collaboration: Medicine’s Best-Kept Secret.

Recent news reports suggest this breakfast could send you to an early grave. This post will help you determine the validity of such claims and whether the supporting research is reliable.

One thing that is almost fundamental to living in the modern age is having someone tell you what to eat.

Just this past weekend NPR informed us, “Eating More — Or Less — Of 10 Foods May Cut Risk Of Early Death.”

Yet the myths that surround nutrition, food, vitamins and so on are legion, as are the recommendations from all kinds of bodies, panels, experts and gurus frequently offering suggestions on what we should eat, how much of it we should eat and why.  Perhaps no one has rendered food advice into a tidier bite-sized portion than Michael Pollan whose “Eat food. Not too much. Mostly plants” stands as a manifesto of sensible nutritional recommendations.

However, the media-consuming public still has a hearty appetite for whatever is new in nutritional advice. And as NPR’s enthusiastic coverage clearly indicates, journalists have an incentive to serve up new servings of food research as soon as it comes out of the oven — even if it hasn’t been fully baked. 

Report debunking “hype” may inadvertently promote it

Another recent example is a report issued by the American College of Cardiology (ACC). It’s a hefty platter of research aiming to review “some of the more popular food trends and dietary patterns,” revisiting eggs, oils, nuts, and a range of other popular diet approaches. The motivation behind the cardiologists’ report seems to be a perceived problem with the media, specifically that several controversial dietary patterns “are mired by hype.” The goal of the report is to “provide clinicians with accurate information for patient discussions in the clinical setting.”  

This makes sense: if we are consulting a cardiologist for ways to maintain and improve our heart health, those cardiologists should be singing from the same science-based song sheet.  

One the whole, I don’t think the ACC report authors did a bad job, and they included a nice simple infographic that neatly captures the essence of the report. But it’s hard not to feel they’ve included some questionable advice. Hoping to “separate fact from fiction,” as the Washington Post puts it, the report repeatedly muddied the line between the two, such as the need to limit eggs (owing to their cholesterol content) and repeating the canard about berries having antioxidant benefits (which contributes to their “superfood” factor, according to the report).

Eggs contain cholesterol but also have lots of other components that might have all sorts of effects in the body, both good and bad. And while blood cholesterol itself is linked to heart disease, there’s mixed evidence on whether eating eggs (which again, contain much more than cholesterol) has any impact at all on your risk of having a heart attack or stroke.

There’s similarly nothing wrong with berries and they seem to be quite a healthful choice. But contrary to the ACC report’s suggestion, it’s not clear that neutralizing all those free radicals floating around in your body is always a good thing. Selling berries based on this “superfood” myth is going to confuse and mislead a lot of people.

We have to remember that much of nutrition research comes from observational studies or studies based on surrogate endpoints. In other words, these aren’t conclusive. So any recommendations need to be couched in proper caveats about association and causation, and the limits of surrogates to tell us about the outcomes (like heart attacks, strokes, and death) we really care about.

Flip-flopping science is often preliminary science

Besides those unsavory bits, the ACC’s analysis of food trends is better than most despite the rather unspicy recommendations. The news release tried to whet our appetites by saying the researchers aimed “to debunk myths on antioxidant pills, juicing and other dietary fads.” Yet other than giving the nod to a diet favoring fruits and vegetables (which is in the same category of motherhood recommendations pertaining to clean underwear) there was little new here. This was reflected in sparse coverage of the ACC report beyond a few articles, including this somewhat congratulatory piece in the Olive Oil Times.

The Post’s coverage may have nailed the essence of this report: “Nutrition can be confusing. Fad diets make headline news, and we’re bombarded with flip-flopping reports about what to eat.” To avoid contributing to the noise around nutrition, here are five questions journalists might want to consider when reporting on the next nutritional ‘superfood,’ diet promise, or health food trend:

1.Do the recommendations come with quality research — preferably from multiple studies — underpinning them?  

When the American College of Cardiology issues a report, are they basing that report on the interpretation of meta-analyses of controlled trials, or is it from just a bunch of researchers pooling their ideas and prejudices around what they think might be healthy for the heart? Thankfully the ACC report did more than skim the available research. It included a generous dive into many of the challenges that arise from studying dietary patterns, commenting on research quality, the limitations of observational research, and the potential for biases. (One wishes those limitations were reflected better in some of the report’s overzealous recommendations and claims, such as that “fruits and vegetables protect against heart disease, many forms of cancer, and other chronic diseases, and may also help slow the aging process.”)

Popular Science also reminded us of how crazy it can be navigating “the landscape of nutritional research.” They said: “a single study doesn’t mean anything. Scientists need to reproduce the same results over and over again, in different circumstances and settings, to determine how likely something is to hold true.” The author’s sensible conclusions? “Stop worrying about new research praising the health benefits of wine or demonizing your favorite wheat product. Instead, stick to the things you know are healthy—and enjoy the rest in moderation.”

2.  Does the research laser in on one particular nutrient or one particular body part?

Focusing on single nutrients (such as Vitamin D or cholesterol) or single body parts (the heart, for example) should set the red flags a-waving. In our incessant search for magic bullets, we often lose sight of the fact our bodies are one big ecosystem that has numerous inputs.  Any journalist should be aware that focusing on the specific health claims for single, isolated types of foods needs many caveats. For example, a reader worried about their brain health might end up running to the store to stock up on the “Ten foods to avoid Alzheimer’s” (as reported in CBS, on the MIND diet) thinking they’ll be reducing their risk of developing Alzheimer’s by 53 percent. But it’s laughable to suggest, as the CBS story does, that eating cheese “no more than once a week” has any appreciable impact on one’s risk of developing Alzheimer’s.

Claims to “separate fact from fiction” often narrow in on the attributes of a particular food, such as the proteins or vitamins or fiber contained in a single apple, as if those individual substances can be extracted, refined, and turned into the magic ingredient. What about just telling people to eat the apple? As the Washington Post article says, “we could spend forever sifting through the minutiae of every single nutrient. Or we could be more practical and focus on the big picture, by looking at the whole diet instead of each individual food.”  

3.How well does the journalist tiptoe through the conflict of interest minefield? 

If you’re a book author, health guru, or food company, there is a lot of money to be made in getting people to buy into your particular diet paradigm. Conflicts–both financial and intellectual–around food might be harder to tease out than in other kinds of research. While conflicted experts, paid by particular food or vitamin manufacturers, might be easy to spot, the reason we see a lot of headline hype around food is that the conflicts of interest might not be so obvious. We have to be wary of some commentator’s bias when they’re trying to give context to new research. Do they have a history of commenting with a “just the facts” approach or do they lean a certain direction because their research or research institution is supported by selling those supplements? Or, maybe they’re speaking from the perspective of a deeply held, personal stake in being a vegetarian, vegan, omnivore or anti-gluten activist and thus assert their own set of biases and perspectives.  

As for the cardiology report, the conflict of interest disclosures were lengthy and the researchers had dozens of ties to groups connected to food (Ever heard of the California Walnut Commission? They’re eager to tell you how walnuts can supposedly improve your sperm health), nutritional supplement companies, a juicing company and various drug companies. Did this bias their perspective?  Popular Science (focusing in on the advice around juicing) doesn’t think so. “Lest you think the researchers just have it in for kale juice, the study’s disclosure of conflicts of interest actually reveal that one of the authors serves as a scientific advisor for Pressed Juicery,” the story says. “Dr. Miller is clearly not shilling for Big Juice. Dr. Miller is gonna tell it like it is.” It’s nice to see that despite a conflict of interest with Big Juice, the report gives the big ‘thumbs down’ to the health fad that suggests you should just juice everything in sight. 

4. How does the new bit of research (or advice) compare to existing guidelines or recommendations?   

Even as a journalist may be tempted to compare a new food claim or bit of nutritional research to assertions made in existing guidelines, they should question whether those comparisons are valid. Is it possible the yardstick being used is not evidence-based or somewhat outdated? Cardiologist Salem Yusuf from McMaster University said recently that many major guidelines on nutrition — including advice to restrict saturated fat intake — are not based on good evidence. It’s hard to dispute that assessment, but Yusuf himself was serving up some observations, such as that increasing fat intake is “protective,” before they were ready to be plated. He was commenting on the results of the ten-year-long PURE study (which stands for the Prospective Urban Rural Epidemiology) which recruited 140,000 people in 17 countries looking at the impact of ‘societal influences’ such as diet, socioeconomic factors, and the ‘tobacco environment.” Even if it’s a very large study, though, we need reminding that an observational study such as PURE is … well … observational, so any suggestion that it proves certain foods are ”protective” is simply overstating the evidence.

5.  What other factors could influence the research?

Studying food, vitamins, supplements and other dietary stuff is a very tricky business. Researchers talking about their own study should be openly discussing confounding factors and pose the question, “What other thing may have influenced the results we observed?” For example, in the ACC report, the authors helpfully advise us to not ignore the healthy user effect. Which is to say, when we are talking about health claims made around nutrition, the research often involves healthy people. Healthy people probably do all kinds of healthy things, not just eat. So when reporting on healthy people, we have to remember that whatever ‘healthy’ effect you’re measuring could be due to something other than the food or nutrient you’re focusing on.  

Even though people are bombarded from all directions with nutrition and dietary advice, many of us take a certain pleasure in hearing what is or isn’t good for us this week. We love to consume this type of news even though a large majority of us probably ignore it. And with good reason, because as the ACC report makes clear, most of the provocative advice that looks great on paper tends to be unspectacular in the real world.

And maybe that’s what journalists need to take home at the end of the day: despite the fun of reporting the results of some new super food, or reading about a food that will help you live to be 100, we shouldn’t take most of this that seriously. In fact, maybe we, too, should ignore it.

You might also like

Comments

We Welcome Comments. But please note: We will delete comments left by anyone who doesn’t leave an actual first and last name and an actual email address.

We will delete comments that include personal attacks, unfounded allegations, unverified facts, product pitches, or profanity. We will also end any thread of repetitive comments. Comments should primarily discuss the quality (or lack thereof) in journalism or other media messages about health and medicine. This is not intended to be a forum for definitive discussions about medicine or science. Nor is it a forum to share your personal story about a disease or treatment -- your comment must relate to media messages about health care. If your comment doesn't adhere to these policies, we won't post it. Questions? Please see more on our comments policy.