Belief in health bullshit is a global problem; a big study points to solutions

Matt Shipman is a public information officer at North Carolina State University and a frequent story and news release reviewer. He tweets as @shiplives.

Children pose with a prototype of the children’s book used in the study, in Rwanda. Credit Matt Oxman/Informed Health Choices.

You’ve probably gotten bad medical advice at some point, and odds are good that it started when you were a kid.

For example, it was conventional wisdom where I grew up that you should put butter on a burn. But despite the fact that “everyone knew this,” it’s not only wrong, but dangerously wrong – it can make the burn worse.

Similar advice is given in East Africa about the healing power of cow dung. It’s literally a bullshit claim that can lead to infection of the burned area.

This sort of misinformation is common. Sometimes it stems from well-intentioned but misinformed people, but it can also stem from false advertising, news releases that exaggerate research findings (intentionally or otherwise), or from news stories that distort health studies – either through misunderstanding the work or in an attempt to snag a larger audience.

Because this misinformation is so widespread, and because it can affect the decisions we make about our health – with consequences for both our well-being and our wallets – people need to be able to think critically about health claims. This mission is what is all about.

And now a recent study, published May 21 in The Lancet, indicates that teaching this sort of critical health literacy to children as early as grade school may make a difference in how they evaluate health claims.

Randomized test involving 120 schools

A group of children show off the first prototype of the children’s book, in Uganda.

An international team of researchers affiliated with the Informed Health Choices project developed a suite of tools focused on improving critical health literacy in children, and wanted to test the efficacy of the tools in areas that have limited resources and where the intervention may do the most good. Given that the team included a researcher from Makerere University in Uganda, the researchers decided to start there.

[Editor’s Note: Matt Oxman, one of the study co-authors, wrote a guest article for in October 2016 about the impetus for this study titled, “Immunizing children against deadly bullshit.” ]

The researchers recruited 120 schools in central Uganda and split them into two groups. The intervention group was provided with a teacher’s guide for the critical health literacy intervention, exercise books, a textbook in the format of a graphic novel or comic book, and other instructional materials. (Note: you can see related materials here.) Teachers from the 60 schools in the intervention group were also asked to attend a two-day training workshop on the intervention. The 60 schools in the control group would teach their normal curriculum, without incorporating the health literacy intervention. Students in both groups were between the ages of 10 and 12.

The intervention focused on 12 key concepts that are fundamental to evaluating health claims. For example, treatments may be harmful, and individual anecdotes aren’t a reliable way to assess a treatment. In short, these are many of the basic concepts behind clinical trials.

An illustration from the children’s book used in the study.

The intervention was taught in a weekly, 80-minute class over the course of nine weeks. At the end of the intervention, students in both the intervention and control groups were given a multiple choice test to determine how well the students understood and could apply the 12 key concepts.

The study was funded by the Research Council of Norway, and the Informed Health Choices project is funded by a mix of government and nonprofit organizations, including the Research Council of Norway, the United Kingdom’s National Institute for Health Research and the Rockefeller Foundation.

Intervention schools do better at detecting bullshit

Of the 5,753 students in the intervention group who took the test, 69 percent of them got at least 13 of 24 questions correct – a passing score. That compares to only 27 percent of the 4,430 students in the control group. Overall, the mean score for schools in the intervention group was 62.4 percent, compared to 43.1 percent for the control group.

In addition, 19 percent of students in the intervention schools got at least 20 questions correct, indicating a very good understanding of the subject matter. Only 1 percent of students in the control group fared as well.

In The Lancet, study authors wrote “We believe we have shown reliably that it is possible to teach critical appraisal of treatment claims on a large scale in a low-income country.”

But how broadly applicable are these findings?

Could this work elsewhere?

The study authors themselves stated that “It is uncertain…how transferable the findings of this study are to other regions and countries.” But they’re hopeful.

Dr. Daniel Semakula (study co-author) talks with children using a prototype of the children’s book, in Uganda. Credit: Matt Oxman/Informed Health Choices.

“I am fairly certain that there is a need for something like what we tested around the world,” Dr. Andy Oxman, an author of the study and Research Director of the Global Health Unit in the Norwegian Institute of Public Health, told “There is quite a bit of evidence documenting that low health literacy is a problem globally, that people have problems assessing health claims, and that poorly informed health choices are common. I am optimistic that the resources we developed and tested can be adapted and used in other countries.”

And other experts agree.

“I think this is a pedagogical intervention that could be applicable in other parts of the world,” said Nancy Berkman, a health literacy researcher who is not associated with the Informed Health Choices project. “Health literacy advocates might take away from this intervention that skills in informed health care decision-making and basic epidemiology can be successfully taught to elementary school children,” said Berkman, who is a fellow at RTI International.

“This is the world’s first study in children that clearly shows health literacy can change through an organized intervention,” said Richard Osborne, a health literacy researcher who is unaffiliated with Informed Health Choices and Chair of Public Health at Deakin University in Australia. “The takeaway is that health literacy interventions can be created, can be applied, and can be evaluated.”

Authors of a related commentary in The Lancet expressed a similar view.

In an interview with, Dr. John Santelli – one of the commentary co-authors and a researcher at Columbia University – noted that “cautious scientists often say that their findings are not directly applicable to other situations.

“However,” Santelli added, “health literacy among adolescents, and often adults as well, is an issue around the globe. Each new cohort of young people need education to support health literacy. Moreover, the research design for the Lancet study was very good. It provides compelling evidence that early adolescents (age 10-12) can benefit from health literacy education. Prior research on adolescent brain development suggests that adolescent cognition improves markedly in early adolescence, reaching levels similar to young adults by about age 12 or 14. That is probably a universal phenomenon. So the new research study, along with prior research, suggests that education on health literacy for early adolescents may be effective in many places.”

But this raises the question of what “effective” means for an intervention like this one. How important are test scores in the real world?

Limitations of the study

“I can’t speak to how valuable this [intervention] would be in other parts of the world because I wouldn’t know what decisions they may be faced with,” said health literacy expert Berkman. “For example, the U.S. is one of the only countries that allows advertising for prescription drugs and so we are pretty much alone in having to be able to evaluate advertising claims.”

“The intervention will need to be tailored in each new setting….because the health literacy needs in one place may not be the same in another,” Osborne said.

Children at a school in Norway use the final version of the children’s book on tablet computers. Credit Matt Oxman/Informed Health Choices.

“[Another] limitation of the study is that the post-test does not evaluate the participants in relation to making real world choices,” Berkman added.

That’s a limitation the study authors also recognized, noting that “It is uncertain…what, if any, effect the programme will have on actual health choices.”

And while the researchers are optimistic about the ability to scale this intervention up for use in reaching a much larger number of kids, they also recognize that there are a number of hurdles that need to be overcome.

“In Uganda and other low-income countries, resources are a limiting factor,” Oxman told “Even though the cost of our approach was only about $4 per child, this is a lot of money relative to the amount currently being spent on education in Uganda and other low-income countries. In countries with more resources, this is less of a problem.”

For example, Oxman noted that an international school in Norway, where researchers piloted the intervention materials, is using the textbook on tablets, eliminating printing costs.

Osborne echoed these sentiments.

“Scaling up is possible, but with tailoring,” he said. “This will require huge investment for several reasons, including most teachers, headmasters and education leaders are forever bombarded with requests to add ‘important’ things to an already busy curriculum.”

What happens next?

Oxman told that the Informed Health Choices team is “conducting market and stakeholder analyses in different countries to make sure that our approach or an alternative approach will work and be used in schools.” For example, the researchers have had the intervention textbook translated into other languages and are conducting tests similar to the Uganda study in Kenya and Rwanda.

In addition, Oxman said the team plans to “continue to develop our list of key concepts and use that as the basis for a spiral curriculum for teaching primary and secondary school children to think critically about treatments and make informed health choices. A spiral curriculum is an approach to education that introduces key concepts to students at a young age and covers these concepts repeatedly, with increasing degrees of complexity.”

Not a silver bullet, but not a dead end is predicated on two core concepts. First, that helping people critically analyze claims about health care interventions can improve public dialogue about health care. And second, that it is important for people to play an active role in making decisions regarding their own care based on accurate, balanced and complete information about the tradeoffs involved in health care decisions.

Critical health literacy, the idea that one can develop the skill to critically appraise information of relevance to health, is essential to both of’s core concepts. That is, of course, why we are writing about this study.

But we are also widely known for our critical evaluations of how research findings are presented in news releases and stories. That’s part of our mission, and it applies to this Informed Health Choices study as well.

The take-away here is that, while this is a promising proof-of-concept study, it is still a proof-of-concept study. We need to see how well this intervention works in other parts of the world, and how (or whether) it affects the decisions people make about their health in the short- and long-term. We also need to see whether it can garner sufficient financial support to be viable on a large scale. And if it does get that support, we need to see whether researchers can get sufficient buy-in from communities to implement it.

So it’s not a silver bullet for poor health decision-making, but it doesn’t appear to be a dead end either. We look forward to following this work, and seeing where it leads.

You might also like


Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Comments are closed.