Read Original Story

How good is the flu shot? Vox delivers an exemplary answer


5 Star


How well do flu shots work? Here’s what the science says.

Our Review Summary

This story asks a simple, straightforward question that annually affects much of the population — do flu vaccines work, and how well?  The story looks at the evidence offered from a host of studies, giving a fair appraisal of what the data does and does not tell us.

[Editor’s note: for another perspective on this issue, see last week’s blog post by Alan Cassels: This flu season, let’s immunize ourselves from the annual infection of exaggerating relative risk reductions.]


Why This Matters

Outbreaks of influenza come as regularly as the leaves changing color in the fall.  And like the autumn colors, the effectiveness of the flu vaccine can change year to year.  This story both explains why there is such variability and answers the questions that most people would, or should, have about getting an annual flu shot.


Does the story adequately discuss the costs of the intervention?


While the story only touches on the costs of flu shots in passing, saying that they’re relatively cheap, we’ll give them the benefit of the doubt on this category.  At least at two points in the story, the issue of costs are raised.  We would have liked it even better had the story mentioned the actual range of possible costs for the flu vaccine.  A quick web search shows the prices ranging from $10 to $30, based on information from the CDC.  And free shots are regularly available at public health centers as well.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?


The story does a good job of explaining what the actual benefits of the influenza vaccine really are.  Just as a recent blog post did, the story goes way beyond the typical media version touting claims of X-percent effectiveness and explains what that means in terms of individual people escaping flu symptoms.  This accuracy of interpretation does carry with it, however, one risk:  For a public so used to hearing that vaccines are, say, 60 percent effective, actually finding out that they’d only prevent one person from getting symptoms for every 33 people vaccinated may not sound good enough for the lay person.

Does the story adequately explain/quantify the harms of the intervention?


The story does mention that the risks of negative reactions to getting a flu shot usually center on inflammation at the site of the injection, a comparatively minor side effect.  It emphasizes that given such a minimal risk, there are very few harms for getting this protection, minimal though it may be in some years.

Does the story seem to grasp the quality of the evidence?


The story gives readers the results of numerous meta-studies that compile partial answers on the vaccine’s effectiveness.  It also rightly points out the limitations of those studies, and the research they’re based on, in that ethics forbids the kind of control versus placebo trials that could show cause and effect.  Nevertheless, the story is clear in pointing out the limitations of the data it presents, giving readers enough information to make their own decisions.

Does the story commit disease-mongering?


The annual influenza outbreaks offer the possibility for disease-mongering but we feel that’s not the case with this story.

Does the story use independent sources and identify conflicts of interest?


The author of this story compiles ample evidence from reliable and respected independent sources concerning influenza.   It does not, however, offer any information concerning conflicts of interest, but with this type of story, we’re hard pressed to see how there would be a conflict.

Does the story compare the new approach with existing alternatives?


The story does mention the meager alternatives to influenza vaccination — that is, proper hand washing hygiene and the isolation of flu patients from those free of the disease.

Does the story establish the availability of the treatment/test/product/procedure?


The story offers answers to basic questions about the value of getting an annual flu shot.  Those questions are obviously based on the fact that flu vaccines are usually readily available for those who want them.  While some years present a possible shortage of vaccine, for most years, vaccination is determined by personal choice, rather than availability.

Does the story establish the true novelty of the approach?


The story appropriately explains that this vaccine has been around for decades. The story is taking a second look at the vaccine’s effectiveness, which as it explains varies from year to year and may not be as high as most people think.

Does the story appear to rely solely or largely on a news release?


The story shows no indications that it was based on a news release.

Total Score: 10 of 10 Satisfactory

Comments (1)

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Shannon Brownlee

October 19, 2015 at 3:38 pm

In 2009, Jeanne Lenzer and I published an article on the shortcomings of the flu vaccine in the Atlantic, “Does the Vaccine Matter?”, and you would have thought we had declared that God is dead. You can find the article here: We were accused of being anti-vaccine (we’re not); stupid (read the article and judge for yourself); Google “optimizers” (whatever that is); and just plain wrong. The one response we did not get was a randomized controlled trial of the vaccine, which would go far in helping settle the question of whether or not the vaccine is effective in the people who need it. The one thing Julia Belluz’ otherwise excellent article on the vaccine did not tell readers is that most of what we call “flu” is not actually caused by influenza virus, which is the only virus that the flu vaccine protects against. It won’t prevent you from getting corona virus or any of the other nasty bugs that can cause flu-like symptoms.