Matt Shipman is a public information officer at North Carolina State University and a frequent story and news release reviewer. He tweets as @shiplives.
Concussions have been in the news a lot recently, in stories that focus on everything from the National Football League to potential diagnostic tools to recovery times or long-term health effects. But we at HealthNewsReview.org have noticed that some of the same flaws crop up repeatedly across many of these stories and news releases — especially in those about tests that propose to identify concussed athletes more quickly and accurately.
For example, few of the concussion-related stories and releases we’ve reviewed have adequately addressed cost – if they’ve addressed it at all.
This post is not a rant about bad writing. Instead, this post is an acknowledgment that concussion research can be incredibly complex. And rather than simply continue to review concussion-related news stories and releases, we want to outline some key points that reporters and press officers may want to consider when writing about concussion research.
We’re optimistic that these guidelines can serve as a resource to help writers craft stories that are more useful to readers. After all, one of HealthNewsReview.org’s fundamental goals is to improve the quality of health-related news.
The first point any story about concussion testing needs to address is accuracy. More specifically, concussion test stories need to address the balance between sensitivity — which is how good the test is at positively identifying people who actually have the problem – and specificity – how good the test is at ruling out people who don’t have the problem.
“No test is 100 percent accurate,” Dr. Steve Atlas said via an email interview. Atlas is an associate professor of medicine at Harvard Medical School and director of the Massachusetts General Hospital Primary Care Practice-Based Research & Quality Improvement Network. He’s also a reviewer for HealthNewsReview.org who’s looked at a number of concussion-related stories and news releases.
“In general, there is an inherent tradeoff between sensitivity and specificity,” Atlas said. “The more sensitive the test is – i.e., picking up all those who have the problem – comes at the cost of a lower specificity – i.e., including patients who the test says has the problem but who really don’t. For many screening tests, such as a mammogram, we accept lower specificity because we don’t want to miss any breast cancers. The result is a lot of false positives. For example, for women in their 40s who undergo regular mammography, 1 in 4 will have a false positive test result (again sacrificing specificity for sensitivity).”
Some news stories and releases talk only about specificity or only about sensitivity – but both need to be addressed to give readers a clear understanding of how accurate a new technique is at detecting concussion. And some stories – like a TIME article we reviewed in March – don’t address accuracy at all.
That’s deeply problematic, because accuracy tells readers the likelihood of false positive and false negative results – and those false results have consequences.
News stories and releases related to concussion testing often fail to address the potential harms associated with these false positive and false negative test results.
“For athletes, there are definitely harms associated with a false positive diagnosis,” said Dr. Patricia Gibbons, an internist at Massachusetts General Hospital with expertise in adult concussions who works with professional athletes. “You could change the course of their career, and there are psychological health risks.”
That said, Dr. Gibbons would be the first to assert that a false negative result can lead to even greater harm.
“What’s the risk of delayed or missed diagnosis? The sooner you get a kid into healing or rehab mode, the safer they are,” said Dr. William Roberts, a professor of family medicine and community health at the University of Minnesota and chair of the Sports Medicine Advisory Committee for the Minnesota State High School League. “You want to give the brain a chance to heal before getting hit again, because the injury could be made worse.”
“Not diagnosing a concussion can potentially lead to risk for second impact syndrome,” Tracey Covassin said in an email interview. “Second impact syndrome is when an athlete has…not recovered from a first concussion and gets another concussion. Although very rare it can be catastrophic, leading to permanent brain damage or death.” Covassin is director of the Sport Concussion Laboratory at Michigan State University and served on the Committee on Sports-Related Concussions in Youth for the National Academies of Sciences, Engineering and Medicine.
In short, as a 2015 news release review noted: “While the test itself may not produce harm, the question is what if the test is wrong? Say the test is positive and in fact the person does not have a concussion? May they receive additional tests or treatments that may cause harm? More importantly, what if the test is normal and the patient has a concussion? The test may falsely reassure, and the patient returns to the activity that caused the problem in the first place, risking additional serious injury.”
We don’t expect every news story or release to provide an exhaustive list of all the possible adverse outcomes associated with a misdiagnosis – but we do think readers should expect stories and releases to at least attempt to address these potential harms in a meaningful way.
There can be significant benefit from early identification of concussion.
“Early concussion diagnoses can decrease the amount of time an athlete has concussion symptoms or impairments,” Covassin said.
However, it is important for those writing stories and releases to be aware that there’s no evidence that more testing leads to less long-term cognitive impairment. Good diagnostic tests are not a panacea that makes athletes safer.
This is something that is often overlooked, as in a 2015 story from the New York Times on a concussion screening test that makes use of flashcards. In its review of that story, HealthNewsReview.org noted that “such tests could give parents and coaches a false sense of security. There is no evidence that any of this actually prevents young athletes from developing long-term cognitive impairment. While this is true for current practice as well as for this new test, seeing children getting tested on the sidelines more frequently might give false reassurance. One could argue that the safest thing for a concerned parent is to avoid contact sports that are prone to involve head injuries.”
HealthNewsReview.org isn’t advocating that parents pull their children out of contact sports. But we are advocating for writers to acknowledge that concussion tests can only detect problems, not prevent them.
One challenge in determining how well new tests perform at detecting concussion is that there is currently no “gold standard” for concluding that a person has a concussion. So, what is the new test being compared to?
A 2015 news release from Hennepin County Medical Center about a new technique that used eye tracking for concussion diagnosis addressed this issue directly: “When doctors look for a biomarker for heart attack, it is relatively easy to check the accuracy of a potential candidate because they can perform a cardiac catheterization and confirm that the heart vessel is blocked and an attack has occurred. There is no analogous capability with brain injury – there is no gold standard diagnostic, no blood test, and no imaging study for definitively concluding that a patient has experienced a concussion.”
And HealthNewsReview.org applauded the way that release openly addressed the difficulties inherent in establishing whether a concussion has occurred.
Other stories and releases fail to address these limitations in a meaningful way.
For example, a release issued earlier this year announced that a “simple blood test” could detect evidence of concussions. As our review of that release noted, “no perfect test exists to quickly diagnose the level of severity of the injury other than symptoms or anatomic changes detected on a computed tomography (CT) scan….we cannot stress enough that encouraging results are not conclusive. The title of the report clearly overstates the study results.”
And it’s important to pay attention to the study designs themselves.
“Only prospective studies that evaluate diagnostic tests at the point of injury and then follows the patient over time to assess outcome can be used to truly evaluate the proposed test for concussion,” Atlas noted. “‘Retrospective’ studies that do the calculation or evaluation after the fact (or after the outcome is known) are more prone to bias.”
Even if a new testing tool or technique is extremely promising, it’s important to make readers aware of how viable that tool or technique will be in the real world. Will school sports teams be able to afford them – or know how to use them if they did?
An April 2016 release from the American Academy of Neurology highlights the potential for using Doppler ultrasound to diagnose concussions. But doesn’t address the practical questions this raises.
In its review of that release, HealthNewsReview.org noted that: “While the study’s principle investigator notes that the device may be ‘less expensive’ than MRI, costs are not broached in the text. To have one of these devices at all sport events, say, starting at high school and beyond, would be quite expensive. Plus, staff would need to be trained to use the device and interpret the results. Dopplers are not used and interpreted by the lay public. The practicality of this was not mentioned.”
Many releases and stories don’t acknowledge the difference between trained professionals administering tests for a research study and coaches or parents administering tests on a sideline. And that difference is significant.
“Concussions are diagnosed by a licensed health care provider,” Covassin said. “Typically if an athlete has an athletic trainer present they will do the initial assessment which includes a multi-faceted approach. This will include a sideline mental status measure, balance test, vestibular/ocular test and a cognitive measure. In addition, it always starts with a symptoms assessment and clinical examination. Then the athlete should be referred to a physician and cleared to play by a physician.”
When asked how difficult it is for non-professionals to diagnosis a concussion at the site of an incident, Covassin said: “Very difficult. And should only be done by a trained, licensed healthcare provider.”
This raises the issue of cost. Training coaches or athletic staff to use any testing tool can be time consuming and expensive.
Even the concussion screening flashcards mentioned in the NYT story require players to have taken a baseline test ahead of time in order to make any sort of accurate assessment. That’s time-consuming.
When writing about research findings, it’s important to place those findings in context.
For example, if a story is focused on chronic traumatic encephalopathy (CTE) in football players, it needs to explain what we know about the connection between high-impact sports and CTE. And this can get tricky, not only because of what we know (or don’t know) about CTE in football players, but because of what we know (or don’t know) about CTE in the general public.
“We don’t really know the incidence of CTE in the normal population,” Roberts said. “That makes it difficult to understand the impact of repeated collisions” in football players.
In other words, because we don’t know how common CTE is among the general population, we can’t say whether CTE is more common among football players. Instead, we have a simple association: some football players have CTE. Does that mean playing football caused the CTE? We don’t know. It’s important to lay these issues out for readers, and to make clear the differences between correlation and causation.
Writing about concussion-related research is challenging. But it’s an important subject, and one that readers are interested in. Hopefully, these guidelines can help reporters and press officers craft stories and releases that offer real analysis of claims about concussion interventions – and give readers the information they need to make informed decisions about their health and care.
Did you enjoy these tips? You might also like 5 tips for writing better health news headlines.