This release summarizes a very small study testing the use of plasma metabolites in diagnosing concussion. The researchers performed a retrospective analysis on blood samples taken from 29 male hockey players who had earlier been diagnosed with or without a concussion at a sports clinic. The authors examined a large number of potential chemicals that could identify patients with a concussion and appear to have identified around 17 that best predicted a concussion in this small population. However hopeful these results may be, there are some important caveats that weren’t highlighted in the news release. One was simply acknowledging the small number of athletes, all were male and participated in one sport, hockey. Whether these results may be generalizable remains to be determined. Second, the results were based upon retrospective analyses. The usefulness of such a test is when applied prospectively — or in real time. Whether the results will be the same when used prospectively is unknown. Finally, and maybe most important, how would these results be used to improve care? And do these results provide additional information beyond current practice or other tests that lead to better identification, classification and ultimately treatment. For all these reasons, the results from this small, preliminary study should be viewed cautiously and not as a clear “game changer.”
We have reviewed several stories and news releases about simple blood tests for concussion and remain wary when we see the term “game changing.” (See our tips on writing about concussion research.)
The impact of concussion and repetitive head trauma — or traumatic brain injury (TBI) — has been highlighted in recent years by studies showing severe damage in professional football players in the United States. This has increased concerns for younger athletes and led to efforts to better identify and treat TBI. Key issues include how to diagnose athletes at the time of a potential injury to determine whether they can return to play, and for those initially thought to have a concussion — how best to treat them and determine when it may be safe to return to competitive play. A large number of tests have been proposed and are in various stages of evaluation including: survey questionnaires, brain imaging studies and blood tests.
A blood test that can determine if a concussion has occurred may be a very important addition to the armamentarium used to diagnose concussion or traumatic brain injury (TBI). But while this new study might seem like a novel, ‘game changing’ discovery, the preliminary nature and other drawbacks warrant caution.
We are told the test is “relatively inexpensive” as would be the drawing of blood, but we’re not given a hint at what the cost is for measuring more than a dozen metabolites. The release might have better addressed the unknowns about this preliminary test by letting readers know that a commercial test is not yet available and its cost remains unclear.
Even though we are told that “with fine tuning we can now look at sets of as few as 20-40 specific metabolites and maintain the diagnostic accuracy level of the test over 90 per cent,” we’re not sure what this means. It appears that the reference to more than 90 percent comes from the study abstract. While this provides some quantitative data on the overall effect, it does not provide the sensitivity (how many with a concussion had a positive test results) or specificity (how many without a concussion didn’t have a positive test result).
A key question is diagnosing a concussion at the time of the potential injury is to determine whether a player can return to the game. It appears this test is not designed for that. The question then is how could this test be used. It isn’t clear.
No potential harms are mentioned, but they certainly exist. We encourage releases on diagnostic and screening tests to address the potential for false positive (over-diagnosis) or negative results (under-diagnosis) that could have an impact on patient care.
Does it matter that this study was done with only 29 hockey players, 12 of whom had concussions and 17 who had not? Is this significant or not? We don’t know because the release offers very little discussion of the evidence.
The release would have done readers a greater service if it had gone beyond stating the results were “preliminary” and acknowledged that so few patients were studied that it is premature to offer a conclusion about the test, that there was no comparison with other methods for identifying a concussion, and that the study was based on retrospective analysis. Among the disadvantages are that some key statistics cannot be measured, and significant biases may affect the selection of controls.
Concussions are a real and growing problem, especially among those who play contact sports. There is no mongering here.
The news release notes the funder and acknowledges that the “The technology is subject to a patent application filed through WORLDiscoveries®, the joint technology transfer office of Lawson and Western.”
The news release mentions that there are other ways to diagnose a concussion. It notes:
“Diagnosis of a clinically significant concussion, or a mild traumatic brain injury, can be difficult as it currently relies on a combination of patient symptom assessment and clinician judgement.”
“This new method…is unique in that previous attempts have looked unsuccessfully for a single highly accurate protein biomarker that can distinguish concussed from non-concussed adolescent patients”
We’ll rate it borderline satisfactory for giving a nod to alternative tests. However, the release would have been more informative if it had included some actual numbers comparing the potential test with traditional tests and other blood tests in development.
We aren’t told if it is available or not or even when it might become available. The release could have been much more transparent about its availability. When we assessed the release it appears that there isn’t a test per se, but a series of chemicals that could be combined into a future commercial test.
It appears that the novelty of measuring metabolites has been well established in this study.
Using the term ‘game-changing’ in the title is unjustified. Few technologies (especially one tested in only 19 individuals) can be called game-changing. This is a very preliminary result, and while encouraging, it in no way provides evidence that this is a game changer. It may be, but that can be said of many tests and treatments in early development that later do not pan out.