This news release describes a recently published journal article on a study that evaluated a blood test that was designed to detect concussions or traumatic brain injury in children. The study reports that head injury causes the release of glial fibrillary acidic protein (GFAP) from the brain into the bloodstream, and that the levels of GFAP in the blood correspond to the severity of the head injury. According to the release, the blood test correctly identified the presence of concussion or traumatic brain injury in 94 percent of the cases. But the release fails to mention that many positive results from the test would be false-positives, which would result in additional testing and may pose additional health risks. The release notes that concussion and traumatic brain injury are usually diagnosed based on symptoms — such as perceived balance problems or reports of headaches or grogginess — and that the blood test provides an objective test that can be used for diagnosis. However, the release fails to speak to a more important aspect of the marker, that is it’s potential negative predictive value. As noted in the study the biomarker has a 98% negative predictive value. That means that a negative result, if the biomarker holds up to additional testing, would rule out concussion.
According to the CDC, approximately 248,000 children were treated for concussions or traumatic brain injuries in U.S. emergency departments in 2009. And that number doesn’t include children who were taken to a family physician or who went misdiagnosed or undiagnosed. Concussions can have significant impacts on a child’s health and well-being, with symptoms ranging from headaches and dizziness to memory problems and behavioral changes. What’s more, the Mayo Clinic notes that having one concussion increases the risk of a child’s having a second concussion — and repeated concussions can have a multiplier effect. The development of an objective test that can accurately diagnose concussions in children without the use of CT scans, would certainly be a significant advance. However, it’s important to recognize the limitations of the current study and to address issues such as potential misdiagnosis.
The release doesn’t address cost at all. The release makes clear that the blood test is not ready for clinical use, stating that researchers “hope it will be commercially available within the next five years.” However, if the release is going to discuss what such a diagnostic tool would look like (e.g., “mobile devices that could diagnose concussions on the spot”), it should also provide a ballpark figure on what such a diagnostic tool might cost.
The release tells readers that the study correctly identified the presence of traumatic brain injuries 94 percent of the time, and that the levels of GFAP in the blood corresponded to the severity of the injuries. However, the release emphasizes only one side of the biomarker story, focusing entirely on the positive predictive value. While important, we think that the other side of the coin should have been addressed as well. Another presumed benefit of an easy-to-use biomarker blood test to detect concussion would be eliminating the necessity of a CT scan if, in fact, the biomarker had a high negative predictive value. The study notes a 98% negative predictive value. That means if the test comes back negative, there is a very low likelihood that the person has suffered a concussion. This could eliminate the need for a CT scan and any additional treatments. In reality, it is unlikely that a positive test would obviate the need for a CT scan. In fact, one could argue that the presence of a positive result would require a CT scan. The 94% accuracy notation in the press release is incorrect. It should have noted a 94% sensitivity (true positive) and a 47% specificity (true negative). The low specificity means that many positive results will be wrong (see more below under “Harms”).
The release is silent on the potential harms that could result from an easy-to-use biomarker test, especially one that has a 47% specificity (the ability of the test to correctly identify those without the condition). A test with only 47% specificity means that more than half of positive results will be “false positives.” The release does not address these “false positives” at all — meaning instances in which the test diagnosed a patient as having brain injury when there was no injury. A false positive could result in additional testing, which may create a financial burden, and — in the case of a CT scan — poses health risks of its own. Second, the release refers to the possibility of using the blood test to make brain injury diagnoses “on the spot.” Quoting one of the researchers who did the study, the release says: “The idea is to get a point-of-care test that could be used on the field to help the coaches, the trainers and the athletic directors, make a decision then and there about whether the child should go back to play.” However, the journal article the release is reporting on notes that GFAP is detectable in the bloodstream “within one hour of injury.” The release does note that such a tool should only be used to “help” adults make decisions about a child’s health, but this still raises some significant questions. What if a coach, for example, takes a blood test before the GFAP has had time to become detectable in the blood stream? Would that give the coach a false sense of confidence that the child is fine and can re-enter the game?
The release does not provide many important aspects of the study, The release says that researchers performed CT scans on 152 children and compared the results of those scans with results from the blood test. In fact, the study was of 257 children, of which 197 had blunt head trauma and 60 were trauma controls. Of the 152 children who received CT scans, 18 were found to have traumatic intracranial lesions. That means that 1-out-of 11 of the children scanned has evidence of a concussion. So the results in the story are really only based on 18 children out of the 152 with presumed traumatic head injury. The study authors note in the paper that “a much larger
number of children will be required to test the precision of the biomarker.”
The release notes that the journal article’s lead author receives funding from NIH. The study itself was partially funded by the NIH, a subtle but potentially important distinction. Further, the journal article notes that the lead author is a scientific consultant for Banyan Biomarkers, Inc. — a company that has intellectual property rights to GFAP testing. The journal article also states that the relevant author “receives no stocks or royalties from the company and will not benefit financially from this publication” — but that potential conflict of interest is still worth mentioning in the release.
The story provides information on the routine use of CT scanning as the standard for traumatic head injury. Since there are a host of biomarkers under study at the present time to detect concussion, we would have liked to have seen a couple of words on the other research taking place. For example, Banyan Biomarkers has one additional biomarker under study at the present time.
The release makes clear that the GFAP test is not available for clinical use, and that the researchers hope to develop a test for clinical applications within the next five years.
There is a robust body of literature on GFAP and brain injury, but the release does not mention that research at all. The journal article notes that “To the best of our knowledge this is among the largest published studies to date assessing GFAP in children and youth with mild and moderate [traumatic brain injury] in an [emergency department] setting.” That’s useful context, and the release should have included it. Even better, it could have told readers how this research fits into that existing body of work — is it consistent with previous findings? Does it expand on previous findings? That would be great to know.
The release calls the test “simple” in several places whereas our review shows that it would be anything but. Some other examples of hyperbole: “Concussions in kids are detectable by blood test,” “Blood test is 94 percent reliable in detecting concussions in kids, confirmed by CT scans.” The story backs off in the last line that states “Researchers plan to do more studies with the blood test, but they hope it will be commercially available within the next 5 years.” That’s not quite good enough, however. In the published results, the authors stated more appropriately, “Further study is required to validate these findings before clinical application.”
Systematic, Criteria-Driven
Monday at #ADA2021, an infectious disease epidemiologist, a journalist, and a researcher shared perspectives on communication of risk in the era of COVID-19: http://ow.ly/ROfR30rMeE7 @AstleyCM @garyschwitzer @berthahidalgo @ADA_DiabetesPro
Shoutouts to @susan_bewley @drkevinknopf @MichaelBaum11 in this call for more than fawning coverage from press releases for these kinds of stories.
Also, linked history inside goes back 5 years just on Grail apparently trying to become Holy Grail. https://twitter.com/garyschwitzer/status/1408849696416841731
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like