A study of about 600 patients at a non-profit Orlando hospital group showed that two markers in the blood of patients who had blows to the head could predict whether they had suffered head injuries. The news release about the study, published in JAMA Neurology, drew immediate widespread press coverage, including stories in the New York Times, Newsweek, NBC News and Time, whose story we also reviewed. We thought the release was well-written, but could have been improved with a better summary of the study evidence, by addressing funders, and by including a caveat that more study is needed. We were a bit troubled by the release talking about children’s injuries when this study only evaluated adults.
Thousands of people — both children and adults — suffer blows to the head and some suffer a concussion (also now know as traumatic brain injury or TBI). Unfortunately, no perfect test exists to quickly diagnose the level of severity of the injury other than symptoms or anatomic changes detected on a computed tomography (CT) scan. In the majority of cases, the symptoms resolve over time but in a minority of cases, symptoms worsen. A blood test for a potential marker of damage to the brain might one day help keep athletes and workers from returning to their work (or play) too soon. A simple test that could distinguish between those who have a mild injury from those with more damage would be useful. We previously reviewed the media’s handling of another approach using eye movements to distinguish between the two groups. In both instances, the research findings were encouraging. But we cannot stress enough that encouraging results are not conclusive. The title of the report clearly overstates the study results. While the test may eventually be useful in the detection of concussion there is little evidence at the moment that it can.
We were glad to see a very brief mention in the release that if the biomarker research is confirmed in larger studies, it might someday replace what the release calls “expensive” CT scans used in emergency rooms to try to diagnose concussion. But there is a tacit assumption that the biomarker would replace the use of CT scans. Experience would suggest, however, that new tests tend not to replace existing methods but are simply added to them. We also want to point out that if this new biomarker test became a commonplace screening tool, it could result in over-screening. Tools for screening have to be evaluated very carefully to have the greatest benefit for the right patients.
The release states that the blood test “was able to detect mild to moderate traumatic brain lesions with up to 97 percent accuracy in patients 18 years and older.” According to the published study, researchers used a statistic called Area Under Curve (AUC), a predictive model, that reflects both the sensitivity and specificity of the test. A value of 0.5 implies that the test has the diagnostic accuracy of a coin flip, whereas a value of 1.0 indicates perfect discrimination between diseased and non-diseased patients. Sensitivity and specificity are important characteristics of any screening or diagnostic test and we encourage health care journalists to incorporate these concepts into their reporting. The 97 percent accuracy figure used in the story appears to refer to the state of metabolic blood levels at 36 to 60 hours after injury. It appears that the initial determination over whether the patient had mild traumatic brain injury was highly accurate.
The release did not mention any potential harms of this screening tool. Just to mention one hypothetical case, it is possible that caregivers relying on the blood levels alone may err and fail to test in other ways for brain injury. In general, medical tests have two potential harms. The first is that the test does not demonstrate the presence of injury when in fact there is one. This would lead to under-diagnosis. The second is the test demonstrates the presence of injury when it does not exist. That would lead to over-diagnosis. While obvious, the report should have included some caveats to its otherwise glowing comments.
The release makes some inaccurate statements about the published study:
“Researchers analyzed nearly 600 patients for 3 years. When cross-checked with scans, the blood test was able to detect mild to moderate traumatic brain lesions with up to 97 percent accuracy in patients 18 years and older. The blood test also indicated which patients were in need of life-saving neurosurgery. This suggests that the blood test could be used by clinicians for up to a week after injury to detect brain injury. This is important because many patients with concussion may not seek medical attention for days after injury.”
Let’s put this into perspective. First, the study lasted for 3 years and did not analyze 600 patients for 3 years. There is a big difference. Subjects in the study were followed for 180 hours. In reality there were two groups of subjects studied, trauma patients with (N=325) and without evidence (N=259) of mild to moderate traumatic brain injury. The blood test did distinguish between the 7 subjects who had severe injury (2 of whom underwent a neurosurgical procedure and 7 who required hospitalization in an intensive care unit (ICU). So the sample of those who truly had significant injury was small indeed.
The release does not include disease mongering.
The release mentions that a lead researcher is “NIH funded” but does not specify funders for this study nor does it include a statement about potential conflicts of conflicts. (Just to clarify — we’re always pleased to see a statement even if no conflicts are known to exist.) The journal article itself has a disclosure footnote that states Papa is an unpaid consultant to Banyan Biomarkers, Inc., while Welch and Lewis disclosed receiving contract research funding from Banyan. Banyan is a for-profit biotech company in Florida seeking to commercialize a biomarker for head injury. We think this connection between the research and the biotech should have been mentioned in the release.
The news release described how trauma patients would routinely be evaluated and possibly have a CT scan in order to diagnose without the use of blood markers. It would have been useful to include a mention of other research involving blood markers for concussion that have been undertaken.
The release makes clear that the test is only part of a clinical trial and not available.
The release claims novelty through this statement, “We have so many diagnostic blood tests for different parts of the body, like the heart, liver and kidneys, but there’s never been a reliable blood test to identify trauma in the brain. We think this test could change that.” In addition, the release mentions the lead researcher’s earlier research involving this blood test.
This release includes some unjustified language. For starters, there is no such thing as a “simple blood test,” particularly one that tries to diagnose a brain injury based on a blood sample. We’ve addressed this in a previous review of an Orlando Health news release. We also have concerns about the use of the term “can” in the headline. This is still preliminary research and more studies are required to prove its usefulness.
Systematic, Criteria-Driven
Even when stories get the facts right, they often fail to provide crucial context and caveats that would help audiences make sense of the news — or lack thereof, says @garyschwitzer of @HealthNewsRevu https://t.co/b7YO2fYt0a
Why fact-checking is not enough to prevent the spread of misleading health care news...@HealthNewsRevu https://t.co/fzC2JbDYGV
Simply fact-checking an article "fails to capture questions of nuance and context that arise – or should arise – whenever evaluating medical evidence." https://t.co/HrYkGj12q1
Comments (1)
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
Sally James
April 12, 2016 at 4:25 pmGreat follow up story by Undark Magazine on this blood marker and the limitations. http://undark.org/2016/04/12/can-a-blood-test-diagnose-a-concussion/?utm_content=buffer40abd&utm_medium=social&utm_source=twitter.com&utm_campaign=buffer
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like