This news release by the University of Miami describes results from a study involving a head-mounted goggle that may diagnose concussions with more accuracy than a physical exam. Called the I-Portal goggle, the device gauges eye movement through video cameras and computers to assess balance, eye movement and reaction time in individuals who may have mild traumatic brain injury (mTBI).
The release suffers from frequent lapses into jargon and misleading statements regarding the new technology. For example, it states that it is “not inconceivable” that the I-Portal will be used on every sideline in the “near future,” even though this PLoS One study was the first clinical, peer-reviewed trial of the device. The news release also uses technical terms, such as sensitivity and specificity, without defining what these words mean or explaining their significance. Although it mentions the sensitivity and specificity figures, it doesn’t provide any other quantitative data or information on the study design. Furthermore, the news release does not disclose funding sources and conflicts of interest — a point we find serious, since the principal investigator is employed by the goggle’s manufacturer.
Mild traumatic brain injuries (mTBI), also known colloquially as concussions, are a short loss of normal brain function in response to a minor head injury and are a common type of sports injury. Although most symptoms resulting from a concussion go away on their own, some concussions can result in lasting effects on thinking, attention, learning and memory. They also increase the risk of having a second, more serious injury — although it’s not clear why. The current recommendation is to avoid physical activity until all symptoms have resolved.
However, concussions are also hard to diagnose, since most symptoms are self-reported and vary in intensity over time. Researchers also say there is not a distinctive threshold for diagnosing mTBI. If a medical device facilitates the process and more accurately makes diagnoses, that would certainly be newsworthy to the public, since more than 1 million cases of mTBI occur each year in America.
The news release mentions that researchers made diagnoses by running tests with the I-Portal goggle, which gauges eye movement through video cameras and computers. But there is no discussion on what the costs for a session with the I-Portal goggle may be.
This sounds like a new technology, and it would have been helpful to know the ballpark range of what it may cost for consumers.
For this reason, we give the news release a Not Satisfactory rating here.
Besides stating the sensitivity and specificity figures, the news release does not provide any additional information on the meaning behind these numbers. These are technical terms that can easily be confused with accuracy.
Sensitivity, also called the true positive rate, refers to the rate at which the test correctly identifies those who do have the condition.
Specificity, also called the true negative rate, identifies the correct proportion of negatives in a test (i.e. its ability to rule out those who do not have the condition).
Although 89% sensitivity and 95% specificity sound as if the test is quite accurate, it misses the main point, which is that the test performs less well than looking at symptoms. The study compared the new “objective test” to standard criteria for diagnosing TBI. What the study showed was that compared to standard methods, this new one missed 11% of those with TBI and over-diagnosed it in 5% of the non-TBI controls.
Although the sensitivity and specificity figures were given in the news release, none of these terms were explained for reporters and general readership. This is why we give it a Not Satisfactory rating here.
The news release did not mention any potential harms that may be associated with the goggles. Almost every health care intervention carries risks, although the severity may vary. The main potential harms are those arising from a false positive or false negative test. This could lead to unnecessary treatment or exclusion from play in the first case or no treatment for an injury in the second and potential for much more severe effects from a second head injury.
Since harms weren’t even addressed, we give the news release a Not Satisfactory rating here.
The news release does not elaborate on the research methods and quality of evidence, including the length and size of the study, its design and limitations. The reader is only given the sensitivity and specificity figures without any explanation.
The study looked at two cohorts of 50 participants with mTBI and one cohort of 100 participants without mTBI to serve as controls. Subjects were given a series of tests looking at their balance, eye movement and reaction time within 166 hours of an injury. They were then followed up for two weeks.
Researchers named two study limitations in the paper. Firstly, they defined the criteria for the TBI diagnosis themselves, since there is not a standard, objective threshold to diagnose mTBI. This may have included some individuals who may not show other “objective signs” on further examination, they wrote. Secondly, researchers relied on medical history and self report for the control population, as these subjects did not undergo baseline testing. As a result, this may have introduced recall bias into the study and chronic, asymptomatic individuals into the control cohort.
Since none of these details on the study was disclosed in the news release, we give it a Not Satisfactory rating here.
There is no disease mongering in this news release.
There was no mention of funding sources and conflicts of interest in the news release. We find this highly problematic, since the principal investigator of this study is employed by Neuro-Kinetics, the manufacturer of the I-Portal google. Not only is he paid by Neuro-Kinetics, but the company is his sole affiliation (meaning he has no other hospitals or university affiliations), as listed on the PLoS One research article.
The news release also did not disclose the funding sources for the study – which included the National Football League (NFL), Underarmor, General Electric and the Department of Defense – through the Head Health Challenge II grant.
For these reasons, we give the news release a Not Satisfactory rating here.
The news release says that mild traumatic brain injury is usually diagnosed through physical exam findings. One of the study authors adds the goggle does not require baseline testing — which includes assessments to gauge reaction time, memory capacity, speed of mental processing, and executive functioning of the brain, according to the Sports Concussion Institute.
It would have been interesting to compare the sensitivity/specificity numbers of the goggle to those of the physical exam (with baseline testing). Sure, 95 percent specificity sounds pretty good itself, but what accuracy can physicians achieve the traditional way (aka through the physical exam)?
Although the news release doesn’t address this, we feel it did a Satisfactory job here by mentioning the alternative.
The news release hints the goggle is not in widespread use — at least not yet. One of the study authors says, “It is not inconceivable that in the near future you will see the I-Portal goggle used on every sideline in America.”
It is our understanding that this PloS One article was the first peer-reviewed paper on the only clinical study of this product. We feel it is misleading and too early to claim that this goggle could be on “every sideline” in the “near future.” Not only that, the wording is also vague. What does the “near future” mean anyway? Until that happens, researchers need to collect more data and gain FDA approval for the device, which all take time.
As a result, we give the news release a Not Satisfactory rating here.
The news release makes it clear that this is a new technology, since one of the study authors had only been evaluating the goggle for two years. An Internet search shows that University of Miami received a NFL grant to test the new I-Portal goggle only in November 2014.
A researcher also comments in the news release that the PLoS One paper is the first to demonstrate “an objective method of diagnosing mTBI that relies on physiologic parameters.”
Therefore, we suggest the news release merits a Satisfactory rating here.
The news release does not use sensational language. While the statement “It is not inconceivable that in the near future you will see the I-Portal goggle used on every sideline in America” is over-the-top we addressed our concerns about that under the novelty criteria.