Read Original Story

A Simple Flashcard Test to Detect Concussions


3 Star


A Simple Flashcard Test to Detect Concussions

Our Review Summary

This story describes an alternative test for concussion that can be administered by parents and volunteer coaches at minimal cost. The omission of any mention of the potential weaknesses of the test is a glaring one. Speaking with an expert who was not associated with the study likely would have shed light on important limitations of the research. Diagnosing and treating concussion in youth sports remains a controversial topic among concussion experts.


Why This Matters

The recent attention paid to the serious consequences of repeated head injuries in professional athletes, such as those playing in the National Football League, has appropriately led to concern in student athletes. Prevention efforts have focused on identifying athletes who should be removed from games and trying to determine how long they should avoid contact. The tests described in this story seek to more accurately identify at-risk athletes. The new test, one that focuses on eye function, is compared to one that is already in use, but which requires professionals to administer. Though the results of this new test look impressive, especially compared to the Standardized Assessment of Concussion (SAC), it would be premature to recommend this for the reasons highlighted below in the review. It is also worth emphasizing that until there is more information on the frequency of such head injuries, the cognitive consequences, and the predictors of these outcomes, the use of this new test or any other may be providing false reassurance to parents of athletes. We have a lot to learn, and though this study may be a step in the right direction, it is a very small one.


Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The story notes several times that the cost per player is minimal, but does not offer details. A check at the King-Devick test website revealed that the tests are sold in packages under an annual license for testing from 5 to 1,000 athletes at a cost of $50 to $1,000. With 100,000 public schools in the United States, the test could be a major revenue stream for the test owners if widely adopted. In addition, while the article talks about this being done by volunteer coaches and parents, one could imagine that this could be required for high school athletes. Performing a baseline test in all athletes could fall to teachers and coaches who are paid and may be a significant burden in terms of time required to administer the baseline tests. It is also unclear how often this in-depth baseline testing would need to be repeated.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?


The story provides a good discussion of how the vision test compared in reliability with the standard testing protocol including a cognitive test and a balance test. The tests were applied to 243 young (age 5-18) and 89 college athletes playing a season of hockey or lacrosse.

Among the 12 students that sustained head injuries, the researchers found the King-Devick test correctly assessed concussion in 75 percent of the injured players and misidentified one. In comparison, the standard testing protocol accurately assessed only two of the concussed athletes and missed 10. The standard tests also misidentified three athletes as having sustained a concussion, according to the study. The study authors concluded that the vision test was the more sensitive, accurate test.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

Quoting a researcher, the story suggests that parents and coaches should “absolutely consider” using the test. However, the small number of young athletes tested and the testing that occurred under the supervision of research personnel (who won’t be on the sidelines of youth sports games to train parents and volunteers doing this test on their own) all suggest that it would be inappropriate to state that this test has in any way been shown to be safe. Only testing in a large number of students with potential injuries, across a range of ages, backgrounds and sports, with attention to measuring cognitive outcomes in a uniform fashion could lead to this test being demonstrated to be the same or better than current practice. In addition, we’d note that a potential harm of this test is that it could give parents and coaches a false sense of security. There is no evidence that any of this actually prevents young athletes from developing long-term cognitive impairment. While this is true for current practice as well as for this new test, seeing children getting tested on the sidelines more frequently might give false reassurance. One could argue that the safest thing for a concerned parent is to avoid contact sports that are prone to involve head injuries.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

Although the article does a good job describing what the study did, there is little attempt to describe limitations or to put this into a larger context — something that the story could’ve done by consulting an independent expert. The story states that parents and coaches who administered the test in this study were all supervised by NYU personnel. Can parents and the general public reliably administer and interpret the test in real-life situations without such supervision? Can they detect reading delays of seconds or fractions of seconds with consistency? That doesn’t seem to have been established. In addition, it would have been important to redo the test in a large number (ideally all) athletes who underwent the baseline test regardless of whether they were injured or not. (The study only retested a small number of non-concussed controls.) One would like to see the effect of the test across a range of injuries – from none observed, to mild, to moderate, to major. Finally, the study cannot tell us–and the story never mentioned–whether implementing such a test as part of routine athletics actually decreases the likelihood of cognitive impairment in a population of athletes. Even if the test is good, is it actually making our student athletes any safer?

Does the story commit disease-mongering?


The story does not engage in disease mongering. If anything, the story could have reinforced the seriousness of the issue of concussion among student athletes. The University of Pittsburgh’s Brain Trauma Research Center, one of numerous institutions conducting concussion-related research, estimates that 300,000 sports-related concussions occur annually in the United States, and that athletes have a 19 percent chance of experiencing a concussion each season.

That being said, we don’t want to be accused of disease-mongering ourselves, so we’d add that it’s not at all clear what the long-term consequences of these concussions are.  There are obviously news stories about tragic events, but how large a problem is it in terms of long-term impairment? And how does it compare to head injuries resulting from non-athletic events? While there is a lot of attention right now on head injuries – and that is a good thing if it leads to more studies – it isn’t clear how serious a problem it is compared to other health risks.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The story includes only one source — the lead author of the study. Considering there are numerous resources now devoted to sports-related concussion research, diagnostics and treatment, and considering there is quite a bit of disagreement among experts as to what are the best methods, this story would have been a lot stronger with additional voices.

Does the story compare the new approach with existing alternatives?


The most commonly used testing method — the Standardized Assessment of Concussion — which measures cognition and a timed tandem gait test of balance were part of the research being reported and were described in the story.

Does the story establish the availability of the treatment/test/product/procedure?


The story does not explicitly state that one can purchase this test, but the implication is that it is available. We’ll give the benefit of the doubt, but we’d prefer it if the story linked to the sole supplier of the product offering yearly packages from its website. It’s also available on the iTunes App Store.

Does the story establish the true novelty of the approach?


The test is portrayed as an inexpensive alternative to standard cognitive and balance testing which can be administered by adults without medical training. The story also notes that “in recent years, trainers working with athletes in sports such as boxing and mixed martial arts, where concussions are common, had begun supplementing the Standardized Assessment with a simple vision exam, known as the King-Devick test.” So it’s clear that the test hasn’t just appeared out of the blue, and has been used in other sports populations.

Does the story appear to rely solely or largely on a news release?


The story relies on one source – the lead author — but quotes in the story are different from the news release so we can assume the author interviewed the study author.

Total Score: 6 of 10 Satisfactory


Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.