NOTE TO READERS: When this project lost substantial funding at the end of 2018, I lost the ability to continue publishing criteria-driven news story reviews and PR news release reviews - once the bread-and-butter of the site going back to 2006. The 3,200 archived reviews, while still educational, are getting old and difficult for me to technically maintain on the back end of the website. So I am announcing that I plan to remove these reviews from the site by April 1, 2021. The blog and the toolkit - two of the most popular features on the site - will remain. If you wish to peruse the reviews before they disappear, please do so by the end of March 2021. After that date you may still be able to access them via the Internet Archive Wayback Machine - https://archive.org/web/.
Read Original Story

School of hard knocks

Rating

2 Star

School of hard knocks

Our Review Summary

Knowing when to allow an athlete to return to play after a concussion can be very difficult. In rare cases, the wrong decision can be devastating. This story describes how a young hockey player’s lingering symptoms after a concussion prompted his former high school to adopt a computer-based testing system to prevent future problems in other athletes.

Testing athletes’ neuropsychological performance before they have a concussion seems like a good idea. “That way,” as the story explains, “after an injury, a retest can accurately reflect whether the brain is back to normal, allowing a safe return to competition.”

Unfortunately, it isn’t that simple—and this story misses an opportunity to explain some of the complexities. To our knowledge, there is no gold standard for measuring neuropsychological performance or the severity of symptoms. Neuropsychological testing is one potential solution, and merits consideration. The story raises a fearful specter—it says 3.5 million Americans have a concussion annually, and repeated concussions can lead to “debilitating headaches, dizziness, and bouts of depression.” The story does not mention the very low prevalence of the most feared outcomes, the so-called post-concussive and second-impact syndromes.  Nor does it say whether most concussions resolve without lasting problems.

In addition, the story provides no scientific evidence that neuropsychological testing actually leads to fewer dangerous outcomes (which include death), nor evidence that neuropsychological testing can detect problems during the key period after a concussion when athletes say they no longer have symptoms. It mentions no potential harms and no computer-free diagnostic options. It quotes two prominent experts who have devoted substantial portions of their careers to studying concussion in athletes and promoting neuropsychological testing—but it does not mention potential conflicts of interest or cite a single dissenting, independent voice who questions the benefits touted by these advocates.

In the end, the story seems to sow fear and tout a solution whose intuitive appeal is enormous but whose scientific value is unknown.

Criteria

Does the story adequately discuss the costs of the intervention?

Not Satisfactory

The story mentions the cost of one type of screening ($1000 annually for athletes playing Winchester High contact sports), but it does not mention the cost of follow-up testing for athletes with suspected concussions nor the cost of the Boston University or Impact programs. School districts are struggling to provide the basics: what’s the true bottom line for adding these tests?

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

The story provides no quantitative estimate of the benefit of neuropsychological testing in young athletes.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

Does neuropsychological testing pose any potential harms? Like other screening tests, does it cause needless anxiety about problems that will never materialize? Does testing unnecessarily withhold athletes from play—and lead to inactivity and depression? Have athletes with “normal” tests ever returned to play and suffered the very problems the testing was intended to prevent? Who owns the sensitive data culled during testing, and how is it protected? What harms are posed by spending money on this instead of on other basic school needs?

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

The story fails to describe the quality of the evidence to support baseline neuropsychological testing in adolescent athletes. Ideally, researchers would be able to show that baseline testing combined with careful post-concussion testing and monitoring leads to fewer bad outcomes. The outcomes of greatest importance are death and the long-term disability caused by the so-called post-concussive and second-impact syndromes. Can the test reliably distinguish between people who should not return to sports and those who can? Have researchers compared the reliability of these tests to standard, low-tech methods? To a gold standard test? IS there a gold standard? (To our knowledge there is not.)

 Finally and most importantly, despite the assurances of the physicians interviewed, the story provides no scientific evidence that neuropsychological testing reduces the rare but serious problems it is designed to prevent. 

Does the story commit disease-mongering?

Not Satisfactory

The story cites CDC statistics saying there are more than 3.5 million sports- and recreation-related concussions annually. This vastly overstates the number of concussions that will afflict the high school athletes who are the main subject of this story. Most concussions in athletes are characterized as “mild” traumatic brain injuries (TBIs). Data currently on the CDC’s web site counts the TOTAL number of all types of TBIs (mild to severe), from all causes in all age groups at 1.4 million. (http://www.cdc.gov/ncipc/tbi/TBI.htm) How many of these are in athletes? A 1997 CDC report put the number at about 300,000 (MMWR Morb Mortal Wkly Rep. 1997 Mar 14;46(10):224-7; http://www.ncbi.nlm.nih.gov/pubmed/9082176) A 2006 report from the National Federation of State High School Associations estimated that about 7.2 million students participate in high school athletics in the U.S. Of these, some 1.2 million will have an injury of some kind. Of those that are injured, a small minority will have a concussion. The story does not mention the low prevalence of the most feared outcomes, the so-called post-concussive and second-impact syndromes; the reader is likely to believe the prevalence is quite high.  Nor does the story say whether most concussions resolve without lasting problems.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The sources of this story are a hockey player who suffered a concussion in high school, his father, and 2 professionals–a neurosurgeon and a researcher–who have devoted substantial portions of their careers to studying concussion in athletes and promoting neuropsychological testing. But there is no mention of potential conflicts of interest (Dr. Lovell created the commercially distributed Impact test)—and no dissenting, independent voice who questions the benefits touted by these advocates.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

The story mentions no alternative methods for determining when it might be safe for an athlete to return to play after a concussion. Many team trainers and physicians employ a variety of low-tech “paper-and-pencil” aids and clinical judgment to make these decisions. How do the newer, computer-based tools compare?

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

The story explains that there are a variety of ways to perform baseline computer screening in athletes who may have a concussion in the future, and mentions several local schools where the programs are available. The story implies that most high schools don’t have such programs, and suggests that their routine use would be beneficial.

Does the story establish the true novelty of the approach?

Satisfactory

The Impact system and other neuropsychological testing systems have been around for years. The story implies this by pointing out that Impact is widely used in professional football and ice hockey, as well as a number of schools.

Does the story appear to rely solely or largely on a news release?

Satisfactory

There is no obvious use of text from the press release.

Total Score: 3 of 10 Satisfactory

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.