Read Original Story

AP’s look at baseline concussion testing for student athletes lacked critical eye

Rating

3 Star

Categories

MICHIGAN ATHLETIC GROUP SAYS CONCUSSION PROGRAM WORKS

Our Review Summary

Soccer PlayersConcussions incurred during sporting events in middle and high schools may have substantial health impacts on young athletes. Recognition of those impacts has grown exponentially in the US, generating earnest efforts to find ways to diagnose possible problems during games, before an athlete returns to the fray and runs the risk of making things worse. A number of companies now market baseline testing products, which usually involve a “baseline” neurological exam at the start of the season. Later, if the athlete is injured during a game, a shorter “sideline” exam can be conducted to compare to the original exam.

This AP story reports on a Michigan-based organization that made baseline testing available to schools in order to gather data over the course of a year about the frequency of potential concussions across ages, genders and type of sport.

The story does a nice job of summarizing the results of that study but treats the concept of baseline testing uncritically. Can the brief sideline diagnostic efforts in the midst of a game actually detect a problem in the making?  It depends on who you “ask.” Baseline testing is currently fashionable in medical circles. The Journal of the American Association (JAMA) has published articles concluding that baseline procedures are valid tests of cognitive function in athletes, a conclusion disputed by articles in other journals. The CDC has a webpage explaining baseline testing that offers details about when to do the tests but with no efficacy critique. And web pages from “The Sports Concussion Institute” tout the testing as evidence-based.

But is it?  We could find no controlled trials showing that neuropsychological testing improves outcomes after sports-related concussions.  And that, in turn, may mean that baseline testing is still a form of expert opinion masquerading as evidence-based science. Since companies are selling baseline testing products, the possibility of industry influence on the conduct of research exists. These details were needed in the to story to present a full picture of what’s at stake.

 

Why This Matters

Sports-related head injuries are increasingly implicated in permanent brain damage.  Determining their frequency and potential severity while there is still time to treat or, ideally, avoid the problem is critical.

Criteria

Does the story adequately discuss the costs of the intervention?

Satisfactory

The story includes details about costs incurred by the state-based organization that conducted the study.

Does the story adequately quantify the benefits of the treatment/test/product/procedure?

Not Satisfactory

While the story heralds the “positive impact” of this testing program and goes on to note that schools participating in the pilot program “removed players for possible concussions at a higher rate” than schools not in the pilot program, the story contains no data that would allow the reader to assess that comparative statement. And if there isn’t any, that should be made clear.

Through no fault of the Michigan High School Athletic Association, we don’t know if removing more students from sports activities saves their brains from long-term harm, since no studies have been done.

Does the story adequately explain/quantify the harms of the intervention?

Not Satisfactory

Pulling high school athletes, or any athletes, from a game or labeling them inaccurately as being concussed and missing additional games (which could happen with a false-positive result) can cause anxiety and disappointment.  Many athletes would be devastated by a lost opportunity to participate in the big game.

Does the story seem to grasp the quality of the evidence?

Not Satisfactory

There is no discussion of the evidence for this method–which makes us wonder why the headline of the story was “Michigan athletic group says concussion program works.” How do they know it works?

Baseline testing (see description of what it is in the summary above) has become an increasingly popular strategy for diagnosing potential head injuries.  But while much reflection on the technique to date—including this story—is uncritical, this type of testing has not yet survived controlled clinical trials, to the best of our knowledge.  In an evidence-based world, the data are not yet in hand–and the story should have made that clear.

Does the story commit disease-mongering?

Satisfactory

Possible head injuries in sport are receiving much-deserved attention.

Does the story use independent sources and identify conflicts of interest?

Not Satisfactory

The story includes no independent medical experts who could provide perspective on this type of program.

Does the story compare the new approach with existing alternatives?

Not Satisfactory

While the story does not compare the baseline testing concept with alternatives, it does note that several companies now market concussion-testing procedures.  It makes no effort to compare those products, or to compare other strategies to evaluate athletes.

Does the story establish the availability of the treatment/test/product/procedure?

Satisfactory

Concussion-testing programs are clearly available.

Does the story establish the true novelty of the approach?

Satisfactory

The story makes the claim that this study is the “first-ever head-injury survey” conducted by the Michigan High School Athletic Association.  That is true.

Does the story appear to rely solely or largely on a news release?

Satisfactory

The organization composed a news release to accompany the release of the study.  If the reporter encountered it, he clearly supplemented that material with his own reporting.

 

Total Score: 5 of 10 Satisfactory

Comments

Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.