Michael Joyce produces multimedia content for HealthNewsReview.org and tweets as @mlmjoyce.
A Mayo Clinic study, published Tuesday in the Journal of Evaluation in Clinical Practice, compares the initial diagnosis given to patients by their primary doctor, with the diagnosis they ultimately received by Mayo internists after being referred for a second opinion.
Many news outlets framed the results the way UPI did in its headline:
Here’s the first problem: that framing — which is picked up from the study itself and the news release from Mayo — assumes that the Mayo diagnosis is the “correct” diagnosis even though the authors themselves warn:
“Because we are a referral practice, there is no longitudinal follow-up of patients to confirm the second opinion”
In other words, because Mayo didn’t follow up with these patients to find out whether their second opinion was right, there’s no way to know whose diagnosis was the “correct” one or whether some patients may have remained undiagnosed or misdiagnosed.
I couldn’t find a single journalist — and there were many who covered this small retrospective study of 286 patients — who challenged this important distinction. And some journalists like Lisa Raffensperger, an editor at STAT, found the language not only misleading but also potentially self-serving for a well-known referral center like Mayo.
Here are the basic findings of the study:
The second problem is how these numbers were handled. Like UPI and many other outlets, WBAL Baltimore claimed in its headline that “1 of 5 patients with serious medical conditions are misdiagnosed.” This makes it sound like the misdiagnosis rate is 20% for all patients. But that number applies only to the very small population of referral patients who often represent unique diagnostic challenges.
Similarly, these shocking headlines by Quartz … “Get a second opinion, doctors usually aren’t right the first time” … and the San Diego Union-Tribune … “First medical diagnosis often incomplete or plain wrong, study finds” … are spectacularly wrong and will not go over well with primary care providers who diagnose problems properly most of the time, who are probably acting in the best interest of their patients when they do seek a second opinion, and who may eventually find that other doctors agree with them.
A third problem is that very few journalists took the time to highlight the significant limitations of the study.
First, the authors admit they have no idea where along the patient’s search for a diagnosis — that is, after how many doctor visits — was the choice made to get a second opinion? In other words, had these patients only had a few visits providing very little information to go on … or numerous visits and tests trying to sort out a complex case? Second, just how experienced were the referring nurse practitioners, physician assistants, or physicians asking for help? Again, this speaks to the complexity of the referred case. Were there more referrals from inexperienced providers that were easier to handle … or more coming from experienced providers needing help with very difficult cases? Third, Mayo is a highly sub-specialized academic center with unique resources, and this makes it quite difficult to compare the Mayo experience with how second opinions are handled elsewhere.
That’s a lot of caveats and this video from Fox 9 News in Minneapolis actually did quite a good job with some of them:
There is a final problem worth mentioning here: there are other studies that have looked at the value of second opinions, but none of the news stories we screened mentioned them. What becomes clear when you look at other studies on second opinions is this: just how different the first impression is from the second opinion has a lot to do with the type of problem in question (ie. cancer, digestive issues, neurological, etc.) and what opinion is being sought (operate or not? best medication? a questionable x-ray or biopsy report?). When you start dividing second opinions into sub-categories like these the results can vary wildly … some studies find the second opinion is in agreement two-thirds of the time, while other studies find the second opinion is in disagreement two-thirds of the time.
Ironically, Mayo explored this topic three years ago and published the results in their own journal, The Mayo Clinic Proceedings. What did they find?
“We found that a second opinion typically confirms the original diagnosis or treatment regimen, but that 90% of patients with poorly defined conditions remain undiagnosed.”
It is important to note that this previous Mayo look into the issue of second opinions was based on a meta-analysis of numerous studies looking at patient-initiated second opinions. So it is not directly comparable to their current study.
“The current Mayo study was really done in a very rigorous, thorough, and thoughtful way,” says Dr. Mark L. Graber, President of the Society to Improve Diagnosis in Medicine who added:
“I agree with the problems and limitations you present. It’s important to point out that this was a very select population. And we certainly need more longitudinal studies. But these are good data. And I’m glad we’re questioning how we’re doing in diagnosis and referrals. That’s the only way we’re going to improve. And when it comes to complicated problems I always say, ‘fresh eyes catch mistakes’!”
By writing about this, we’re not making any recommendations about second opinions. Let’s be clear: we don’t make health care recommendations. But we do try to help people think about the way that research findings can be – and often are – miscommunicated. And are we implying that the Mayo Clinic is not a good source of second opinions. Certainly not. Mayo is well known to provide excellent first, second, third or even fourth opinions. We simply wanted to highlight the first, second, third and fourth problems we found in the coverage of this study.