The worst-kept secret in journalism circles recently was that the New York Times was planning an article critical of the Dartmouth Atlas. That article was published online last night.
Among the main points in the article:
“The mistaken belief that the Dartmouth research proves that cheaper care is better care is widespread.”
“the atlas’s hospital rankings do not take into account care that prolongs or improves lives.”
“Even Dartmouth’s claims about which hospitals and regions are cheapest may be suspect.”
“failing to make basic data adjustments undermines the geographic variations the atlas purports to show.”
The Times has also published the correspondence it had with the Dartmouth team about methodology questions.
The Dartmouth team challenges each of these criticisms in a pdf statement online. The team says the Times made at least five factual errors and several misrepresentations. They write: “What is truly unfortunate is that the Times missed an opportunity to help educate the American public about what our research actually shows — or about the breadth of agreement about what our findings mean for health care reform.”
Oddly, the Times called the Atlas researchers “a once obscure research group at Dartmouth College.” I guess if you consider “once obscure” as 20 years ago you might be right. Putting that phrase in the opening sentence opened the discussion on a questionable note.
The Times is correct in stating that “The debate about the Dartmouth work is important.”
But as Merrill Goozner wrote on his blog today,
“…as is often the case in journalism, the attempt to reduce complex realities into a single-factor analysis that can be summarized in a headline or a single “why this story is important” paragraph can leave a mistaken impression. Regional variation in Medicare spending is one indicator of gross overutilization. Something is happening when a hospital in McAllen, Texas does twice as many knee implants per Medicare beneficiary as a hospital in Rochester, Minnesota.
But that by itself tells us nothing about why that overutilization occurs. Greedy doctors or hospitals working in a fee-for-service system are part of the problem. Cost shifting due to high levels of uninsurance and illness severity also may account for some or even much of those differences. And, as the Times report points out, quality differentials have to be taken into account.
The pushback by Dartmouth defenders has already begun. Columbia University professor Andrew Gelman on his blog suggests the reporters need a lesson in statistics. My complaint with today’s story comes from an entirely different angle. Higher quality care lowers costs, it doesn’t raise costs.”
Thoughtful people should read a lot about the criticism and the countering comments from Dartmouth. This is not something that can be absorbed in a headline, in a nut graf, or in a quickie talk show segment.
Disclosure: I worked at Dartmouth throughout the 90s, right down the hall from many of these Atlas researchers. I consider many of them my friends. My work is currently supported by the Foundation for Informed Medical Decision Making, one of whose co-founders was Dr. Jack Wennberg of Dartmouth, whose work was the basis for the Atlas project.
Comments
Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.
pnschmidt
June 3, 2010 at 11:02 amThe key to understanding the atlas comes from looking at the process for improving care implemented by another member of the Dartmouth team, Gerry O’Connor and his work in registries. In, for example, cystic fibrosis, the registry showed the discrepancies in outcomes. It did not show the reason for the discrepancies. (This is well covered in Atul Gawande’s book “Better” or his New Yorker article, “The Bell Curve.”) Knowing that discrepancies exist does not explain how to address them, but one needs this knowledge in order to generate the testable hypotheses that will address them. Dartmouth’s O’Connor, Wennberg, and others like Eugene Nelson have each led targeted projects to synthesize the outcomes data (like what’s in their atlas) with detailed analyses of the processes that led to those outcomes in order to improve care. Given this, the “atlas” is aptly named: a geographic atlas neither tells me where to go nor how to get there, it just informs my choices.
Rose Hoban, RN, MPH
June 3, 2010 at 9:24 pmAnother one of the implications of the article is that the Dartmouth data might be lacking because as the authors themselves ‘acknowledged in interviews that in fact it mainly shows the varying costs of care in the government’s Medicare program.’
Um. Maybe the Times reporters might have mentioned that there’s no other dataset out there where you can compare similar people across the country the way you can with Medicare beneficiaries. That’s because hospitals and insurers call their data ‘proprietary.’
Boy. Would I love to see Dartmouth Atlas methods applied across all age groups, income groups and all kinds of insurance statuses.
Our Comments Policy
But before leaving a comment, please review these notes about our policy.
You are responsible for any comments you leave on this site.
This site is primarily a forum for discussion about the quality (or lack thereof) in journalism or other media messages (advertising, marketing, public relations, medical journals, etc.) It is not intended to be a forum for definitive discussions about medicine or science.
We will delete comments that include personal attacks, unfounded allegations, unverified claims, product pitches, profanity or any from anyone who does not list a full name and a functioning email address. We will also end any thread of repetitive comments. We don”t give medical advice so we won”t respond to questions asking for it.
We don”t have sufficient staffing to contact each commenter who left such a message. If you have a question about why your comment was edited or removed, you can email us at feedback@healthnewsreview.org.
There has been a recent burst of attention to troubles with many comments left on science and science news/communication websites. Read “Online science comments: trolls, trash and treasure.”
The authors of the Retraction Watch comments policy urge commenters:
We”re also concerned about anonymous comments. We ask that all commenters leave their full name and provide an actual email address in case we feel we need to contact them. We may delete any comment left by someone who does not leave their name and a legitimate email address.
And, as noted, product pitches of any sort – pushing treatments, tests, products, procedures, physicians, medical centers, books, websites – are likely to be deleted. We don”t accept advertising on this site and are not going to give it away free.
The ability to leave comments expires after a certain period of time. So you may find that you’re unable to leave a comment on an article that is more than a few months old.
You might also like