Guest post: Where is the voice of consumers in “Top* Doctors/Best Hospitals” rankings?

Harry Demonaco photo.jpgThis is a guest post from one of our medical editors, Harold DeMonaco, director of the Innovation Support Center at the Massachusetts General Hospital.


There are approximately 800,000 practicing physicians in the United States and not all of them can be the best. Finding the best doctor is a hit or miss process and even when you think you have found one, you really have no idea if your impression is valid. US News and World Report recently released a searchable database of the “Top Doctors” in the United States. Presumably consumers now have the ability to find the best physicians to care for them or family members. But, is this more than simply a popularity contest and is care better when provided by the “top physicians” ?

How are these lists put together and is their any evidence that care is in fact better? In his 2004 book, The Wisdom of Crowds, James Surowiecki, suggested that the collective wisdom of groups results in decisions that are often better that those made by any single individual. He postulates that four criteria appear to be important: diversity of opinion, independence, decentralization and aggregation. Let’s examine the Top Doctors methods and see if they come close to Surowiecki’s requirements.

The Top Doctors list is based on nominations from physician peers either practicing locally or more recently, anywhere in the US. Decentralization requires the nominator to have personal information about the nominee. A primary care physician who nominates a cardiologist to whom he has referred patients is likely to have some understanding of that individual’s capabilities and is likely to have sought out the opinion of others as well. The nomination process is presumably an independent one with a very small likelihood of concerted efforts to “stuff the ballot box.” And US News and World Report along with Castle Connolly Medical provide the aggregation of data. So, based on the opinion of
the experts, the list should provide consumers with good information. But there is a big piece missing; diversity or the voice of the consumer.

Why is it that medicine is one of the few domains where the views of the insiders is held in such high esteem as to almost completely ignore the voice of the consumer? Where is the voice of the consumer and what about the quality of care? One could argue that the technical nature of surgery is such that it would be impossible for the patient or family to rate the skill of the surgeon. The success or failure of surgery is a complicated blending of patient factors, skill of the surgeon and other caregivers and unfortunately luck. There have been attempts at the state level to provide consumers with information about surgical outcomes at hospitals with the usual issues related to severity of illness and small sample size for individual physicians. Perhaps then peer evaluation is the right method for technical specialties such as cardiac surgery when combined with information about outcomes. But how should expertise of a cardiologist, endocrinologist or other “non-technical medical specialties” be evaluated? Should it be purely based on peer evaluation/nomination? Or, should the patient review and outcome be included in the measures?

A number of websites are now available that provide for patient reviews of physicians, including those providing primary and specialty care. Although a step in the right direction, they suffer from relatively small numbers of reviews for any individual doctor and do not consider measurable outcomes such as quality of life. Given the small numbers, a single bad review from a disgruntled patient can dramatically alter the doctors overall rating. But the same could be true for plumbers, electricians or any other service provider. Yes, I actually do believe that physicians are service providers.

The Centers for Medicare and Medicaid are now rating hospitals based on the quality of care provided to patients for admitted for a small number of diagnoses and has a pilot program for patient review of inpatient care. But there is no such program for ambulatory care.

In the final analysis, being listed as a top physician or being associated with a top tier hospital (such as the Massachusetts General Hospital) is satisfying in that you or your institution has a good reputation. While a good reputation may relate to the quality of care provided by a doctor of hospital, it is no guarantee especially for an individual patient. The US News and World Report provides the consumer with a listing of those doctors and hospitals with the best reputations. At the moment that is about the best we can hope for.

* I changed the original headline from reading “Best Doctors” to “Top Doctors” because the US News ranking actually was entitled “Top Doctors.” There’s also a trademarked company called “Best Doctors.”

ADDENDUM: See the blog post on the hospital rankings by Paul Levy, former President and CEO of Beth Israel Deconess Medical Center in Boston.

Then see Maggie Mahar’s spin-off, “Hospital CEOs Reveal Their Top Priorities.”

You might also like


Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Skeptical Scalpel

August 4, 2011 at 8:29 pm

I am not sure how one could incorporate consumer input into the doctor rating equation. There are numerous articles and blog posts about how anonymous ratings by a few disgruntled patients can damage the reputation of a doctor.
There are many other shortcomings of physician ( and hospital ( ratings.

Gary Schwitzer

August 5, 2011 at 2:27 pm

Thanks for the note.
And yes, I did get your followup note with the link to this story that raises questions about patients’ perception of quality.
It raises a more fundamental question in my mind about how we’ve not done a good enough job educating consumers about what quality measures they should be concerned about in health care.
Yet we have health insurance companies peddling “consumer driven health care plans” that are supposed to put consumers in the driver’s seat of a system they don’t know how to drive!

Harold DeMonaco

August 8, 2011 at 10:14 am

Thank you both for your comments. This is an interesting area with no clear answers at the moment. One might say it is an “opinion rich and data poor environment.”
The BMJ published two opinion pieces a couple of years ago that generated virtually no response from the media. The question posed was, “Will doctor rating sites improve the quality of care?”
Here is the “Yes” response:
And here is the “No” response:
But physicians aren’t the only professionals who are being subjected to consumer reviews, and the lawyers don’t seem to like it either:
“So from my perspective, most attorneys instinctively have a negative reaction towards the idea of being reviewed by their customers. We’re not used to it as attorneys. We’re not used to actually having people talking about the quality of our services in an open, public forum. So my general observation is that when lawyers start to realize that their customers are going to start reviewing them, they instinctively recoil in horror. And I think we’re going to have to work through that. I think that’s part of our future. It’s the reality of today. And we’re going to have to have that catharsis.”
In fact, I would argue that no one really likes to be placed in a position of being judged. Physicians are no different than anyone else in this regard. Healthcare delivery is changing and is doing so quickly as a result of cost shifting from insurers to patients, the widespread availability of the internet and the increasing sophistication of the patient population. If other economic domains are any indication, the rating of physicians by their patients is likely to increase. Whether these ratings have any bearing on the quality of care delivered is still an open question.