WRONG: a book about evaluating evidence

Science and business journalist David Freedman’s book, “Wrong: Why Experts Keep Failing Us – And How to Know When Not To Trust Them,” hits on many health/medical/science journalism themes – although that is not the main focus of his book, which also addresses finance wizards, relationship gurus, celebrity CEOs, high-powered consultants and more.

The book’s introduction leads with the story of Dr. John Ioannidis – now at Stanford – who says:

“Amazingly, most medical treatment simply isn’t backed up by good, quantitative evidence.”

But Freedman also writes about:

How surrogate or proxy measurements or markers tend to lead researchers astray.

(We’ve written about this before on this site, under these headlines:

Surrogate outcomes, progression-free survival: important issues for journalists & consumers to grasp;

Do I need to start a Surrogate Watch blog?;

Beware surrogate markers: colon polyps, arterial plaque, cholesterol, tumor markers.)

Tossing out inconvenient data, fabrication/falsification of medical research data (We’ve written about this as well.)
Problems with even the “gold standard” of randomized clinical trials (We’ve touched on one aspect of this issue.)
Medical journals’ publication bias for positive findings

“One group of studies analyzed by Kay Dickersin, the publication-bias expert at Johns Hopkins, and her folleagues found that for every negative study rejected by a journal, there was an average of about ten negative studies that weren’t submitted for publication.”

We hear a lot about “the wisdom of the crowds.” He has a chapter, “The Idiocy of Crowds.”

Freedman describes as “the certainty principle” what it is that many of us want in advice:

“We’re heavily biased to advice that is simple, clear-cut, actionable, universal and palatable.”

That description seems to match much of the public dialogue on issues such as screening tests. Journalists often do a poor job handling uncertainty in situations where the answers are not clear-cut – such as screening test discussions. And the public dialogue suffers as a result.

I recommend “Wrong” and love some of the key quotes from it: