The significance (?) of statistical significance

Posted By



Carl Bialik, in his “The Numbers Guy” column in the Wall Street Journal, takes a closer look at statistical significance and discussed the premise that “you can have a real effect which is nonetheless trivial in the practical sense.”

We often see this in health care news stories – in which the emphasis may be solely on statistical significance in a study without any mention of its clinical significance – or what difference the finding may really have on peoples’ lives.

You might also like


Please note, comments are no longer published through this website. All previously made comments are still archived and available for viewing through select posts.

Susan FItzgerald

April 5, 2011 at 12:32 pm

Statistical significance = if it couldn’t be chance, it must be the drug — Hah! This is why you see the FDA approving new drugs that don’t do much of anything except make people a lot of money and drive up medical costs.
Between placebo and nocebo effects, margin of error and cherry-picking facts, there isn’t much medication I would consider taking, certainly nothing on the market less than two years. Even if it doesn’t do what it’s supposed to, that drug will certainly have negative side-effects.

Joe McCarthy

April 6, 2011 at 7:46 am

I was surprised to learn that the origin of the [in]famous label “lies, damn lies, and statistics” is medicine. Gilles Frydman began a November 2008 post on about Lies, Damn Lies and Statistics: Collective Statistical Illiteracy with the following historical quote:

“Look at the dozens of operations by me this year without a death,” says the operator. His less enthusiastic neighbor thinks of the proverbial kinds of falsehoods, “lies, damned lies and statistics” and replies “reports of large number of cases subjected to operations seldom fail to beget a suspicion of unjustifiable risk”.

In “Some Surgical Sins”, John B. Robert A.M, M.D., Chairman’s Address on Surgery and Anatomy, 45th Meeting of the AMA, June 1894.

Phillipa Rispin

April 6, 2011 at 9:03 am

For an amusing take on statistical significance, see today’s XKCD:
Mouse over the bottom panel for a pop-up.