Statistical controversies in clinical research: statistical significance—too much of a good thing …

open access

problems with statistical significance

Fisher's caveats notwithstanding, P values have become an obsession in clinical research, with the magic ‘P < 0.05’ seemingly dominating all other considerations, at least in the regulatory context of granting new therapeutics market authorization. We argue here that P values per se are not the problem, but rather an excessive reliance on P values to dichotomize reality between ‘no treatment effect’ and ‘some treatment effect’. The P value is the probability of observing data as extreme as the data observed in the absence of any real treatment effect. The P value is often misunderstood or abused, in particular to make exaggerated claims about an effect of interest [5]....

 statistical significance going forward
In randomized clinical trials, prespecified criteria to gauge statistical significance should not be so broad as to be fuzzy, nor so strict as to be silly. Going forward, the proper use of statistical significance may well be just as Fisher intended it, as a pragmatic guide to inform evaluation rather than as a strict binary boundary that separates real treatment effects from lack thereof.....