School of Psychological Science

Research abstracts – Professor Geoff Cumming

Professor Geoff Cumming

Cumming, G. (in press). p values versus confidence intervals as warrants for conclusions that results will replicate. In B. Thompson & R. Subotnik (Eds.) Research methodologies for conducting research on giftedness. Washington, DC: APA Books.

Abstract: Replication is fundamental to science, so statistical analysis should give information about replication. In this chapter the author investigates what null hypothesis significance testing (NHST) and p values tell us about replication. Giftedness researchers frequently use the Pearson correlation r, and so the discussion focuses on r. The arguments for reform of statistical practices by shifting emphasis from NHST to CIs and other better techniques are briefly reviewed. A small survey is reported of statistical practices in recent giftedness research, which suggests this field in some ways lags other areas of psychology in adoption of CIs and other reform techniques. The main discussion illustrates the remarkably large variation of p values with replication, and the problems this causes: Any p value could easily have been very different, simply because of sampling variability, and anything other than very small p values provide very little information indeed. By contrast, CIs provide quantitative information about replication. They also give fuller answers to typical research questions asked by giftedness researchers. Giftedness research could be greatly enhanced scientifically by a shift from reliance on p values to CIs, meta-analysis, meta-analytic thinking, and other better methods.

Keywords: Replication, p values, confidence intervals, giftedness, meta-analytic thinking, statistical reform

Wolfe, R., & Cumming, G. (2004). Communicating the uncertainty in research findings: Confidence intervals. Journal of Science and Medicine in Sport, 7, 138-143. (Subject of invited editorial: Marshall, S. (2004). Testing with confidence: The use (and misuse) of confidence intervals in biomedical research. Journal of Science and Medicine in Sport, 7, 135-137.)

Abstract: Confidence intervals (CIs) are an effective means of quantifying the uncertainty inherent in study results. Alongside considerations of possible sources of bias in the study design, CIs describe the extent to which study results are applicable in general, beyond the participants involved in the study. We discuss CI use and interpretation for some commonly encountered study designs. We highlight a close relation between CIs and p-values, such that presentation of 95% CIs can make it redundant to state whether a corresponding p-value is less than 0.05. We encourage researchers to use CIs to present their research findings, rather than relying on p-values alone.

There is a free ESCI download (ESCI JSMS) to go with the above JSMS article, available from the ESCI site.