The New Statistics: Estimation for better research

Understanding the New Statistics by Geoff CummingThe book

Cumming, G. (2012). Understanding The New Statistics: Effect Sizes, Confidence Intervals, and Meta-Analysis. New York: Routledge

  • Explains estimation, with many examples.
  • Designed for any discipline that uses statistical significance testing.
  • For advanced undergraduate and graduate students, and researchers.
  • Comes with free ESCI software.
  • May be the first evidence-based statistics textbook.
  • Assumes only prior completion of any intro statistics course.
  • See the dance of the confidence intervals, and many other intriguing things.

The main message of the book is summarised in this short magazine article, and in this radio talk (transcript and podcast). You can download the book's Preface, Contents, and a sample chapter. There is further information about the book available from the publisher, and a Users’ Guide [PDF 114KB] that explains how to get the best out of the book.

The author

My main current research is in the area of statistical cognition, which is the study of how people understand - or misunderstand - statistical concepts, and various different ways to present the results of statistical analyses.  I advocate the evidence-based practice of statistics, meaning that our selection of a statistical technique should be supported by cognitive evidence that people understand it well. 

I am especially interested in replication, which is the topic of Chapter 5 in the book.  One of many reasons that CIs are better than p values is that CIs generally give quite good information about what is likely to happen on replication of an experiment, whereas a p value gives almost no information about replication.  The dance of the p values illustrates how p values vary enormously with replication, thus indicating how terribly uninformative they are.

Reviews

In this book Cumming does exactly what's needed - he clearly explains significance-testing and confidence-interval techniques, and in the process leaves no doubt as to which should be preferred. I hope that this book will be read by anyone who plans to do research of any kind. Geoffrey Loftus, University of Washington, Seattle, USA

It is about time that the ‘new statistics’ get the place they deserve in statistics education, to further true understanding instead of following statistical rituals. Geoff Cumming’s excellent book is a milestone towards reaching that goal. Peter Sedlmeier, Chemnitz University of Technology, Germany

Geoff Cumming is a preeminent author in statistics reform and his new book is an important work in this area. Written in a clear, accessible way, Cumming covers essential topics in data analysis, including effect size estimation, confidence intervals, and meta-analysis. Readers in the social and behavioral sciences will learn new ways to look at their own results and avoid common traps of traditional significance testing. Rex Kline, Concordia University, Canada

Cumming makes strong arguments for confidence intervals and meta-analysis as better ways of representing and thinking about research. His graphical, interactive approach makes the ‘new statistics’ highly accessible and remarkably impactful on our statistical understanding. Robin K. Henson, University of North Texas, USA

See also a recent review by Eoin O'Connell in Significance: Statistics Making Sense  

The software

ESCI (Exploratory Software for Confidence Intervals)

  • Pronounced 'ESS-key'
  • Developed to accompany the book.
  • Runs under Microsoft Excel.
  • Easy and free to download.

This Readme [PDF 254KB] has the latest information about ESCI, and hints for getting going. [updated 4 July 2011]

To download and use ESCI

  • Choose one of the three versions below.
  • Download ESCI as a zipped folder.
  • Unzip and save the contents in a single folder on your hard drive.
  • Open an ESCI module—a regular Excel workbook—in Excel.
  • Make sure macros are enabled.
  • Adjust zoom factor if necessary, so the display fits your screen.
  • Enjoy the mean heap, dance of the p values, and other neat simulations.

ESCI for Windows (Excel 2007 or 2010) [ZIP 7MB] (updated 4 July 2011)
ESCI modules are .xlsm macro-enabled files.

ESCI for Windows (Excel 2003) [ZIP 3.9MB] (updated 4 July 2011)
ESCI modules are .xls files.
This version runs fastest and has best interactivity. Choose this if you have access to Excel 2003.

ESCI for Macintosh (Excel 2011) [ZIP 5.7MB] (updated 4 July 2011)
ESCI modules are .xlsm macro-enabled files.

Earlier ESCI

ESCI modules released during 2001 to 2010 were designed to accompany various journal articles. To a large extent, but not entirely, those earlier modules are now superseded by ESCI that accompanies the book. The earlier modules and information about the journal articles are still available.

The video

View the dance of the p values video (see Chapter 5 in the book).