This NCEE Technical Methods report examines how to address the problem of missing data in the analysis of data in Randomized Controlled Trials (RCTs) of educational interventions, with a particular focus on the common educational situation in which groups of students such as entire classrooms or schools are randomized. Missing outcome data are a problem for two reasons: (1) the loss of sample members can reduce the power to detect statistically significant differences, and (2) the introduction of non-random differences between the treatment and control groups can lead to bias in the estimate of the intervention's effect. The report reviews a selection of methods available for addressing missing data, and then examines their relative performance using extensive simulations that varied a typical educational RCT on three dimensions: (1) the amount of missing data; (2) the level at which data are missing--at the level of whole schools (the assumed unit of randomization) or for students within schools; and, (3) the underlying missing data mechanism. The performance of the different methods is assessed in terms of bias in both the estimated impact and the associated standard error. Appendices include: (1) Missing Data Bias as a Form of Omitted Variable Bias; (2) Resources for Using Multiple Imputation; (3) Specifications for Missing Data Simulations; (4) Full Set of Simulation Results; and (5) Standards for Judging the Magnitude of the Bias for Different Missing Data Methods. (Contains 12 tables, 4 exhibits and 88 footnotes.)
Processing Request
No Comments.