What Can Be Learned from Empirical Evaluations of Non-Experimental Methods?

Y

33

Institute of Education Sciences (ED)
National Science Foundation (NSF)

R305B140026
2015028500

Evaluation Methods; Program Evaluation; Program Effectiveness; Comparative Analysis; History; Randomized Controlled Trials; Innovation; Research Design

10.1177/0193841X18776870

Given the widespread use of non-experimental (NE) methods for assessing program impacts, there is a strong need to know whether NE approaches yield causally valid results in field settings. In within-study comparison (WSC) designs, the researcher compares treatment effects from an NE with those obtained from a randomized experiment that shares the same target population. The goal is to assess whether the stringent assumptions required for NE methods are likely to be met in practice. This essay provides an overview of recent efforts to empirically evaluate NE method performance in field settings. We discuss a brief history of the design, highlighting methodological innovations along the way, and suggest future areas for consideration in the design, implementation, and analysis of WSCs. [This is the in press version of an article published in "Evaluation Review."]

As Provided

Yes

2018

ED586414