Pitfalls of Data Analysis. ERIC/AE Digest.

N

4

Office of Educational Research and Improvement (ED), Washington, DC.

Causal Models; Comparative Analysis; Data Analysis; Error of Measurement; Research Design; Research Methodology; Research Problems; Sampling; Statistical Bias; Statistical Significance; Statistics; Test Results; Validity

Abuses and misuses of statistics are frequent. This digest attempts to warn against these in three broad classes of pitfalls: sources of bias, errors of methodology, and misinterpretation of results. Sources of bias are conditions or circumstances that affect the external validity of statistical results. In order for a researcher to make legitimate conclusions about the specified population, two characteristics must be present in the sample: representative sampling and valid statistical assumptions. There are number of ways that statistical techniques can be applied incorrectly, and these errors in methodology can lead to invalid or inaccurate results. Three of the most common are designing experiments with insufficient statistical power, ignoring measurement error, and performing multiple comparisons. There are also a number of problems that can arise in the context of substantive interpretation. These problems usually involve determining the significance of findings, avoiding confusion between precision and accuracy, and unraveling the causal relationships among variables. Paying attention to these sources of difficulty can allow the presentation of information in the clearest way possible. (Contains six references.) (SLD)

1997

ED410231