Examining the Rule of Thumb of Not Using Multilevel Modeling: The “Design Effect Smaller Than Two” Rule.

EDUCATION research; DESIGN research; MONTE Carlo method; METHODOLOGY; DATA structures

Educational researchers commonly use the rule of thumb of “design effect smaller than 2” as the justification of not accounting for the multilevel or clustered structure in their data. The rule, however, has not yet been systematically studied in previous research. In the present study, we generated data from three different models (which differ in the location of the clustering effect). With a 3 (design effect) × 5 (cluster size) × 4 (number of clusters) Monte Carlo simulation study we found that the rule should not be applied when researchers: (a) are interested in the effects of higher-level predictors, or (b) have a cluster size less than 10. Implications of the findings and limitations of the study are discussed. [ABSTRACT FROM PUBLISHER]

Copyright of Journal of Experimental Education is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)