Visual analysis ofwithin-subject data: A comparison within and across certification levels

Title without typo: Visual Analysis of Within-Subject Data: A Comparison Within and Across Certification Levels The field of behavior analysis has relied on the visual inspection of data to draw conclusions about the effects of experimental manipulations on dependent variables (Baer, Wolf, & Risley, 1968; Baer, Wolf, & Risley 1987; Kazdin, 1982). Although previous researchers have consistently found low levels of interrater agreement when manipulating the statistical parameters of graphs (e.g., DeProspero & Cohen, 1979; Ottenbacher, 1986), research conducted by Kahng et al. (2010) disagrees with the finding that poor interrater agreement is common among applied behavior analysts. The purpose of the current study was to replicate and extend the findings of Kahng et al. In Experiment 1, Editors of several applied behavior analysis journals and BCBA-Ds provided a dichotomous response and numerical rating to the question whether a set of simulated ABAB graphs displayed experimental. High levels of agreement were found for Experiment 1. In Experiment 2, Masters-level BCBAs were also asked to reply to a set of simulated ABAB graphs similar to those presented in Experiment 1. Low levels of agreement were found for Experiment 2. Additionally, In Experiment 3 the effects of multiple exemplar training on participant interrater agreement were examined. Improved performance across all three participants was observed. However, an increasing trend in responding was observed across baseline, training, and generalization suggesting repeated exposure to the ABAB graphs led to improvement. Taken together, the findings of Experiment 1 and Kahng et al. suggest that consistent analysis of intrasubject data is possible under certain circumstances.