Investigating Interrater-reliability in Assessing Social Behavior of Monodelphis Domestica

Background: Reliable, consistent, and objective data is a goal all studies aim to achieve, but many struggle to obtain when subjective biases between researchers can occur. Inter-rater reliability (IRR) is a statistical measure used to quantify the degree of agreement between researchers qualitatively scoring the same phenomenon. The primary goal of this study is to enhance the methodology used to achieve optimal IRR. Using an established ethogram, our team of researchers scored the social behavior of the adult gray short-tailed opossum (Monodelphis domestica) to propose an effective method for achieving high IRR that can contribute to future research in data accuracy across multiple disciplinary fields, including clinical research and practice. Methods: A team of two raters used the Behavioral Observation Research Interactive Software (BORIS) to individually score the social behavior of each paired Monodelphis domestica subjects based on an ethogram with species-typical behaviors through observational recordings. Raters scored animal subjects’ behaviors while pausing throughout each observational video. After the first session of scoring, the team discussed misunderstandings of the ethogram during a review of our respective behavioral scorings for the first-paired animals throughout various timestamps. The team proceeded to individually score the behavior of the remaining four-paired subjects with continued pausing and without further consultations between raters on IRR. To measure IRR for every subject, Cohen’s Kappa Coefficient was used due to its strict statistical outcome that accounts for the possibility of false agreement between raters. Results: Initial scoring sessions for the first-paired subjects yielded low IRR scores (k=0.356, k=0.317). Subsequent scoring sessions for the second-paired subjects demonstrated an increase in IRR (k=0.743, k=0.730). Our final scoring sessions for the third-paired subjects resulted in the highest IRR amongst all scoring sessions (k=0.906, k=0.879). A progressive trend ...