Standard Setting with Artificial Neural Networks: TIMSS 2015 Mathematics Case

Center for Open Access in Science. Vojvode Vlahovica 57c, Belgrade, Serbia 11000. e-mail: ojer@centerprode.com; Web site: http://centerprode.com/ojer.html

Y

10

Researchers

Standard Setting; Artificial Intelligence; Mathematics Education; Foreign Countries; Achievement Tests; Elementary Secondary Education; International Assessment; Mathematics Achievement; Mathematics Tests; Cutting Scores; Multiple Choice Tests; Grade 8

2560-5313

This study aims to demonstrate the optimal way to determine the cut-off score to be used to interpret the total scores obtained from an achievement test or scale using the Artificial Neural Networks method. To this end, the multiple-choice item responses in the Booklet-11 Mathematics subtest at the 8th grade level in the TIMSS 2015 Turkey sample dataset were used to determine the cut-off score for the achievement test. The item responses in the "Students Like Learning Mathematics Scale" in the TIMSS 2015 8th grade Mathematics Student Questionnaire were used to determine the cut-off score for the scale. The data were accessed from the TIMSS international database and the data were analyzed in MATLAB R2017b software. As a result of the study, the most appropriate cut-off score to be used for the evaluation of the total scores obtained from the TIMSS 2015 8th grade level Booklet-11 Mathematics subtest was determined as 45.5 out of 0-100 points with the Artificial Neural Network analysis method. The overall level of agreement between the cut-off score and the pass/fail classification based on 400 points, which is the lowest level of the TIMSS International Benchmark, was determined as 81%. The most appropriate cut-off score to be used for the evaluation of the scores obtained from the Students Like Learning Mathematics Scale (SLLSS) in the TIMSS 2015 8th grade student survey was determined as 19.6 out of 9-36 points. The overall level of agreement between the cut-off score and the classification of students who like/don't like learning mathematics using the criterion based on the expression given in the original scale description was found to be 83%. The results concluded that the validity of the standard-setting studies conducted with the artificial neural network method was high. As a result, researchers are recommended to use the Artificial Neural Networks method to determine the cut-off score to be used in the interpretation of the total scores obtained from the achievement test or the total scale scores obtained from the scales.

As Provided

2024

EJ1408250