Design and Evaluation of a Semantic Indicator for Automatically Supporting Programming Learning

Y

6

Semantics; Programming; Syntax; Feedback (Response); Misconceptions; Problem Solving; Introductory Courses; Computer Science Education; Classification; Correlation; Summative Evaluation; Accuracy; Intelligent Tutoring Systems; Teaching Methods; Online Courses; Foreign Countries

How to support students in programming learning has been a great research challenge in the last years. To address this challenge, prior works have mainly focused on proposing solutions based on syntactic analysis to provide students with personalized feedback about their grammatical programming errors and misconceptions. However, syntactic analysis falls short on informing learners how they solve the programming problem, even if one key learning outcome of programming relates to the development of an individual's ability to solve a problem. In this article, we introduce an indicator to analyze beginners' code based on semantic proximity. This indicator adapts an edit distance algorithm (i.e., the Levenshtein distance) to express the proximity of the students' code with the expected solution provided by the teacher, in order to express the learners' capacity to solve the given problem. To process our indicator, we applied machine learning techniques to a dataset from an introductory programming course with a sample of 166 students. The first results are encouraging. On the one hand, the semantic indicator can be used to automatically classify source codes as semantically correct or incorrect in 58% of the cases. On the other hand, the indicator is correlated with teachers' summative evaluations of students' codes. Even if further investigations must be conducted to improve the indicator's accuracy, the results of this study make it possible to use our approach as the foundations for future research in semanticbased intelligent and awareness programming systems. [For the full proceedings, see ED599096.]

As Provided

2019

ED599192