The Informational versus Dynamical Role of Feedback in Tool-Mediated Learning and Assessment

Abstract

Where do we need to draw the boundary of the cognitive system we investigate, when we intend to assess student competencies: The isolated mind, or the situated student interacting with complex disciplinary tools? If we choose to assess the latter, what are the critical features of the feedback between the disciplinary tool and the student that maintain the student-tool cognitive system intact? In this study, we investigated the role of compiler feedback on student learning and assessment performances in the tool-mediated practice of programming. A previous study indicated that the need to comply with a compiler’s constraints, i.e. the need to write syntactically correct code, may be an important facilitator of conceptual learning through the dynamical feedback it generates, but at the same time may also inhibit immediate performance in conceptual tasks. We implemented an experimental 2x2 between-subjects design according to the prior study, with a learning experiment followed by an assessment experiment, each with a tool-mediated - compiler activated - versus tool-dissociated - compiler deactivated - experimental condition. We implemented a subtly varied operationalization of the tool-dissociated experimental conditions: Instead of a full deactivated compiler with no feedback, students received compiler feedback, but this feedback was only minimally informative. This feedback would introduce the need to comply with constraints to the tool-dissociated conditions, while still denying informative feedback for learning. Hence, tool-mediated learning should remain superior to tool-dissociated learning, while conversely tool-dissociated assessment should no longer facilitate performances as compared to the tool-mediated experimental conditions. Our study results confirmed precisely these assumptions. This confirms a complex relationship between compliance with constraints, feedback, learning, and immediate conceptual performance. While compliance with constraints may compromise immediate conceptual performance, it appears at the same time an important driver of feedback conducive to conceptual learning.