Assessing Live Programming for Program Comprehension

Abstract

Previous research on the effects of live program composition in computer science education has shown mixed results; while live programming is well-received by students and can improve the program composition process in some contexts, the resulting programs may be hard to understand, potentially making the paradigm unfeasible for collaborative and general-purpose programming. In this paper, we explore to what extent programs created in Algot, a live programming language, can be understood by tertiary-level students. We conducted an experimental, within-subjects study (n=41) measuring how well students at this level could comprehend programs composed in Algot and Python. We asked our participants to explain the programs and answer questions on them related to tracing, reverse tracing, conceptual extrapolations, and (optionally) time complexity. Despite the participants' lack of familiarity with Algot, students performed better after viewing most programs in Algot than Python, but primarily for problems involving trees and matrices. Our results contribute to the body of research on live programming in computer science (CS) education and complement recent research on the benefits of Algot for program composition, suggesting that Algot can be useful as a more general learning resource in CS tertiary education.