Gerd Kortemeyer


Last Name

Kortemeyer

First Name

Gerd

ORCID

0000-0001-6643-9428

Organisational unit

02263 - AI for Education / AI for Education

Filters

2019 - 201922020 - 202520
Reset filters

Search Results

Publications 1 - 10 of 22
  • Grading assistance for a handwritten thermodynamics exam using artificial intelligence: An exploratory study
    Item type: Journal Article
    Kortemeyer, Gerd; Nöhl, Julian; Onishchuk, Daria (2024)
    Physical Review Physics Education Research
    [This paper is part of the Focused Collection in Artificial Intelligence Tools in Physics Teaching and Physics Education Research.] Using a high-stakes thermodynamics exam as the sample (252 students, four multipart problems), we investigate the viability of four workflows for AI-assisted grading of handwritten student solutions. We find that the greatest challenge lies in converting handwritten answers into a machinereadable format. The granularity of grading criteria also influences grading performance: employing a finegrained rubric for entire problems often leads to errors and grading failures, as the model appears to be unable to keep track of scores for more than a handful of rubric items, while grading problems in parts is more reliable but tends to miss nuances. We also found that grading hand-drawn graphics, such as process diagrams, is less reliable than mathematical derivations due to the difficulty in differentiating essential details from extraneous information. Although the system is precise in identifying exams that meet passing criteria, exams with failing grades still require human grading. We conclude with recommendations to overcome some of the encountered challenges.
  • Toward AI grading of student problem solutions in introductory physics: A feasibility study
    Item type: Journal Article
    Kortemeyer, Gerd (2023)
    Physical Review Physics Education Research
    Solving problems is crucial for learning physics, and not only final solutions but also their derivations are important. Grading these derivations is labor intensive, as it generally involves human evaluation of handwritten work. AI tools have not been an alternative, since even for short answers, they needed specific training for each problem or set of problems. Extensively pretrained AI systems offer a potentially universal grading solution without this specific training. This feasibility study explores an AI-assisted workflow to grade handwritten physics derivations using MathPix and GPT-4. We were able to successfully scan handwritten solution paths and achieved an R-squared of 0.84 compared to human graders on a synthetic dataset. The proposed workflow appears promising for formative feedback, but for final evaluations, it would best be used to assist human graders.
  • Attending lectures in person, hybrid or online at a technical university: how do students choose after the pandemic, and what about the outcome?
    Item type: Journal Article
    Kortemeyer, Gerd; Dittmann-Domenichini, Nora; Merki, Claudia (2025)
    Discover Education
    In the aftermath of the COVID-19 pandemic, students have more choice of how to attend courses than ever before; for a large number of courses at a technical university, they are still able to watch the lectures live online or in recorded format later. We found that interactivity may bring back students into the classroom almost as effectively as withholding alternatives. However, we also found that these attendance modes by themselves have little influence on exam grades. Instead, exam outcomes are more strongly associated with students attitudes and interest. We also found indicators that focussing on live lectures during the running semester (on-campus or online) may be associated with better exam outcomes than intensive preparation phases before exams.
  • Performance of the pre-trained large language model GPT-4 on automated short answer grading
    Item type: Journal Article
    Kortemeyer, Gerd (2024)
    Discover Artificial Intelligence
    Automated Short Answer Grading (ASAG) has been an active area of machine-learning research for over a decade. It promises to let educators grade and give feedback on free-form responses in large-enrollment courses in spite of limited availability of human graders. Over the years, carefully trained models have achieved increasingly higher levels of performance. More recently, pre-trained Large Language Models (LLMs) emerged as a commodity, and an intriguing question is how a general-purpose tool without additional training compares to specialized models. We studied the performance of GPT-4 on the standard benchmark 2-way and 3-way datasets SciEntsBank and Beetle, where in addition to the standard task of grading the alignment of the student answer with a reference answer, we also investigated withholding the reference answer. We found that overall, the performance of the pre-trained general-purpose GPT-4 LLM is comparable to hand-engineered models, but worse than pre-trained LLMs that had specialized training.
  • Could an artificial-intelligence agent pass an introductory physics course?
    Item type: Journal Article
    Kortemeyer, Gerd (2023)
    Physical Review Physics Education Research
    Massive pretrained language models have garnered attention and controversy due to their ability to generate humanlike responses: Attention due to their frequent indistinguishability from human-generated phraseology and narratives and controversy due to the fact that their convincingly presented arguments and facts are frequently simply false. Just how humanlike are these responses when it comes to dialogues about physics, in particular about the standard content of introductory physics courses? This case study explores that question by having ChatGPT, the preeminent language model in 2023, work through representative assessment content of an actual calculus-based physics course and grading the responses in the same way human responses would be graded. As it turns out, ChatGPT would narrowly pass this course while exhibiting many of the preconceptions and errors of a beginning learner. A discussion of possible consequences for teaching, testing, and physics education research is provided as a possible starter for more detailed studies and curricular efforts in the future.
  • Quick-and-Dirty Item Response Theory
    Item type: Journal Article
    Kortemeyer, Gerd (2019)
    The Physics Teacher
  • Assessing confidence in AI-Assisted grading of physics exams through psychometrics: An exploratory study
    Item type: Journal Article
    Kortemeyer, Gerd; Nöhl, Julian (2025)
    Physical Review Physics Education Research
    This study explores the use of artificial intelligence in grading high-stakes physics exams, emphasizing the application of psychometric methods, particularly item response theory, to evaluate the reliability of AI-Assisted grading. We examine how grading rubrics can be iteratively refined and how threshold parameters can determine when AI-generated grades are reliable versus when human intervention is necessary. By adjusting thresholds for correctness measures and uncertainty, AI can grade with high precision, significantly reducing grading workloads while maintaining accuracy. Our findings show that AI can achieve a coefficient of determination of R² ≈ 0.91 when handling half of the grading load, and R² ≈ 0.96 for one-fifth of the load. These results demonstrate AI's potential to assist in grading large-scale assessments, reducing both human effort and associated costs. However, the study underscores the importance of human oversight in cases of uncertainty or complex problem solving, ensuring the integrity of the grading process.
  • Writing Virtual Reality Teaching Resources
    Item type: Journal Article
    Kortemeyer, Gerd (2023)
    The Physics Teacher
    Simulations can provide opportunities for engaged exploration in physics teaching and learning. Beyond the two-dimensional world of screen-based simulations, abstract concepts like vectors (for example, of electric fields) can frequently be visualized better in a three-dimensional virtual reality (VR) environment. These visualizations can be immersive, where the user is able to walk around, look around, and intuitively interact with objects in virtual space. Finally, it has been shown that this bodily acting out of physics scenarios ("embodiment") can lead to even better learning results of particularly basic mechanics concepts.
  • AIt: Intention-preserving Automatic Alt-text Generation for Educational Content
    Item type: Other Conference Item
    Chatain, Julia; Kortemeyer, Gerd; Fender, Andreas (2024)
    EARLI SIG 6 & SIG 7 Book of Abstracts. Instructional Design and Technology Enhanced Learning: Current States and Future Perspectives
  • Virtual-Reality graph visualization based on Fruchterman-Reingold using Unity and SteamVR
    Item type: Journal Article
    Kortemeyer, Gerd (2022)
    Information Visualization
    The paper describes a method for the immersive, dynamic visualization of undirected, weighted graphs. Using the Fruchterman-Reingold method, force-directed graphs are drawn in a Virtual-Reality system. The user can walk through the data, as well as move vertices using controllers, while the network display rearranges in realtime according to Newtonian physics. In addition to the physics behind the employed method, the paper explains the most pertinent computational mechanisms for its implementation, using Unity, SteamVR, and a Virtual-Reality system such as HTC Vive (the source package is made available for download). It was found that the method allows for intuitive exploration of graphs with on the order of 10² vertices, and that dynamic extrusion of vertices and realtime readjustment of the network structure allows for developing an intuitive understanding of the relationship of a vertex to the remainder of the network. Based on this observation, possible future developments are suggested.
Publications 1 - 10 of 22