ConStat: Performance-Based Contamination Detection in Large Language Models
OPEN ACCESS
Loading...
Author / Producer
Date
2024
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Public benchmarks play an essential role in the evaluation of large language models. However, data contamination can lead to inflated performance, rendering them unreliable for model comparison. It is therefore crucial to detect contamination and estimate its impact on measured performance. Unfortunately, existing detection methods can be easily evaded and fail to quantify contamination. To overcome these limitations, we propose a novel definition of contamination as artificially inflated and non-generalizing benchmark performance instead of the inclusion of benchmark samples in the training data. This perspective enables us to detect any model with inflated performance, i.e., performance that does not generalize to rephrased samples, synthetic samples from the same distribution, or different benchmarks for the same task. Based on this insight, we develop ConStat, a statistical method that reliably detects and quantifies contamination by comparing performance between a primary and reference benchmark relative to a set of reference models. We demonstrate the effectiveness of ConStat in an extensive evaluation of diverse model architectures, benchmarks, and contamination scenarios and find high levels of contamination in multiple popular models including Mistral, Llama, Yi, and the top-3 Open LLM Leaderboard models.
Permanent link
Publication status
published
Book title
Advances in Neural Information Processing Systems 37
Journal / series
Volume
Pages / Article No.
92420 - 92464
Publisher
Curran
Event
38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
large language models; model evaluation; contamination detection
Organisational unit
03948 - Vechev, Martin / Vechev, Martin
Notes
Poster presentation on December 11, 2024.
Funding
101070617/22.00164 - European Lighthouse on Secure and Safe AI (SBFI)
207967 - Elegant and Efficient Quantum Programming: Compilation, Optimization, Analysis (SNF)
207967 - Elegant and Efficient Quantum Programming: Compilation, Optimization, Analysis (SNF)
Related publications and datasets
Is new version of: https://openreview.net/forum?id=ALISPmDPCq