Open access
Date
2019-11-04Type
- Working Paper
ETH Bibliography
yes
Altmetrics
Abstract
SMT solvers are at the basis of many applications, such as program verification, program synthesis, and test case generation. For all these applications to provide reliable results, SMT solvers must answer queries correctly. However, since they are complex, highly-optimized software systems, ensuring their correctness is challenging. In particular, state-of-the-art testing techniques do not reliably detect when an SMT solver is unsound.
In this paper, we present an automatic approach for generating test cases that reveal soundness errors in the implementations of string solvers, as well as potential completeness and performance issues. We synthesize input formulas that are satisfiable or unsatisfiable by construction and use this ground truth as test oracle. We automatically apply satisfiability-preserving transformations to generate increasingly-complex formulas, which allows us to detect many errors with simple inputs and, thus, facilitates debugging.
The experimental evaluation shows that our technique effectively reveals bugs in the implementation of widely-used SMT solvers and applies also to other types of solvers, such as model-counting constraint solvers. We focus on strings here, but our approach carries over to other theories and their combinations. Show more
Permanent link
https://doi.org/10.3929/ethz-b-000375243Publication status
publishedPublisher
Department of Computer Science, ETH ZurichSubject
automatic testing; soundness testing; string solvers; SMT solversOrganisational unit
03653 - Müller, Peter / Müller, Peter
Related publications and datasets
Is previous version of: https://doi.org/10.3929/ethz-b-000397450
More
Show all metadata
ETH Bibliography
yes
Altmetrics