Don't Throw it Away! The Utility of Unlabeled Data in Fair Decision Making
METADATA ONLY
Author / Producer
Date
2022-06
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
METADATA ONLY
Data
Rights / License
Abstract
Decision making algorithms, in practice, are often trained on data that exhibits a variety of biases. Decision-makers often aim to take decisions based on some ground-truth target that is assumed or expected to be unbiased, i.e., equally distributed across socially salient groups. In many practical settings, the ground-truth cannot be directly observed, and instead, we have to rely on a biased proxy measure of the ground-truth, i.e., biased labels, in the data. In addition, data is often selectively labeled, i.e., even the biased labels are only observed for a small fraction of the data that received a positive decision. To overcome label and selection biases, recent work proposes to learn stochastic, exploring decision policies via i) online training of new policies at each time-step and ii) enforcing fairness as a constraint on performance. However, the existing approach uses only labeled data, disregarding a large amount of unlabeled data, and thereby suffers from high instability and variance in the learned decision policies at different times. In this paper, we propose a novel method based on a variational autoencoder for practical fair decision-making. Our method learns an unbiased data representation leveraging both labeled and unlabeled data and uses the representations to learn a policy in an online process. Using synthetic data, we empirically validate that our method converges to the optimal (fair) policy according to the ground-truth with low variance. In real-world experiments, we further show that our training approach not only offers a more stable learning process but also yields policies with higher fairness as well as utility than previous approaches.
Permanent link
Publication status
published
External links
Editor
Book title
FAccT '22: 2022 ACM Conference on Fairness, Accountability, and Transparency
Journal / series
Volume
Pages / Article No.
1421 - 1433
Publisher
Association for Computing Machinery
Event
5th ACM Conference on Fairness, Accountability, and Transparency (FAccT 2022)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Fairness; Desicion making; Label bias; Selection bias; Variational autoencoder; Fair representation