SPEAR: Exact Gradient Inversion of Batches in Federated Learning
OPEN ACCESS
Loading...
Date
2024
Publication Type
Conference Paper
ETH Bibliography
yes
Citations
Altmetric
OPEN ACCESS
Data
Rights / License
Abstract
Federated learning is a framework for collaborative machine learning where clients only share gradient updates and not their private data with a server. However, it was recently shown that gradient inversion attacks can reconstruct this data from the shared gradients. In the important honest-but-curious setting, existing attacks enable exact reconstruction only for batch size of $b=1$, with larger batches permitting only approximate reconstruction. In this work, we propose SPEAR, the first algorithm reconstructing whole batches with $b >1$ exactly. SPEAR combines insights into the explicit low-rank structure of gradients with a sampling-based algorithm. Crucially, we leverage ReLU-induced gradient sparsity to precisely filter out large numbers of incorrect samples, making a final reconstruction step tractable. We provide an efficient GPU implementation for fully connected networks and show that it recovers high-dimensional ImageNet inputs in batches of up to $b \lesssim 25$ exactly while scaling to large networks. Finally, we show theoretically that much larger batches can be reconstructed with high probability given exponential time.
Permanent link
Publication status
published
Book title
Advances in Neural Information Processing Systems 37
Journal / series
Volume
Pages / Article No.
106768 - 106799
Publisher
Curran
Event
38th Annual Conference on Neural Information Processing Systems (NeurIPS 2024)
Edition / version
Methods
Software
Geographic location
Date collected
Date created
Subject
Organisational unit
03948 - Vechev, Martin / Vechev, Martin
Notes
Poster presentation on December 10, 2024.
Funding
101070617/22.00164 - European Lighthouse on Secure and Safe AI (SBFI)
Related publications and datasets
Is new version of: 10.48550/arXiv.2403.03945Is new version of: https://openreview.net/forum?id=lPDxPVS6ix