Metadata only
Date
2021-06Type
- Conference Paper
Abstract
We present DeepT, a novel method for certifying Transformer networks based on abstract interpretation. The key idea behind DeepT is our new Multi-norm Zonotope abstract domain, an extension of the classical Zonotope designed to handle ĝ.,"1 and ĝ.,"2-norm bound perturbations. We introduce all Multi-norm Zonotope abstract transformers necessary to handle these complex networks, including the challenging softmax function and dot product. Our evaluation shows that DeepT can certify average robustness radii that are 28× larger than the state-of-the-art, while scaling favorably. Further, for the first time, we certify Transformers against synonym attacks on long sequences of words, where each word can be replaced by any synonym. DeepT achieves a high certification success rate on sequences of words where enumeration-based verification would take 2 to 3 orders of magnitude more time. Show more
Publication status
publishedExternal links
Book title
Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation (PLDI '21)Pages / Article No.
Publisher
Association for Computing MachineryEvent
Subject
Abstract interpretation; Robustness Certification; Deep learning; Adversarial attacks; Transformer networksOrganisational unit
03948 - Vechev, Martin / Vechev, Martin
More
Show all metadata