Search
Results
-
Self-Supervised Contrastive Learning with Adversarial Perturbations for Defending Word Substitution-based Attacks
(2022)Findings of the Association for Computational Linguistics: NAACL 2022In this paper, we present an approach to improve the robustness of BERT language models against word substitution-based adversarial attacks by leveraging adversarial perturbations for self-supervised contrastive learning. We create a word-level adversarial attack generating hard positives on-the-fly as adversarial examples during contrastive learning. In contrast to previous works, our method improves model robustness without using any ...Conference Paper -
TempCaps: A Capsule Network-based Embedding Model for Temporal Knowledge Graph Completion
(2022)Proceedings of the Sixth Workshop on Structured Prediction for NLPTemporal knowledge graphs store the dynamics of entities and relations during a time period. However, typical temporal knowledge graphs often suffer from incomplete dynamics with missing facts in real-world scenarios. Hence, modeling temporal knowledge graphs to complete the missing facts is important. In this paper, we tackle the temporal knowledge graph completion task by proposing TempCaps, which is a Capsule network-based embedding ...Conference Paper -
Beyond prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations
(2022)Proceedings of the 2022 Conference on Empirical Methods in Natural Language ProcessingRecent work has demonstrated that pre-trained language models (PLMs) are zero-shot learners. However, most existing zero-shot methods involve heavy human engineering or complicated self-training pipelines, hindering their application to new situations. In this work, we show that zero-shot text classification can be improved simply by clustering texts in the embedding spaces of PLMs. Specifically, we fit the unlabeled texts with a Bayesian ...Conference Paper -
KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation
(2021)Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language ProcessingWe present Knowledge Enhanced Multimodal BART (KM-BART), which is a Transformer-based sequence-to-sequence model capable of reasoning about commonsense knowledge from multimodal inputs of images and texts. We adapt the generative BART architecture (Lewis et al., 2020) to a multimodal model with visual and textual inputs. We further develop novel pretraining tasks to improve the model performance on the Visual Commonsense Generation (VCG) ...Conference Paper -
A Geometry-Inspired Attack for Generating Natural Language Adversarial Examples
(2020)Proceedings of the 28th International Conference on Computational LinguisticsGenerating adversarial examples for natural language is hard, as natural language consists of discrete symbols, and examples are often of variable lengths. In this paper, we propose a geometry-inspired attack for generating natural language adversarial examples. Our attack generates adversarial examples by iteratively approximating the decision boundary of Deep Neural Networks (DNNs). Experiments on two datasets with two different models ...Conference Paper