Download PDFOpen PDF in browserSemantic-Guided Latent Space Backdoor Attack: a Novel Threat to Stable DiffusionEasyChair Preprint 1397917 pages•Date: July 15, 2024AbstractStable Diffusion (SD) models have achieved remarkable success in text-to-image synthesis, but their security vulnerabilities remain largely unexplored. In this paper, we introduce a novel semantic-guided latent space backdoor attack (SG-LSBA) that leverages the semantic information in the text input to inject stealthy and seman- tically coherent backdoors into SD models. Our approach outperforms existing methods by crafting context-aware semantic triggers, identifying target visual features in the latent space, and employing an adversarial optimization framework. Extensive evaluations demonstrate the high success rates, strong semantic relevance, and exceptional stealthiness of SG-LSBA. Our findings highlight the urgent need for considering the complex interplay between semantics and latent representations in developing robust defenses against backdoor attacks in SD models. We make our code and datasets publicly available to facilitate further research and development of secure and reliable text-to-image synthesis models. The code is available at https://github.com/paoche11/SG-LSBA. Keyphrases: Latent space backdoor attack, Semantic triggers, Stable Diffusion models, diffusion models, semantic guidance
|