This repository contains a course project for Introduction to Information Security. The project surveys existing defense mechanisms designed to mitigate jailbreak attacks in Large Language Models (LLMs) and organizes them into a structured taxonomy.
The accompanying paper analyzes how current research addresses jailbreak vulnerabilities and categorizes defense strategies based on when they intervene in the LLM pipeline.
📄 Full Paper
Large Language Models such as GPT-4, Bard, and LLaMA are powerful text generation systems, but they remain vulnerable to jailbreak attacks. Jailbreak attacks attempt to bypass safety mechanisms through adversarial prompts, leading models to generate harmful or restricted content.
This project reviews existing research on defense mechanisms against jailbreak attacks and categorizes them into two main groups:
These approaches modify or strengthen the model before user interaction.
Examples include:
- Fine-tuning models with safety datasets
- Generating adversarial training examples
- Selecting specialized model architectures based on prompt characteristics
These approaches intervene during or after the model processes a user query.
Examples include:
- Prompt template transformations
- Self-reminder or safety prompts
- Token-level probability manipulation during decoding
- Survey of recent research on LLM jailbreak defense mechanisms
- A taxonomy of defense strategies based on intervention stage
- Comparative discussion of advantages and limitations of each category
- Identification of future research directions for improving LLM safety
- LLM security and safety
- Jailbreak attacks
- Defense mechanisms
- Prompt-based safety techniques
- Token-level decoding defenses
Course: Introduction to Information Security Institution: Korea University Semester: 2024-2
- Miryeong Kang
- Minjae Kang