Hybrid Unsupervised + Supervised Latent Space Deepfake Detection Framework
The rapid evolution of generative AI has enabled the creation of highly realistic synthetic human faces (deepfakes). While these technologies offer creative and commercial value, they introduce major risks in:
- Digital forensics
- Biometric authentication systems
- Identity fraud
- Large-scale misinformation
Traditional CNN classifiers often rely on surface-level artifacts that disappear as generative models improve. This project introduces a representation-learning-driven detection framework that focuses on latent distribution modeling rather than pixel artifacts.
We propose a hybrid pipeline:
β-VAE Unsupervised Representation Learning → Latent Space Supervised Classification
This bridges generative modeling and discriminative learning for robust deepfake detection.
Instead of asking:
"Does this image look fake?"
We ask:
"Does this face follow the natural latent distribution of real human faces?"
- Source: FFHQ (Flickr-Faces-HQ) Dataset
- Diversity across:
- Age, gender, ethnicity
- Lighting and pose
- Expressions and backgrounds
- Generated using Stable Diffusion — Realistic Vision V5.1
- Resolution: 512×512
- Inference Steps: 28
- Guidance Scale: 3.0
- Hyper-realistic portrait prompt engineering
| Category | Samples |
|---|---|
| Real | 5,012 |
| Fake | 1,800 |
| Total | 6,812 |
Preprocessing Steps
- Resized to 64×64
- Normalized pixel values
- Converted to PyTorch tensors
The β-VAE learns structured, disentangled latent representations.
Loss Function
L = MSE + β · KL(q(z|x) || p(z))
Where β = 4, enforcing stronger regularization.
| Component | Details |
|---|---|
| Encoder | 3 Convolutional layers |
| Latent Space | 128-dimensional embedding |
| Decoder | Symmetric Transposed CNN |
| Training | 50 epochs with Early Stopping |
The β-VAE learned compact generative representations of faces.
✔ Stable convergence
✔ No posterior collapse
Used KL divergence thresholds (95th percentile) for anomaly detection.
Observation:
Failed to detect most diffusion-generated fakes due to distribution overlap.
Insight: Pure anomaly detection is insufficient when trained on mixed data.
Extracted 128-D latent vectors → used as discriminative features.
| Setting | Details |
|---|---|
| Classifier | Logistic Regression |
| Split | 80/20 stratified |
| Scaling | Standard Scaler |
| Labels | Real (0), Fake (1) |
| Metric | Score |
|---|---|
| Accuracy | 98% |
| Fake Precision | 95% |
| Fake Recall | 97% |
| ROC-AUC | 0.997 |
- t-SNE showed clear class clusters
- ROC curve demonstrated near-perfect separability
- Confusion matrix showed minimal misclassification
- β-VAE captures generative facial structure, not just textures
- Latent embeddings act as forensic fingerprints
- Hybrid modeling outperforms standard CNN classifiers
- Diffusion models mimic pixels but not true latent distributions
- Biometric security systems
- Digital identity verification
- Social media deepfake filtering
- AI forensic analysis
- Secure KYC pipelines
- Video-based deepfake detection
- Temporal modeling
- Attention mechanisms
- Real-time forensic deployment
- PyTorch
- Convolutional Neural Networks
- Variational Autoencoders
- Scikit-learn
- t-SNE Visualization
- Diffusion Models
- β-VAE Paper — Higgins et al., ICLR
- FFHQ Dataset — NVLabs
- Stable Diffusion Realistic Vision V5.1
- PyTorch Vision Models
This project demonstrates that structured latent space modeling combined with lightweight supervised learning provides a powerful and generalizable deepfake detection framework — integrating:
Generative Modeling + Representation Learning + AI Security