🧠 CCDEPLRL — Foundations of Deep Learning

Course Code: CCDEPLRL

Name: Jay Arre P. Talosig

Section: MSCS25T32

Professor: Mr. Davood Pour Yousefian Barfeh

Semester: March–June 2026

Institution: National University

This repository contains all six laboratory assignments for the Foundations of Deep Learning course at National University (MSCS25T32). Each assignment is a self-contained Jupyter Notebook built with PyTorch, progressing from basic tensor mathematics to full CNN evaluation — mirroring the real-world journey of a deep learning practitioner.

The course follows a deliberate, cumulative progression across three phases:

Phase 1 ──► Mathematical Foundations (No Dataset)
Phase 2 ──► Neural Networks on Grayscale Images (FashionMNIST)
Phase 3 ──► Convolutional Neural Networks on Color Images (CIFAR-10)

Each completed assignment includes:

• A fully executed Jupyter Notebook (.ipynb) with inline outputs, plots, and explanations
• A companion Markdown documentation (.md) summarizing the approach, results, and key findings

CCDEPLRL/
├── Assignment-1/
│   ├── assignment_1_intro_dl.ipynb       # Tensors & Activation Functions
│   └── ASSIGNMENT_1_INTRO_DL.md          # Markdown documentation
├── Assignment-2/
│   ├── assignment_2_neural_networks.ipynb # Feedforward NN on FashionMNIST
│   └── ASSIGNMENT_2_NN.md                 # Markdown documentation
├── Assignment-3/
│   ├── assignment_3_optimization.ipynb    # SGD vs Adam, Loss Curves
│   └── ASSIGNMENT_3_OPTIMIZATION.md      # Markdown documentation
├── Assignment-4/
│   ├── assignment_4_regularization.ipynb  # Dropout & Early Stopping
│   └── ASSIGNMENT_4_REGULARIZATION.md    # Markdown documentation
├── Assignment-5/
│   ├── assignment_5_cnn.ipynb             # CNN on CIFAR-10
│   └── ASSIGNMENT_5_CNN.md               # Markdown documentation
├── Assignment-6/
│   ├── assignment_6_cnn_evaluation.ipynb  # Confusion Matrix & Error Analysis
│   └── ASSIGNMENT_6_CNN_EVALUATION.md    # Markdown documentation
├── assignments/
│   └── Assignments.pdf                    # Course assignment specification
├── assets/
│   └── banner.png                         # README banner image
├── week 1/
│   └── Chapter 1 Introduction to Deep Learning.pdf
├── week 2/
│   └── Deep Learning-syllabus.pdf
├── week 3/
│   └── Module 1.pdf
├── week 4/
│   └── Module 2.pdf
├── week 5/
│   └── Module 3.pdf
├── week 6/
│   └── Module 4.pdf
├── .gitignore
├── LICENSE
└── README.md

• Industry standard for deep learning research and academic work
• Dynamic computation graph — ideal for learning, you see exactly what's happening layer-by-layer
• torchvision provides built-in, production-ready FashionMNIST and CIFAR-10 data loaders
• Native Google Colab support with GPU acceleration (T4 / A100)
• Transparent autograd engine for understanding backpropagation

The dataset strategy is deliberate and mirrors increasing course complexity:

Objective: Become familiar with Google Colab, basic tensor operations, and activation functions used in deep learning.

What's Implemented:

• Scalar, vector, matrix, 3D, 4D, and 5D tensor creation with shape inspection
• Element-wise operations, matrix multiplication (torch.matmul), transpose, reshape, aggregations
• Sigmoid — formula, implementation, gradient, and vanishing gradient analysis
• Tanh — formula, implementation, zero-centered advantage, gradient visualization
• ReLU — formula, implementation, dying ReLU problem, and comparison
• Combined 4-panel comparison plot: all 3 functions + all 3 gradients overlaid

Key Findings:

📂 View Assignment 1 →

Objective: Build and train a feedforward neural network on FashionMNIST from scratch.

What's Implemented:

• FashionMNIST loading with torchvision (60k train / 10k test, normalized)
• Feedforward MLP: Input(784) → Hidden(256) → Hidden(128) → Output(10)
• Full training loop with cross-entropy loss and Adam optimizer
• Training & validation accuracy and loss curves plotted across epochs
• Conceptual explanation of forward propagation, backpropagation, and the role of hidden layers

📂 View Assignment 2 →

Objective: Empirically compare SGD and Adam optimizers on the same network architecture.

What's Implemented:

• Reuses the MLP architecture from Assignment 2 for a controlled experiment
• SGD trained with momentum (momentum=0.9, lr=0.01)
• Adam trained with default parameters (lr=0.001, β₁=0.9, β₂=0.999)
• Side-by-side training and validation loss/accuracy curves
• Analysis of convergence speed, final accuracy, and sensitivity to learning rate selection

📂 View Assignment 3 →

Objective: Demonstrate overfitting and the measurable impact of regularization techniques.

What's Implemented:

Controlled Experiment Design:

• Architecture: 2-layer MLP (784 → 256 → 128 → 10), Adam optimizer (lr=0.001), seed=42
• Baseline: No dropout, no early stopping — trained for 50 epochs to intentionally induce overfitting
• Regularized: Dropout (p=0.3) + Early Stopping (patience=7) — trained for up to 50 epochs

Results:

• Full 2×2 grid visualization plotting accuracy and loss curves for both models
• Direct overlay comparison of both models on the same axes
• In-depth discussion: Dropout mechanics, Early Stopping trigger analysis, and underfitting vs overfitting diagnosis

📂 View Assignment 4 →

Objective: Build a CNN architecture for CIFAR-10 RGB image classification.

What's Implemented:

• CIFAR-10 loading (50k train / 10k test, normalized per-channel: mean=[0.4914, 0.4822, 0.4465])
• Sample grid visualization of all 10 CIFAR-10 classes
• SimpleCNN Architecture:

  Conv(3→32, 3×3) → ReLU → MaxPool(2×2)
  Conv(32→64, 3×3) → ReLU → MaxPool(2×2)
  Flatten → Linear(64×6×6 → 512) → ReLU → Dropout(0.5)
  Linear(512 → 10)
  

• Training with learning rate scheduler (StepLR)
• Per-class accuracy breakdown across all 10 CIFAR-10 categories
• Explanation of why CNNs outperform fully-connected layers for image data (parameter sharing, translation invariance)

📂 View Assignment 5 →

Objective: Comprehensive forensic evaluation of the trained CNN on the held-out test set.

What's Implemented:

• Full evaluation of SimpleCNN on 10,000 unseen CIFAR-10 test images
• Heatmap Confusion Matrix generated with sklearn + visualized with seaborn
• Side-by-side grids of Correctly Classified images with predicted labels
• Grids of Incorrectly Classified images — true label vs model prediction for error forensics
• Error Analysis: Identified confused class pairs (cat↔dog, automobile↔truck, airplane↔ship) and their root causes (low resolution, background bias, limited model depth)
• Future Improvement Roadmap: ResNet residual connections, data augmentation, transfer learning from ImageNet pre-trained weights

📂 View Assignment 6 →

Each assignment follows a structured branching strategy:

main ──► assignment/1-intro-deep-learning   ──► PR & Merge ──► main
main ──► assignment/2-neural-networks       ──► PR & Merge ──► main
main ──► assignment/3-loss-and-optimization ──► PR & Merge ──► main
main ──► assignment/4-regularization        ──► PR & Merge ──► main
main ──► assignment/5-intro-cnn             ──► PR & Merge ──► main
main ──► assignment/6-cnn-evaluation        ──► PR & Merge ──► main

Branch naming convention: assignment/<N>-<short-topic> (kebab-case, 2–4 words max)

Per-assignment workflow:

1. Create feature branch from main
2. Build notebook with code, plots, and explanations
3. Generate companion Markdown documentation
4. Commit with conventional commit messages
5. Open PR → Review → Merge to main

• Python 3.10+
• PyTorch 2.x + torchvision
• NumPy, Matplotlib, Seaborn
• scikit-learn

1. Navigate to any Assignment-N/ folder
2. Open the .ipynb file on GitHub
3. Click the "Open in Colab" badge at the top of the notebook
4. Select Runtime → Change Runtime Type → T4 GPU
5. Run all cells — GPU environment is pre-configured

# Clone the repository
git clone https://github.com/flexycode/CCDEPLRL.git
cd CCDEPLRL

# Create a conda environment
conda create -n dl-course python=3.11
conda activate dl-course

# Install PyTorch (CUDA 12.1 for GPU, or CPU-only)
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
# or CPU-only:
pip install torch torchvision

# Install remaining dependencies
pip install numpy matplotlib seaborn scikit-learn jupyter

# Open Jupyter
jupyter notebook

# Clone and enter the repository
git clone https://github.com/flexycode/CCDEPLRL.git
cd CCDEPLRL

# Create a virtual environment
python -m venv venv
source venv/bin/activate   # Linux/Mac
# or
venv\Scripts\activate      # Windows

# Install all dependencies
pip install torch torchvision numpy matplotlib seaborn scikit-learn jupyter

# Open Jupyter
jupyter notebook

1. Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.
2. Chollet, F. (2021). Deep Learning with Python (2nd ed.). Manning Publications.
3. Zhang, A., Lipton, Z., Li, M., & Smola, A. (2023). Dive into Deep Learning. — d2l.ai
4. PyTorch Documentation — pytorch.org/docs
5. Nair, V., & Hinton, G. E. (2010). Rectified Linear Units Improve Restricted Boltzmann Machines. ICML 2010.
6. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks. NeurIPS 2012.
7. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting. JMLR.
8. He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. CVPR 2016.
9. Shorten, C., & Khoshgoftaar, T. M. (2019). A Survey on Image Data Augmentation for Deep Learning. Journal of Big Data.

This project is licensed under the MIT License — see the LICENSE file for details.