C1 Research Computing Coursework: Interpolator

Overview

This repository contains a full-stack, research-grade system capable of learning and serving neural-network models to interpolate 5-dimensional numerical datasets. This project was developed for the C1 Research Computing Coursework as part of the MPhil in Data Intensive Science at the University of Cambridge.

System Architecture

The system consists of the following core components:

• Machine Learning Package (fivedreg): A Python package implementing data loading (from .npz), standardisation, model training, and inference. The model uses a lightweight, fully configurable neural network optimized for fast CPU training.
• FastAPI Backend: Exposes REST endpoints (/health, /upload, /train, /predict) to handle dataset uploads and model operations.
• Next.js Frontend: A modern, interactive UI featuring dedicated pages for uploading datasets, configuring hyperparameters to train models, and making predictions with a 5-input form.
• Documentation: A comprehensive Sphinx-based local HTML documentation build.
• Containerization: A docker-compose.yml file to seamlessly serve both the frontend and backend in containers.
• Testing Suite: Comprehensive tests validating system components.

Repository Structure

• backend/: Contains the FastAPI application (main.py), the fivedreg machine learning package, pyproject.toml configuration, profiling scripts, and the tests/ suite.
• frontend/: Contains the Next.js web application and package.json.
• docs/: Sphinx documentation source files and Makefile.
• docker-compose.yml: Configuration for containerized deployment.
• build_docs.sh: Shell script to generate local documentation.
• start.sh: Shell script to launch the technology stack locally without Docker.
• start_docker.sh: Shell script to launch the stack using Docker Compose.

Getting Started

Running with Docker (Recommended)

You can launch the entire technology stack using Docker Compose. Run the following script from the root directory:

./start_docker.sh

Alternatively, you can manually run:

docker-compose up --build

Running Locally

To launch the frontend and backend services locally without Docker, use the provided shell script:

./start.sh

Ensure you have the required Python dependencies (as declared in backend/pyproject.toml) and Node.js installed.

Usage

Once the application is running, navigate to the frontend interface in your browser:

• Upload (/upload): Submit a 5D .npz dataset using the file upload interface.
• Train (/train): Configure network hyperparameters (e.g., layers, neurons, learning rate, max iterations) and initiate the training process.
• Predict (/predict): Enter 5 numerical values into the interactive form to query the trained model and view the prediction results.

Testing

A comprehensive testing suite is located under backend/tests/. It validates system reliability, checking both the fivedreg ML module and the FastAPI endpoints.

Documentation

Detailed API references, user guides, installation instructions, and test suite descriptions are generated using Sphinx. To build the documentation locally, execute:

./build_docs.sh

You can then view the HTML documentation through your web browser via standard file:// URLs.

Performance and Profiling

Comprehensive benchmarking of the model's computational characteristics has been conducted. The profiling analyzes training time, memory usage, Mean Squared Error (MSE), and R-squared scores. It establishes scaling behavior across datasets of 1K, 5K, and 10K samples. The detailed results and plots can be found in the "Performances and profiling" section of the Sphinx documentation.