S1: Statistical Methods for Data Intensive Science - Coursework

This repository contains the coursework and solution for the "S1: Statistical Methods for Data Intensive Science" module (MPhil DIS).

Project Overview

The objective of this assignment is to evaluate the performance of a photon detector by analyzing calibration data. Given a dataset of 1,000 measured energies ($E$) at seven known true laser energies ($E_0$), the task is to estimate five unknown calibration parameters ($\lambda, \Delta, a, b, c$) and their associated uncertainties. The measured energy is modeled as a normal distribution where the mean and variance are functions of these five parameters.

Methodology

The analysis progresses through four distinct statistical approaches to estimate the parameters and robustly quantify their uncertainties:

1. Sample Estimates: Initial parameter estimation using sample means and standard deviations followed by least-squares fitting.
2. Individual Maximum Likelihood (ML) Fits: Unbinned ML fits applied separately to the data at each known $E_0$ level.
3. Simultaneous ML Fit: A comprehensive unbinned ML fit performed simultaneously across the entire dataset using the MINUIT algorithm.
4. Bootstrapping: A non-parametric bootstrap (2,500 samples with replacement) to rigorously evaluate parameter uncertainties and cross-check the errors reported by the ML fits.

Key Findings

A detailed discussion of the results is included at the end of the solution notebook. A significant finding of this analysis is that the MINUIT (Hesse) errors consistently underestimate the true uncertainty compared to the bootstrap estimates, and a reparameterization of the model alleviates this issue. Another critical insight is that the three fitting methods (sample estimates, individual ML fits, and simultaneous ML fit) are theoretically similar with each other, with sample estimates being almost equivalent to the individual ML fits, and the individual ML fits being an approximation and simplification of the simultaneous ML fit.

Repository Structure

• notebooks/solution.ipynb: The main Jupyter Notebook containing the complete end-to-end analysis, plotting, and discussion.
• notebooks/instructions.ipynb: The original assignment instructions and problem description.
• s1_sol/: A custom, pip-installable Python package. The core statistical functions and fitting wrappers are implemented in s1_sol/methods.py.
• figs/: Directory containing all generated output plots (Figures 1.1 through 4.2, plus a custom Figure 5.1 for the diagnostic discussion).
• results.json: The formatted output file containing the final parameter estimates and error bounds from all methods.
• sample.csv: The unique calibration dataset used for this analysis.

How to Run and Reproduce

To reproduce the results, figures, and results.json file:

1. Install the local package: Navigate to the root of the repository and install the s1_sol package in editable mode:

   pip install -e .

2. Run the Notebook: Open notebooks/solution.ipynb and execute all cells from start to finish. The notebook is fully self-contained once the package is installed and will automatically populate the figs/ folder and overwrite results.json.

Acknowledgements & LLM Usage

GitHub Copilot was utilized to assist with writing certain portions of the codebase, primarily for drafting docstrings and matplotlib plotting boilerplate.