A simple implementation of probabilistic machine learning models with uncertainty quantification, focusing on Gaussian Processes and Deep Ensembles. This project addresses the shortcomings of prior work by providing a detailed comparison of the models with respect to calibration, model performance, and robustness under distributional shifts.
The associated paper is available on arXiv.
This project uses Hydra for configuration management. To run the code (from the xai-gp directory):
python main.py model=model_config.yaml data=data_config.yamlDeepGPModel: Deep Gaussian ProcessDSPPModel: Deep Sigma Point ProcessesDeepEnsembleRegressor: Deep Ensemble for regressionDeepEnsembleClassifier: Deep Ensemble for classification
CASP: Protein regression datasetESR: Seizure classification dataset
We provide optimized hyperparameters for the models inside the conf/models directory.
You can also set up a virtual environment using Poetry. Poetry can be installed using pip:
pip install poetry
Then initiate the virtual environment with the required dependencies (see poetry.lock, pyproject.toml):
poetry config virtualenvs.in-project true # ensures virtual environment is in project
poetry install
The virtual environment can be accessed from the shell using:
poetry shell
IDEs like Pycharm will be able to detect the interpreter of this virtual environment.
We save the weights of our optimized model inside the results/weights directory.
We utilize code from mvaldenegro to determine the calibration of our models. Additionally, the implementation of our DeepGP models is based on this example from the GPytorch Documentation.
This project is licensed under the GNU Affero General Public License v3.0 - see the LICENSE file for details.