feat: Add PhotonCT Walnuts dataset loader (#173) - #186
Open
Paramveersingh-S wants to merge 1 commit into
Open
Conversation
Resolves CambridgeCIA#173 by introducing a native PyTorch dataset loader for the PhotonCT Walnuts dataset. Features: - Added PhotonCTWalnutDataset supporting 'High', 'Low', and 'Total' energy bins. - Integrated native flat-field air correction matching the calibration table. - Reshapes projections correctly from column-major MATLAB format. - Added download.py script to fetch walnuts selectively.
Member
|
Fantastic! this would need some testing, bear with me. @AndreaSainz , maybe you can test this PR? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces a native PyTorch
Datasetloader for the newly published PhotonCT Walnuts dataset, expanding LION's support for spectral image reconstruction and deep learning data.Resolves #173.
Problem
The newly released PhotonCT Walnuts dataset consists of high-resolution, multi-energy cone-beam CT scans (~296 GB total) stored in MATLAB's 16-bit unsigned integer
.rawformat. To train models within LION effectively, we need a native PyTorch integration that can read these projections iteratively, convert them from Fortran order (column-major) to standard PyTorch tensors, and properly normalize the data against the physical detector's flat-field air tables.Solution
I have implemented
PhotonCTWalnutDatasetinsideLION.data_loaders.photonct_walnuts, which mimics the structural simplicity of the existing originalwalnutsloader, while adding native spectral features and MATLAB-compatible data reshaping.Key implementations:
'High','Low', and'Total'energy bins. TheLowbin is dynamically constructed fromTotal - Highper the official physical data acquisition standard.air_table_{bin}.rawand performs the native physical linearisation mapping (-log(proj) + log(airtable)).numpy.fromfilecombined with.reshape(..., order='F')to correctly cast the MATLAB column-major data into contiguous C-order vectors for PyTorch compatibility.download.py) to fetch the Calibration Table and individual walnuts incrementally from Zenodo, preventing disk saturation from the massive full dataset size.Verification
[1, 505, 2063]tensor structures.__len__matches the correct number of projections and the-logtransformation avoids-infdomains via epsilon clamping (1e-6).