Pytorch support - #276

Draft

cpozogonzalez wants to merge 148 commits into

mainfrom

pytorch_support

Commits on Jul 21, 2026

add pytorch template

tjarkmiener
authored and
cpozogonzalez
committed
79caedc
First commit with new structure to support MultiDLFrameworks

pguzman
authored and
cpozogonzalez
committed
feae15e
some bug fixed and refactored some part

pguzman
authored and
cpozogonzalez
committed
ed53c87
refactored to avoid dependencies with tensorflow and pytorch. Added tqdm in keras.

pguzman
authored and
cpozogonzalez
committed
0f88ecc
fixed a missing var when using pytorch and removed commented code.

pguzman
authored and
cpozogonzalez
committed
899317b
Added DataLoader in Factory format

pguzman
authored and
cpozogonzalez
committed
5a53710
Fix unit test??

pguzman
authored and
cpozogonzalez
committed
dc2b18c
fixed some bugs and fixed framework parameter

pguzman
authored and
cpozogonzalez
committed
ef870c9
Added nets, load checkpoints in pytorch

pguzman
authored and
cpozogonzalez
committed
d339494
fixed unit test

pguzman
authored and
cpozogonzalez
committed
e8615f5
formated code (round 1) and fixed a function that does not exist in pytorch legacy code

pguzman
authored and
cpozogonzalez
committed
6caef05
fixed the training command binary

pguzman
authored and
cpozogonzalez
committed
d59bcb6
Fixed flags in training and fixed a bug in validation index

pguzman
authored and
cpozogonzalez
committed
49205a9
Fixed __main__ and minor changes

pguzman
authored and
cpozogonzalez
committed
5b81fb0
refactored keras files and fixed minor issues...

pguzman
authored and
cpozogonzalez
committed
7926e8c
added first sketch of prediction

pguzman
authored and
cpozogonzalez
committed
0b77156
fixed _predict_with_model

pguzman
authored and
cpozogonzalez
committed
52466bc
updated pytorch_loader

pguzman
authored and
cpozogonzalez
committed
fc8fae7
skeleton for training in pytorch

pguzman
authored and
cpozogonzalez
committed
548ddf5
fixed cuda memory problem in pytorch

pguzman
authored and
cpozogonzalez
committed
78f1e1b
fixed parameter log saving and added missing files

pguzman
authored and
cpozogonzalez
committed
6ff757b
added some comments

pguzman
authored and
cpozogonzalez
committed
0da2621
fixed some bugs in training and dataloader

pguzman
authored and
cpozogonzalez
committed
578db9e
fixing training direction

pguzman
authored and
cpozogonzalez
committed
3d91d13
cleaning up pytorch training module.

pguzman
authored and
cpozogonzalez
committed
7b09b86
added the fix for CI

pguzman
authored and
cpozogonzalez
committed
37f10be
removed unused code

pguzman
authored and
cpozogonzalez
committed
8177bd5
fixed minor bugs

pguzman
authored and
cpozogonzalez
committed
6e6ce9c
removed unused call

pguzman
authored and
cpozogonzalez
committed
6ffbf9f
fixed some bugs in task type. Still there is a bug to fix

pguzman
authored and
cpozogonzalez
committed
91f7c75
fixed the bug in Type training (blocking)

pguzman
authored and
cpozogonzalez
committed
9baeec0
Added Augmentation and removed unused code.

pguzman
authored and
cpozogonzalez
committed
19eac76
Added Diffusion model and adapted the type loss

pguzman
authored and
cpozogonzalez
committed
27ab9ec
Fixed Diffusion Regression code and plot bug fix

pguzman
authored and
cpozogonzalez
committed
30a7285
clean up and added DBB in diffusion model

pguzman
authored and
cpozogonzalez
committed
00c1170
fixed some minor bugs

pguzman
authored and
cpozogonzalez
committed
674a922
rem,oved commented code and added gpu memory trick

pguzman
authored and
cpozogonzalez
committed
575daae
modification in training and added another network for testing

pguzman
authored and
cpozogonzalez
committed
2208a23
fixed bug in pytorch loader

pguzman
authored and
cpozogonzalez
committed
2ec8dac
Adds new models
cpozogonzalez
committed
a08265d
Update the git ignore
cpozogonzalez
committed
f34ae12
Change the pyproyect file, to the new structure of folders
cpozogonzalez
committed
2fef763
Add the Subarray Descriptions and refactoring the predict task using keras
cpozogonzalez
committed
cecc3dd
Separete the model and update the references
cpozogonzalez
committed
abb7241
Add some changes in the predictions tasks
cpozogonzalez
committed
4e5f853
Fix the problems related with the predict model and add the loading bar
cpozogonzalez
committed
55067fe
Add the MC dl2 predictions
cpozogonzalez
committed
a61be79
Minor changes and optimization
cpozogonzalez
committed
fd37df1
Support the last dl1-data-handler version
cpozogonzalez
committed
1e2cd85
Fix the pytorch loader and add some changes in the Pylithing validation
cpozogonzalez
committed
b2662ee
Refactoring CTLearnPL
cpozogonzalez
committed
ae66a8f
Update the predictions tools and the ThinResNet model DBB
cpozogonzalez
committed
5bb253c
Add the warning message about the imbalance classes.
cpozogonzalez
committed
e7f2558
Add the class weight informations
cpozogonzalez
committed
afee303
Remove the weight calculation from the energy and cameradirrection tasks
cpozogonzalez
committed
b3593ae
Add the warnings for class weights
cpozogonzalez
committed
8381454
Add the weight only if the task is type, inside the CTLearnPL.py
cpozogonzalez
committed
c4830f2
Update the pytorch dataloader unit test
cpozogonzalez
committed
b215a7b
Fix the problems with preditions
cpozogonzalez
committed
a160b16
Update the feuture vectors storage
cpozogonzalez
committed
e7486d7
Remove the comment in the imports
cpozogonzalez
committed
a854081
Apply some changes to solve the problems after rebase
cpozogonzalez
committed
bfc6c20
Update the test CI, to be more conservative with the space, and update the pytorch loader.
cpozogonzalez
committed
8ae0a01
Separete, the env_files in to differents (pytorch and tensorflow) and optimazate the CI
cpozogonzalez
committed
0d15985
Update the name in the env_file and change the CI
cpozogonzalez
committed
1e83841
Fix the predict problems after the rebase
cpozogonzalez
committed
dc80ad4
Optimaze batch size
cpozogonzalez
committed
2e42a18
Update the enviroments files/*.env
cpozogonzalez
committed
12990ad
Add some comments in all the functions and class and the multiGPU training support
cpozogonzalez
committed
4c7c6fd
Add the log scaling
cpozogonzalez
committed
332f234
Unify the keras and pytorch pipelines with the CTApipe Traitlets
cpozogonzalez
committed
f78199f
Update the unit test and fix the onnx train problems
cpozogonzalez
committed
6f73f3f
Update the env files
cpozogonzalez
committed
9abade2
Change the name of the envioroments
cpozogonzalez
committed
25935fe
Solve the problem in the FLoat loading
cpozogonzalez
committed
6009bd3
Add the numbers of workers etc options for the dataloader.
cpozogonzalez
committed
997e256
Add a warning in the pytroch onxx train
cpozogonzalez
committed
45ea6bc
Add the Pytorch Keras Resnet Model
cpozogonzalez
committed
982fc71
Update the enviroment files
cpozogonzalez
committed
d5cd9a3
Change the energy criterion by default to mean
cpozogonzalez
committed
790143e
Solve some bug after the rebase
cpozogonzalez
committed
e951976

Commits on Jul 22, 2026

Update the CI to include test
cpozogonzalez
committed
a61a306
Fix the CI
cpozogonzalez
committed
5a238b8
Solving flake 8 problems
cpozogonzalez
committed
92dae04
Fix test
cpozogonzalez
committed
03efde1

Commits on Jul 23, 2026

install tf and torch in same env and run CI with the same job
tjarkmiener
committed
5470007
run latest and lowest supported versions of tensorflow and torch
tjarkmiener
committed
c7d1b2e
run Ci with schedule at 2:00 UTC
tjarkmiener
committed
bfe0961
fix minimal torch version to >=2.2.X
tjarkmiener
committed
78c0d59
Add the opncv in the pyproject
cpozogonzalez
committed
34cbfbb
Fix the test
cpozogonzalez
committed
a36ac31
make abstract level of singleCNN and ResNet to inherit the different frameworks
tjarkmiener
committed
a09087b
fix some model set up for keras/pytorch
tjarkmiener
committed
77ef080
fix pytorch models
tjarkmiener
committed
9fbf5de
remove unused imports
tjarkmiener
committed
b5ded08
added tests for models in keras and pytorch
tjarkmiener
committed
822e39b
update pytorch model
tjarkmiener
committed
44071df
fix keras ResNet model
tjarkmiener
committed
18f8f35
fix tests for keras and pytorch models
tjarkmiener
committed
62a217f
change activation function in test models
tjarkmiener
committed
3d5fdfa
add random seed
tjarkmiener
committed
38db4b2
fix models and polish tests (WIP)
tjarkmiener
committed
502d28c

Commits on Jul 24, 2026

fix models and CI for ResNet models
tjarkmiener
committed
312c88a
fix bug where reduction ratio and input channel lead to filter number 0 which is invalid
tjarkmiener
committed
73a4ab3
test also attention layers in ResNet
tjarkmiener
committed
2175726
fix __all__ for pytorch models
tjarkmiener
committed
6d67e24
only use cpu torch pypi for CI and organize cache in CI
tjarkmiener
committed
a6f0dba
fix torch vision
tjarkmiener
committed
89b0023
fix some stuff
tjarkmiener
committed
7a300a9
added isolated attention layer tests
tjarkmiener
committed
3b672fd

Commits on Jul 27, 2026

Commits on Jul 28, 2026

Commits on Jul 29, 2026

Change the architectur for the dirrection
cpozogonzalez
committed
231efa9
Fix the test
cpozogonzalez
committed
ce2e070

Commits on Jul 31, 2026

fix tests for Keras and prepare for pytorch
tjarkmiener
committed
e54b93f
remove imports
tjarkmiener
committed
9f0cc0c
added missing train model file for keras
tjarkmiener
committed
40d275e
fix rebasing
tjarkmiener
committed
2085205
added prediction tools in init
tjarkmiener
committed
4210049
move model_type config to base tool
tjarkmiener
committed
38bdf27
test also LoadedModel to resume pretrained model of contest
tjarkmiener
committed
d8377d1
change output dir name
tjarkmiener
committed
4bd9548

Commits on Aug 3, 2026

added pytorch for training tools
tjarkmiener
committed
285e54c
fix conftest
tjarkmiener
committed
80519bd
bump pytorch to 2.4.0 so numpy 2 is supported
tjarkmiener
committed
15e6309

Commits on Aug 4, 2026

test also validation loss from pytorch models in tests
tjarkmiener
committed
0871bdc
unify tensorboard writers for pytorch to match keras structure, i.e. one folder for train and one for validation
tjarkmiener
committed
e13241e
Merge pull request #278 from ctlearn-project/fix_training_tools
cpozogonzalez
authored
b7adb71
fix tests with hardcopy
tjarkmiener
committed
131369f
polish loader tests
tjarkmiener
committed
1fcd303
polish core testing
tjarkmiener
committed
52ab4e7
fix lst1 predict tool test for Keras
tjarkmiener
committed
33b16cb
polish lst1 predict tool test
tjarkmiener
committed
c1fd58e

Commits on Aug 5, 2026

add pytorch prediction in existing tools and add the test for pytorch
tjarkmiener
committed
f2dc264

Commits on Aug 6, 2026

polish prediction pytorch part
tjarkmiener
committed
bcb8b6a
further polishing
tjarkmiener
committed
0b0ede5
move framework setup to utils to be reused in LST1 prediction tool
tjarkmiener
committed
9fea9ba
more polishing
tjarkmiener
committed
ec8c1d2
added pytorch models into LST1 prediction tool
tjarkmiener
committed
e329d5a
even more polishing
tjarkmiener
committed
773b460
Merge pull request #279 from ctlearn-project/fix_predict_tools
tjarkmiener
authored
ab2901c