Hi,
Thank you for this fantastic project and detailed preprocessing pipelines. I really enjoyed reading your paper as well.
One small note on the loss function in /main/case_1_kolmogorov/no_dm/fno/dm/train_dm.py (probably in other train.py) which I think is a trivial issue:
Feel free to correct me but I think the loss function is already averaged over batch size each time you computed loss with the loss.mean(), so the training size averaging formulation is incorrect because you would average it twice
|
train_loss /= len(train_loader) |
The train_loss accumulation
|
train_loss += loss.item() |
should be changed to train_loss += loss.item() * l_fidel.shape[0]
I think it would only affect the scale of your results but everything else is all good.
Cheers,
XS
Hi,
Thank you for this fantastic project and detailed preprocessing pipelines. I really enjoyed reading your paper as well.
One small note on the loss function in
/main/case_1_kolmogorov/no_dm/fno/dm/train_dm.py(probably in other train.py) which I think is a trivial issue:Feel free to correct me but I think the loss function is already averaged over batch size each time you computed loss with the
loss.mean(), so the training size averaging formulation is incorrect because you would average it twiceNeuralOperator_DiffusionModel/case_1_kolmogorov/no_dm/fno/dm/train_dm.py
Line 209 in 087f84b
The train_loss accumulation
NeuralOperator_DiffusionModel/case_1_kolmogorov/no_dm/fno/dm/train_dm.py
Line 204 in 087f84b
should be changed to
train_loss += loss.item() * l_fidel.shape[0]I think it would only affect the scale of your results but everything else is all good.
Cheers,
XS