Fine tuning with RL

[terryfrankcombe]

Hi folks

Is there a recommended way to load a pretrained model for RL without loading the entire checkpoint?

I'm training with TD3 and I want to refine the model with new data and a modified reward function. If I call torchfort_rl_off_policy_create_system("td3_system",... then torchfort_load_model("td3_system",... with the filename for a saved checkpoint policy/model.pt I get an Invalid usage. (Invalid model name provided.) error.

I guess one approach would be to load the entire checkpoint then try to kill the history, but I don't know how to do that either.

How do I set up for fine tuning?

Ciao
Terry

[azrael417]

Hello Terry,

that is indeed a missing functionality. I implemented that into this branch:
#107

can you have a look and evaluate and let me know what you think?

Best Regards
Thorsten

[terryfrankcombe]

Wow, that was fast! I will take me a couple of days to make time to test.

[terryfrankcombe]

Looks to work just fine.