_no_load_in_8bit module list have custom ignored layers

[ArthurZucker]

Feature request

Could be awesome to have the property _no_load_in_8bit = [] for the 8bit quantisation to allow custom layers to not be converted.
CC @younesbelkada

Motivation

Would increase flexibility

Your contribution

Would be testing on Jukebox

[younesbelkada]

Hi @ArthurZucker
Thanks a lot for the feature request!
I have addressed a commit in #18646 that should support adding an argument no_load_in_8bit_modules in from_pretrained function. Could you try it in your usecase and let me know if this helped?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[younesbelkada]

Closing as completed in #18646