AttributeError: add_special_tokens conflicts with the method add_special_tokens in RobertaTokenizerFast

[berkatil]

Hi,
When I run AutoTokenizer.from_pretrained("rrivera1849/LUAR-CRUD"), I get AttributeError: add_special_tokens conflicts with the method add_special_tokens in RobertaTokenizerFast error.
transformers verision is 4.44.2

thanks.

[JackOE3]

I got the same error.

[spookyQubit]

@berkatil , the requirements.txt mentions transformers==4.14.1 (see https://github.com/LLNL/LUAR/blob/main/requirements.txt#L42C1-L42C21). Maybe that is the reason you are getting this error?

Have you tried installing by following https://github.com/LLNL/LUAR?tab=readme-ov-file#installation ?

[berkatil]

Thank you. Yes, actually I tried with transformers==4.25, it works with that too, probably would work with 4.14.1. I did not realize the requirements file. It might be good to add to huggingface too because I was just checking that page first.