Learn-to-learn on Arbitrary Textual Conditioning: A Hypernetwork-Driven Meta-Gated LLM

Accepted by ICML2026

arxiv: https://arxiv.org/abs/2605.01973

This project is implemented on the basis of LlamaFactory 0.8.3. Questions to the basic usage can refer to the original repo: https://github.com/hiyouga/LlamaFactory/tree/v0.8.3.

Data Preprocessing

All the data are converted to the messages format (in .jsonl).

Task descriptions of CrossFit&UnifedQA are automatically generated by GPT-4o. Results are in data/task_descs.yaml.

Experiment: metaicl (settings: non-nli-to-nli)

training and eval

bash train_eval_MetaICL_non-nli-to-nli.sh

evaluation only

bash train_eval_MetaICL_non-nli-to-nli.sh

For questions, please email:

jiluoaaron@hotmail.com