Could you please share the sft config you used?

[CurryxIaoHu]

I would like to reproduce the results reported in this paper. Could you please share the SFT config/yaml of LLaMAFactory when fine-tuning the models?

[chenjiawei30]

We have explicitly detailed all the necessary hyper-parameters and fine-tuning settings in the "Implementation Details" section of our paper.

The detailed configuration is provided below for fine-tuning the Qwen2.5-7B model on the ConsistentChat dataset. You can use this to reproduce freely.

### model
model_name_or_path: /ciphome/hf_models/Qwen/Qwen2.5-7B

### method
stage: sft
do_train: true
finetuning_type: full
deepspeed: examples/deepspeed/ds_z3_config.json

### dataset
dataset: 0112_dialog_flow_data
template: qwen
cutoff_len: 4096
max_samples: 100000
overwrite_cache: true
preprocessing_num_workers: 16

### output
output_dir: saves/Qwen2.5-7B-ConsistentChat/full/sft
logging_steps: 10
save_steps: 500
plot_loss: true
overwrite_output_dir: true

### train
per_device_train_batch_size: 1
gradient_accumulation_steps: 2
learning_rate: 1.0e-5
num_train_epochs: 3
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: true
ddp_timeout: 180000000