jinzhuoran / rwku Goto Github PK

impressive work!

Hi,
Thanks for sharing the impressive code!

The computation cost of this repo is higher than expected. As LLaMA-Factory suggested, a 7B model would only require 60GB GPU memory for full fine-tune. However, it requires about 160 GB when I run full/run_ga.sh.

Which step increases the cost?

Hi, thanks for the great work.

I would like to know how the values ​​of "MIA Set" and "Flu" in the table are reported with the obtained results.

#3 (comment)
For example, "Loss -2.492", "Loss -2.370" and "Entropy 6.978" in the above file.

Hello. I am trying to replicate the baseline results for Target unlearning from the paper, however I have been getting consistently worse results for both LLama3-8B-instruct and Phi-3 Mini-4K-Instruct. Here is a comparison:

I am running this command to get the baseline on the first 100 celebrities (1-100):

WANDB_DISABLED=true python src/train_bash.py --stage ga
--model_name_or_path mms://core-ai-nlp/Meta-Llama-3-8B-Instruct/1
--dataset ${id}Positive --dataset_dir RWKU-dataset
--output_dir ./saves/unlearn_bench/People/${id}/${method_name}/${model_name} --overwrite_cache
--overwrite_output_dir --cutoff_len 512 --preprocessing_num_workers 16
--per_device_train_batch_size 8 --per_device_eval_batch_size 8 --gradient_accumulation_steps 1
--lr_scheduler_type cosine --logging_steps 10 --warmup_steps 20 --save_steps 30000
--eval_steps 30000 --evaluation_strategy steps --load_best_model_at_end --template llama3
--learning_rate 6e-8 --num_train_epochs 1.0 --val_size 0.0000001 --plot_loss
--output_result_dir ./results/unlearn_bench/People/${id}/${method_name}/${model_name}
--fp16 --eval_dataset_dir RWKU-dataset/RWKU/Target/
--target ${id} 2>&1 | tee ./logs/unlearn_bench/People/${method_name}/${model_name}${id}.log