soarsmu / attack-pretrain-models-of-code Goto Github PK
View Code? Open in Web Editor NEWReplication Package for "Natural Attack for Pre-trained Models of Code", ICSE 2022
License: MIT License
Replication Package for "Natural Attack for Pre-trained Models of Code", ICSE 2022
License: MIT License
I use the Defect-detection in CodeXGLUE to attack fine-tuning CodeBert model according to README , the parameters are similar to:
# fine-tuing
python run.py \
--output_dir=./adv_saved_models \
--model_type=roberta \
--tokenizer_name=microsoft/codebert-base \
--model_name_or_path=microsoft/codebert-base \
--do_train \
--train_data_file=../preprocess/dataset/train.jsonl \
--eval_data_file=../preprocess/dataset/valid.jsonl \
--test_data_file=../preprocess/dataset/test.jsonl \
--epoch 5 \
--block_size 512 \
--train_batch_size 24 \
--eval_batch_size 64 \
--learning_rate 2e-5 \
--max_grad_norm 1.0 \
--evaluate_during_training \
--seed 123456 2>&1 | tee train.log
# attack
python gi_attack.py \
--output_dir=./saved_models \
--model_type=roberta \
--tokenizer_name=microsoft/codebert-base-mlm \
--model_name_or_path=microsoft/codebert-base-mlm \
--csv_store_path ./attack_genetic_400_800.csv \
--base_model=microsoft/codebert-base-mlm \
--use_ga \
--train_data_file=../preprocess/dataset/train_subs.jsonl \
--eval_data_file=../preprocess/dataset/test_subs_400_800.jsonl \
--test_data_file=../preprocess/dataset/test_subs.jsonl \
--block_size 512 \
--eval_batch_size 64 \
--seed 123456 > attack_gi_400_800.log 2>&1 &
I completed the test on 2732 sets of test data, but it was found that ASR was higher than the data in the report: 53.62% in the report, and I got about 65.19%. The calculation method is to use the Successful items count / Total count in the log.
I'm wondering if this is a bug in my process or some experimental error. Thanks!
Hello,
When I run this command
python get_substitutes.py \ --store_path ./data_folder/processed_gcjpy/valid_subs.jsonl \ --base_model=microsoft/codebert-base-mlm \ --eval_data_file=./data_folder/processed_gcjpy/valid.txt \ --block_size 512
in the /attack-pretrain-models-of-code/CodeXGLUE/Authorship-Attribution/dataset directory I got this error
FileNotFoundError: [Errno 2] No such file or directory: './data_folder/processed_gcjpy/cached_valid.pkl'
In the get_example_batch function of run_parser.py, there is an error in the substitution algorithm for variable names. In your code, the problem of variable name length inconsistency before and after substitution is handled by accumulating differences, but only if the row is replaced sequentially from left to right. Failure to do so results in substitution misplaced code that produces syntax errors.
Your code in get_example_batch:
for index, pos in enumerate(replace_pos[line]):
code[line] = code[line][:pos[3]+diff] + pos[1] + code[line][pos[4]+diff:]
diff += pos[2]
Bug fixed code in get_example_batch:
for index, pos in enumerate(list(sorted(replace_pos[line], key=lambda x:x[3]))):
code[line] = code[line][:pos[3]+diff] + pos[1] + code[line][pos[4]+diff:]
diff += pos[2]
In GA-Attack, when counting the number of variables that need to be changed for a successful mutation, old_word == mutant[old_word] is used, is there a problem?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.