openmatch / ance-tele Goto Github PK

Hi Sun Si,
Congrats on your amazing job! Both the idea and results are very impressive.

I have one question on how you construct the "Tele-neg" set for each query. From equations (4) (5) (6) of your paper, it seems like you create the new "Tele-neg" set by merging "ANCE-neg", old "Tele-neg", and "lookahead-neg". The sampling coefficient for "ANCE-neg" and "lookahead-neg" are \alpha and \beta, and both of them are equal to 0.5 (the best hyper-parameter you mention in the paper).

But from your code preprocess/combine_marco_negative.py, it seems like you just directly merge two sets without any sampling coefficient. Can I know which part of your code contains the sampling coefficients for "ANCE-neg" and "lookahead-neg"?

Best wishes,
Hansi Zeng

I got this error when trying to run ANCETele with my data.

ValueError: Trainer: evaluation requires an eval_dataset.

When I printed eval_dataset in train_dr.py, None came out. Is this the error that occurred?
My data is a bit small. Is the problem caused by the data being too small?
If the problem arises due to the characteristics of the data, please let me know how to supplement the data.

eval_dataset = train_dataset_cls(
    tokenizer, 
    data_args, 
    is_eval=True, 
    cache_dir=data_args.data_cache_dir or model_args.cache_dir
) if data_args.eval_path is not None else None
print("eval_dataset ", eval_dataset)

eval_dataset None

And I printed queries, positives, and qid in combine_negative.py, and are these three variables shaped as follows?

with open(file_2, "r", encoding="utf-8") as fi,
open(output_file, "w", encoding="utf-8") as fw:
for line in tqdm(fi):
data = json.loads(line)
query = data["query"]
positives = data["positives"]
qid = "_".join(str(ids) for ids in query)
print("query")
print(query)
print("positives")
print(positives)
print("qid")
print(qid)

query
[20808, 23500, 1498, 4373, 2371, 13964, 28059, 2052, 22819, 1513, 2259, 2180, 555, 2227, 2182, 18, 1545, 2116, 3691, 2371, 4000, 3669, 2052, 4049, 2496, 2259, 2332, 18119, 35]
positives
[[17, 2, 3907, 2079, 8936, 23999, 2138, 5418, 2173, 20808, 23500, 2021, 2119, 1537, 3747, 27135, 2079, 10502, 3766, 2259, 7365, 6041, 2069, 4036, 2530]]
qid
20808_23500_1498_4373_2371_13964_28059_2052_22819_1513_2259_2180_555_2227_2182_18_1545_2116_3691_2371_4000_3669_2052_4049_2496_2259_2332_18119_35

In the code below, it only enters "if" and "else" never enters, so diff_num is 0 even after the for statement is over, is this related to the absence of eval_testset?
How should we solve the problem?
if qid in query_data_dict:
print("if qid in query_data_dict:")
f1, offset = query_data_dict[qid]
f1.seek(offset, 0)
data1 = json.loads(f1.readline())
negatives = data["negatives"] + data1["negatives"]
neg_num_list.append(len(negatives))
else:
print("else")
negatives = data["negatives"]
diff_num += 1