Dataset cited paper in acl/pdf_parses.jsonl about corwa HOT 6 CLOSED

20200705v1/acl/pdf_parses.jsonl is simply the filtered version of 20200705v1/full/pdf_parses/* by only selecting those papers with ACL IDs. CORWA-train and -test are human-annotated, and they are selected by taking those related work sections that have the fewest number of missing cited papers that are not covered by S2ORC. All the rest of the S2ORC ACL papers are in the distant set. The training and test set is split by year (before & after 2019).

Originally I hesitated to share related_work.jsonl files because they are a simple derivation of S2ORC's 20200705v1/full/pdf_parses/*, but eventually all this information is included in the latest released CORWA_{train, test, distant}.jsonl files.

from corwa.

Note that many wired file formats exist because of historical reasons during my project development. Unless you are interested in replicating my exact experiment, you can start from data/CORWA_{train, test, distant}.jsonl for your custom usage. You can use the code under pipeline/ to tag your own related work sections with CORWA annotations.

from corwa.

Thank you for your reply. I am also interested in testing my model on your test dataset, and making comparison with your finetuned LED. Would you please provide more information on how the ROUGE score of LED generation reported in your paper was obtained? For each span in the test dataset, if one or more references are not in S2ORC or ACL of S2OCR, did you just leave them out?

from corwa.

@mandalbiswadip Can you resolve this request? Thanks.

from corwa.

I have updated the evalution script under experiments/evaluate_LED.ipynb. Let me know if you find this helpful

from corwa.

Horray! That looks great!

from corwa.