Evaluation software for BioNLP-ST 2016 and BioNLP-OST SeeDev and Bacteria Biotopes tasks.

• BioNLP-OST 2019 website
  • Bacteria Biotopes 2019 task
  • SeeDev task
• BioNLP-ST 2016 website

The current version contains the data for evaluating on training and development sets for SeeDev and BB 2019 tasks. For the evaluation on the test set of the BB 2016, please use the online service.

• Java >= 8
• Apache Maven >= 3.3

git clone https://github.com/Bibliome/bionlp-st.git
cd bionlp-st
mvn clean install

Maven produces a JAR file in bionlp-st-core/target/bionlp-st-core-0.1.jar.

java -jar bionlp-st-core-0.1.jar -help
java -jar bionlp-st-core-0.1.jar -list-tasks
java -jar -task TASK -train|-dev -prediction PRED [-alternate] [-detailed] [-force]

where

• TASK is the short name of the task. Available tasks are:
  • SeeDev-full
  • SeeDev-binary
  • BB19-norm
  • BB19-rel
  • BB19-kb
  • BB19-norm+ner
  • BB19-rel+ner
  • BB19-kb+ner
  • BB-cat
  • BB-event
  • BB-kb
  • BB-cat+ner
  • BB-event+ner
  • BB-kb+ner
• PRED is the location of your predictions (.a2 files), either a directory of a ZIP archive.
• Specify -alternate to display several additional measures.
• Specify -detailed to display a document-per-document evaluation, including reference-prediction pairings.
• Specify -force to evaluate even if errors where found.