The qa-transfer from ml-lab

Question Answering through Transfer Learning

This is the original implementation of "Question Answering through Transfer Learning from Large Fine-grained Supervision Data". [paper] [poster]
Most parts were adapted & modified from "Bi-directional Attention Flow". [paper] [code]
Evaluation scripts for SemEval were adapted & modified from SemEval-2016 official scorer.
Please contact Sewon Min (email) for questions and suggestions.

Codes include

General

Python Packages

First, download data (SQuAD, WikiQA, SemEval-2016, GLoVe, NLTK). This will download files to $HOME/data

chmod +x download.sh; ./download.sh

Then, pretrain the model on SQuAD.

chmod +x pretrain.sh
./pretrain.sh span 		# to pretrain BiDAF on SQuAD
./pretrain.sh class		# to pretrain BiDAF-T on SQuAD-T

You can use trained model from original BiDAF code. Just place saved directory to out/squad/basic/00.

Finetune the model on WikiQA / Semeval.

chmod +x train.sh; ./train.sh DATA finetune RUN_ID PRETR_FROM STEP

DATA: [wikiqa | semeval]
RUN_ID: run id for finetuning. use unique run id for the same data.
PRETR_FROM: [basic | basic-class]. use basic for span-level pretrained model, and basic-class for class-level pretrained model.
STEP: global step of pretrained data. For a quick start, use 18000 for span-level pretrained model and 34000 for class-level pretrained model. However, monitoring tensorboard and pick the best global step is recommended, because results would depend much on the quality of pretrained model.

Finally, evaluate your model.

chmod +x evaluate.sh; ./evaluate.sh DATA RUN_ID START_STEP END_STEP

This is just for a quick tutorial. Please take a look at run.md for details about running the code.