This repository test the Bert model on the CLOTH Dataset. The framework of implementation is provided by the Pytorch Pretrained Bert repo.

The finetuned Bert model reach 86.0% accuracy on the test set, which is the same as the performance of Amazon Turkers. Leaderboard

The code is tested with Python3.6 and Pytorch 0.4.0

You can require the dataset from our website, and decompress it in the root of this repo.

First step is to preprocess data by running

python data_util.py

You should assign the name of the Bert model that you want to use in the file. It is "bert-large-uncased" by default.

You can run the model with the best hyper-parameter setting by

./run.sh