The zeroshotwiki from cogcomp

zeroshotwiki's Introduction

Code and data for NAACL 2022 Demo: Towards Open Domain Topic Classification

Requirements

To install required Python packages, run

pip install -r requirement.txt

We recommend to create a new conda environment of Python 3.7 to avoid potential incompatibility.

Using our models

To download datasets/models on this page, please right-click on the links to "Save link as ..."

Users can download our pre-trained ESA-WikiCate and TE-WikiCate models (model.zip), and evaluation data (data.zip). To evaluate ESA_WikiCate, for example, run

python src/eval_ESA.py --data_path data/yahoo/test.txt --label_path data/yahoo/label_names.txt --esa_path model/ESA_WikiCate --n_jobs N_JOBS

To evaluate TE-WikiCate, for example, run

CUDA_VISIBLE_DEVICES=xxx python src/eval_TE.py --data_path data/yahoo/test.txt --label_path data/yahoo/label_names.txt --model_path model/TE_WikiCate/ --simple_hypo

For the multi-labeled Situation dataset, add --multi_label to the above two commands.

Training

We provide the data for training TE-WikiCate (wikipedia.zip), which is about 12GB. To train your own model, extract the zip file under data directory and run

CUDA_VISIBLE_DEVICES=xxx python src/train_wiki_entailment.py --data_file data/wikipedia/tokenized_wiki.txt --cate_file data/wikipedia/page2content_cate.json --save_dir SAVE_DIR

zeroshotwiki's People

Contributors

Stargazers

Watchers

zeroshotwiki's Issues

Use Bert-Wiki model with Transformer's pipeline for zero-shot text classification

Hi,
This work is amazing, I've been playing around with the demo as well and thank you for making all of this publicly available.
I was wondering if there's a way to integrate the Bert-Wiki model directly with Transformer's pipeline for zero-shot text classification (as shown here). I'm particularly interested in providing the class descriptions (along with the names) and using multi_label classification.
Thanks!

Cannot Download Training Data

Hi, thanks for making the repo for your interesting work public.
I would like to re-implement this for a class project but I could not access the link to download the wikipedia training data.
Please advise.

Trainning stops after a while

The training script stops at 3% each time.

getting error message when training

--:/shared/tiha/code_review/ZeroShotWiki> CUDA_VISIBLE_DEVICES=1 python src/train_wiki_entailment.py --data_file data/data/wikipedia/tokenized_wiki.txt --cate_file data/data/wikipedia/page2content_cate.json --save_dir "model2"
07/26/2022 16:40:32 - WARNING - transformers.modeling_utils - Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForSequenceClassification: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.transform.LayerNorm.bias']

This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPretraining model).
This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
07/26/2022 16:40:32 - WARNING - transformers.modeling_utils - Some weights of BertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier.weight', 'classifier.bias']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Traceback (most recent call last):
File "src/train_wiki_entailment.py", line 451, in
main()
File "src/train_wiki_entailment.py", line 351, in main
if os.path.exists(train_loader_file):
File "/shared/tiha/miniconda3/envs/zsw/lib/python3.7/genericpath.py", line 19, in exists
os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType

Recommend Projects