Implementation for Convolutional Neural Networks for Sentence Classification of Kim (2014) with PyTorch and Torchtext.

• rand: All words are randomly initialized and then modified during training.
• static: A model with pre-trained vectors from word2vec. All words -- including the unknown ones that are initialized with zero -- are kept static and only the other parameters of the model are learned.
• non-static: Same as above but the pretrained vectors are fine-tuned for each task.
• multichannel: A model with two sets of word vectors. Each set of vectors is treated as a 'channel' and each filter is applied to both channels, but gradients are back-propagated only through one of the channels. Hence the model is able to fine-tune one set of vectors while keeping the other static. Both channels are initialized with word2vec.# text-classification-cnn Implementation for Convolutional Neural Networks for Sentence Classification of Kim (2014) with PyTorch.

To run the model on [SST-1] dataset on [multichannel](Model Type), just run the following code.

python train.py --mode multichannel

The file will be saved in

saves/best_model.pt

To test the model, you can use the following command.

python main.py --trained_model saves/best_model.pt --mode multichannel

Dataset is in data dir. You can set your own path in SST1.py and args.py for data and embeddings. We experiment the model on the following three datasets.

• SST-1: Keep the original splits and train with phrase level dataset and test on sentence level dataset.

For word2vec.sst-1.pt is a subset of word2vector. We just select the word appearing in the SST-1 dataset and generate this file with the vector_preprocess.py. You can select these from any kind of word embedding text file and generate in following format.

word vector_in_one_line 

and then run

python vector_preprocess.py file_in embed.pt

Here you can get embed.pt for the embedding file. Remember change the argument in args.py file with your own embedding.

Adadelta is used for training.

For training time, when

torch.backends.cudnn.deterministic = True

is specified, the training will be ~3h because deterministic cnn algorithm is used (accuracy v.s. speed).

Other option is that

torch.backends.cudnn.enabled = False

but this will take ~6-7x training time.

Deterministic Algorithm for CNN.

• More experiments on SST-2 and subjectivity
• Parameters tuning