Giter Club home page Giter Club logo

text-classification-models-tf's Introduction

Text Classification Models with Tensorflow

Tensorflow implementation of Text Classification Models.

Implemented Models:

  1. Word-level CNN [paper]
  2. Character-level CNN [paper]
  3. Very Deep CNN [paper]
  4. Word-level Bidirectional RNN
  5. Attention-Based Bidirectional RNN [paper]
  6. RCNN [paper]

Semi-supervised text classification(Transfer learning) models are implemented at [dongjun-Lee/transfer-learning-text-tf].

Requirements

  • Python3
  • Tensorflow
  • pip install -r requirements.txt

Usage

Train

To train classification models for dbpedia dataset,

$ python train.py --model="<MODEL>"

(<Model>: word_cnn | char_cnn | vd_cnn | word_rnn | att_rnn | rcnn)

Test

To test classification accuracy for test data after training,

$ python test.py --model="<TRAINED_MODEL>"

Sample Test Results

Trained and tested with dbpedia dataset. (dbpedia_csv/train.csv, dbpedia_csv/test.csv)

Model WordCNN CharCNN VDCNN WordRNN AttentionRNN RCNN *SA-LSTM *LM-LSTM
Accuracy 98.42% 98.05% 97.60% 98.57% 98.61% 98.68% 98.88% 98.86%

(SA-LSTM and LM-LSTM are implemented at [dongjun-Lee/transfer-learning-text-tf].)

Models

1. Word-level CNN

Implementation of Convolutional Neural Networks for Sentence Classification.

2. Char-level CNN

Implementation of Character-level Convolutional Networks for Text Classification.

3. Very Deep CNN (VDCNN)

Implementation of Very Deep Convolutional Networks for Text Classification.

4. Word-level Bi-RNN

Bi-directional RNN for Text Classification.

  1. Embedding layer
  2. Bidirectional RNN layer
  3. Concat all the outputs from RNN layer
  4. Fully-connected layer

5. Attention-Based Bi-RNN

Implementation of Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification.

6. RCNN

Implementation of Recurrent Convolutional Neural Networks for Text Classification.

References

text-classification-models-tf's People

Contributors

dongjun-lee avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

text-classification-models-tf's Issues

Got this trouble running on Google Cloud Platform TPU

Tensorflow 2.x
Python 3.7.3

Traceback (most recent call last):
File "train3.py", line 74, in
model = WordCNN(vocabulary_size, WORD_MAX_LEN, NUM_CLASS)
File "/home/migueltuxd/bucket4testingtpuss/TPU/text-classification-models-tf-master/cnn_models/word_cnn.py", line 54, in init
self.optimizer = tf.train.AdamOptimizer(self.learning_rate).minimize(self.loss, global_step=self.global_step)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/optimizer.py", line 413, in minimize
name=name)
File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/training/optimizer.py", line 564, in apply_gradients
raise RuntimeError("Use _distributed_apply() instead of "
RuntimeError: Use _distributed_apply() instead of apply_gradients() in a cross-replica context.

Shared too on stackoverflow:https://stackoverflow.com/questions/61704387/wordcnn-trouble-with-distributed-apply-and-apply-gradients

data_utils

data_utils.py function build_char_dataset

some seem not very reasonable:

when model = "char_cnn":  
    char_dict["<pad>"] = char_dict["<unk>"] = char_dict["a"] = 0 after onehot in char_cnn.py

when model = "vd_cnn": 
    char_dict["<pad>"] = char_dict["a"] = 0, 
    char_dict["<unk>"] = char_dict["b"] = 1

What version of Tensorflow the code runs with?

It crashes in Tensorflow 1.4 with the error below:

Traceback (most recent call last):
File "train.py", line 47, in
model = VDCNN(alphabet_size, CHAR_MAX_LEN, NUM_CLASS)
File "text-classification-models-tf/cnn_models/vd_cnn.py", line 70, in init
tf.nn.softmax_cross_entropy_with_logits_v2(logits=self.logits, labels=y_one_hot))
AttributeError: module 'tensorflow.python.ops.nn' has no attribute 'softmax_cross_entropy_with_logits_v2'

Word-level CNN model's result is poor in SST-2 dataset.

Have you test the SST-2 dataset ?
I have run word_cnn model on SST-2 dataset . Only get 0.38 accuracy. But the accuray is 0.45 in paper.
How to change the model to improve the accuracy? Is something wrong?
The word_rnn model can get 0.45 accuracy, why diff is so large?

ResourceExhaustedError: OOM

find a error when train the att_rnn model

Machine: 4 * Tesla P100-PCIE-16GB, memory: 256G

ResourceExhaustedError (see above for traceback): OOM when allocating tensor of shape [563354,256] and type float [[Node: embeddings/Adam/Initializer/zeros = Const[dtype=DT_FLOAT, value=Tensor<type: float shape: [563354,256] values: [0 0 0]...>, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.