Giter Club home page Giter Club logo

a-pytorch-tutorial-to-text-classification's Introduction

This is a PyTorch Tutorial to Text Classification.

This is the fourth in a series of tutorials I plan to write about implementing cool models on your own with the amazing PyTorch library.

Basic knowledge of PyTorch, recurrent neural networks is assumed.

If you're new to PyTorch, first read Deep Learning with PyTorch: A 60 Minute Blitz and Learning PyTorch with Examples.

Questions, suggestions, or corrections can be posted as issues.

I'm using PyTorch 1.1 in Python 3.6.


27 Jan 2020: Working code for two new tutorials has been added — Super-Resolution and Machine Translation


Contents

Objective

Concepts

Overview

Implementation

Training

Inference

Frequently Asked Questions

Objective

To build a model that can label a text document as one of several categories.

We will be implementing the Hierarchial Attention Network (HAN), one of the more interesting and interpretable text classification models.

This model not only classifies a document, but also chooses specific parts of the text – sentences and individual words – that it thinks are most important.


"I think I'm falling sick. There was some indigestion at first. But now a fever is beginning to take hold."


"But think about it! It's so cool. Physics is really all about math. What Feynman said, hehe."


"I want to tell you something important. Get into the stock market and investment funds. Make some money so you can buy yourself some yogurt."


"How do computers work? I have a CPU I want to use. But my keyboard and motherboard do not help."

"You can just google how computers work. Honestly, its easy."


"You know what's wrong with this country? Republicans and democrats. Always at each other's throats"

"There's no respect, no bipartisanship."


Tutorial in Progress

I am still writing this tutorial.

In the meantime, you could take a look at the code – it works!

We achieve an accuracy of 75.1% (against 75.8% in the paper) on the Yahoo Answer dataset.

a-pytorch-tutorial-to-text-classification's People

Contributors

sgrvinod avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

a-pytorch-tutorial-to-text-classification's Issues

about the pack&pad operation

Hi, sgrvinod, I've learned a lot from your tutorials.

The usage of pack&pad operation can eliminate redundant padding calculations in the sequence, which is verified by the tutorial. But I'm curious about how to deal with it if a CNN layer is concatenated after it ?

If we just pad_packed_sequence the packed results from RNN/linear layers, then how can we deal with the padding=0? Use another mask operation or any other methods?

Hope for your reply, thanks!

How is trained model used for validation model?

Hello,

I am currently trying to improve the HAN model and this is the best implementation of code I found on github. So first a fall, thank you for that.

As I drove deep into the code, I could not figure out one thing. In machine learning generally, we use a model to train and then use that model for validation. However, in train.py in line 110 I found below code

train(train_loader=train_loader,
            model=model,
            criterion=criterion,
            optimizer=optimizer,
            epoch=epoch)

      # One epoch's validation
      acc = validate(val_loader=val_loader,
                     model=model,
                     criterion=criterion)


Now as I understand first the training happens then validation happens. My question is shouldn't this model be saved somewhere and then validation must occur using that model. Right now, the training model is trained seperately and testing model is tested seperately.

For example, In your classify.py code, you have loaded model from best_checkpoint. Shouldn't the training and testing happen like that?

Hoping for your reply

/content/a-PyTorch-Tutorial-to-Text-Classification/utils.py:184: SyntaxWarning: "is" with a literal. Did you mean "=="? sg = 1 if algorithm is 'skipgram' else 0 Traceback (most recent call last): File "/content/a-PyTorch-Tutorial-to-Text-Classification/train.py", line 12, in <module> with open(os.path.join(data_folder, 'word_map.json'), 'r') as j: FileNotFoundError: [Errno 2] No such file or directory: '/media/ssd/han data/word_map.json'

Running train.py
I thought you said this code works? Would you please give some instructions?

Some Encourage

Your code is great to me! I just want to encourage you from my bottom of my heart. And thank you!

Order of files to be run

Hello,

First a fall, thank you for creating this tutorial. I am a relatively newbie to python programming. So,

This might sound embarrasing. But which file should I run first?

Will u finish this tutorial?

Your tutorials are too good to be true, will u finish this text classification tutorial? I really enjoy reading your repo!

Encoding and padding training data takes too long.

I am runing this model on the kaggle Fake news dataset.

The preprocess stage is taking a lot of time.
Specifically, it keeps on running on the "Encoding and Padding Data" step or sometimes gets killed due to out of memory error.

Is this because of the large training data ?

I have set both the sentence_limit and word_limit to be 105 .

Regarding the changed accuracy

Hey Mr Sagar,

I just noticed that you found a new accuracy of 75.1%. Previously it was 74.8%. I tried comparing the changes in new files with previous files but I could not pin point how the accuracy changed by 0.3%. Did the execution of decrease of learning rate made the difference? Can you please point me out the change you think you made to get this accuracy difference?

Thank you

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.