Giter Club home page Giter Club logo

pytorch-ganbert's Introduction

Pytorch-GANBERT

A Pytorch implementation of GAN-BERT paper

"bert.py" and "qc-fine_bert.py" files are basically the same with different datasets; this is just my laziness. "labeled_and_unlabeled.tsv" is the mixed version of labeled and unlabeled data from original tensorflow implementation of the paper.

Important Note

Running this model only once with only one seed might not be enough. In my experiments, out of 10 runs, only 2 yielded reasonable results. I believe this is due to the difference of scheduling of the optimizers between this implementation and the original TensorFlow implementation.

pytorch-ganbert's People

Contributors

osmanmutlu avatar

Stargazers

Affandy Fahrizain avatar Duchengyuan avatar

Watchers

James Cloos avatar  avatar

pytorch-ganbert's Issues

Question

Hiii Osman,

Thank you for your clear implementation, its really helpful!
My question is there anyway that we can get the generator's output as Text?
Is it possible to see what exactly it is the generating?

Discriminator predicts only one class for all samples

Although the loss gets smaller when training, in testing model only predict one class for the entire test set.

The only exception was when I was training with my data (balanced binary classification data) after ossilating between the two classes through epochs (always achieving 50% on dev), in epoch 20 the model finally predicted different classes for different samples and achieved a score around 60% on development set. I could not replicate this phenomenon again.

some questions for code

Hi, thanks for the excellent implementation of pytorch version GAN-BERT! I have 3 questions for the code:

  1. Generator in GAN-BERT is a MLP, in model.py it makes the following transformation of noise's dimension: 100 -> 512 -> 512, why just only 100 -> 512?
  2. Why should we import parameters of TensorFlow version in def convert_to_tf_param_name and def get_weights_from_tf of model.py?
  3. In general GAN, we often train discriminator for several times then train generator once in one epoch. But in ganbert.py, it seems you train discriminator only once then train generator once. Will this operation cause the problem mentioned in issue#1?

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.