A Pytorch LSTM Language model trained on Penn Tree Bank Dataset.

• Introduction
• Setup
• Run the code
• Training
• Sample Generation
• Play with the model
• Connect with me
• License

This is a PyTorch Implementation for an LSTM-based language model. The model is trained on Penn Tree Bank dataset using Adam optimizer with a learning rate 0.001 and 10 epochs.

The code is using pipenv as a virtual environment and package manager. To run the code, all you need is to install the necessary dependencies. open the terminal and type:

• git clone https://github.com/Khamies/LSTM-Language-Generator.git
• cd LSTM-Language-Generator
• pipenv install

And you should be ready to go to play with code and build upon it!

• To train the model, run: python main.py

• To train the model with specific arguments, run: python main.py --batch_size=64. The following command-line arguments are available:

  • Batch size: --batch_size
  • bptt: --bptt
  • Learning rate: --lr
  • Embedding size: --embed_size
  • Hidden size: --hidden_size
  • Latent size: --latent_size

The model is trained on 10 epochs using Adam as an optimizer with a learning rate = 0.001 and batch size = 32, you can find all the model settings in settings.py. Here is the loss curve for the training step:

• Negative Likelihood Loss

  

Here are some generated samples from the model:

he said <pad> is n't expected to be the first quarter of
the company said will be used to repay the company's

To play with the model, a jupyter notebook has been provided, you can find it here

@misc{Khamies2021LSTM-Language-Generator,
author = {Khamies, Waleed},
title = {A PyTorch Implementation for an LSTM-based language model},
year = {2021},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/Khamies/LSTM-Language-Generator}},
}

For any question or a collaboration, drop me a message here

Follow me on Linkedin!

Thank you ❤️