A brief tutorial that uses Keras to build a basic Recurrent Neural Network Language Model and uses it generate new sentences in stories.

You'll need IPython/Jupyter to run the notebook (keras_rnn_generation.ipynb) or alternatively the regular Python file is provided. To run the code you'll need Keras, as well as either TensorFlow or Theano as the backend for Keras (the most recent version of Keras uses TensorFlow by default so you'll need to configure it if you want to use Theano). The other requirements are Numpy and h5py.

The dataset used in this tutorial is the ROCStories corpus. You'll just need to complete a form to get access to the full dataset, but a sample of it is contained in example_train_stories.csv and example_test_stories.csv. The tutorial loads these files so just swap them out with the ROC dataset files when you download them (you'll need to do just a few lines of pre-processing to remove the header and IDs).

In given code with the example datasets, the tokenizer and trained model weights are saved to 'example_tokenizer.pkl' and 'example_rnn_weights.h5', respectively. However, for demonstrating generation, I load the weights from an already trained model (too large to fit on Github), rnn_weights_96000.h5, that was trained on 96,0000 stories in the ROCStories corpus. The accompanying tokenizer for pre-processing is tokenizer_96000.pkl. You should be able to download these files and run the generation code as is, or of course just swap them out with your own trained model files.