Creative Automatic Title from Synopsis generator

Authors: Jonas Molz, Albert Negura

This is a project done as part of the (2018) Natural Language Processing Course at the University of Maastricht.

This system represents an automatic title generator which attempts to utilize various NLP techniques to create a new title for a movie based on the Synopsis / genres provided.

To do so, we first look at existing movie synopses and attempt to do some genre-based classifcation. Using this, a mapping for each word in the title to each genre in generated, where the frequency of the word appearing in the respective genres is used as a way to weigh the word.

This weight is then intended to be passed as an input to an RNN (LSTM) as described in Konstantin Lopryrev's paper. This RNN takes an input text (summary, synopsis, etc.) and encodes it into a distributed representation. A decoder is then used to transform the representation into a title.

The system currently follows the flowchart on the left, and is intended to follow the flowchart on the right:

Datasets used are

Wikipedia: https://www.kaggle.com/jrobischon/wikipedia-movie-plots

MPST: https://www.kaggle.com/cryptexcode/mpst-movie-plot-synopses-using-tags

The summaries on the datasets contain a variety of words, with names being the most prominent for every movie.

Furthermore, some words (adverbs or articles such as 'the') have a much higher frequency than most other words (a few hundred fold increase in most cases).

The average length of movie titles in the dataset is about 2 words, while the average length of the Wikipedia summaries is 350 words.

The following notebooks are available in this repo:

1. CATS-MAIN.ipynb: contains all the preprocessing steps that have been / can be done with the data for this project. Furthermore, contains the Naive Bayes genre classifier.

2. GenrePredictor.ipynb: contains a Python 3 implementation of the Lopyrev model, which attempts to generate a title based on the synopsis. Was supposed to contain a genre-based approach to generating a headline.

3. vocabulary-embedding.ipynb: Python2: generates pickle dumps containing processed training data and initialized embedding. Code originally at https://github.com/udibr/headlines (this is a modified version).

4. train.ipynb: Python2: trains a network for Lopyrev's model, generating titles from the synopsis of movies, and saves the model. Code originally at https://github.com/udibr/headlines (this is a modified version).

5. predict.ipynb: Python2: generates predictions based on the saved Lopyrev model. Code originally at https://github.com/udibr/headlines (this is a modified version).

6. kar_et_al_tag_generator.ipynb: generates tags based on Kar's model, which can then be used in conjunction with the genre classifier to generate headlines.

For Genre Classification, MultinomialNB was the primary model used. It has high accuracy on most genres, with decreasing accuracy depending on how common the genre is in the training dataset used.

For generating titles, Lopyrev's model was used and trained on

Usually, training the models requires a significant amount of RAM. This message appeared far too often minutes or, sometimes, hours into the training. A beefy machine is necessary for Lopyrev's or Kar et. al's model.

Genre Classification can be overfit very easily, mostly due to class imbalance. Tag generation suffers from a similar effect, although partly because the network has been undertrained due to memory issues.

Handpicked Genres:

Example Output:

Best prediction for Genres: 

['sports/comedy', 'horror thriller', 'action, fantasy']

Actual Genres: 

['thriller']

Title: 

['Deadpool']

Note that the generated titles do not contain common English adverbs and articles (the, for, of, just, etc.).

Handpicked Titles:

Example Output of Title Generation Network:

(Actual Title) HEAD: gamera vs gyaos
(Processed Synopsis) DESC: series volcanoes^ erupt japan eruption mt futago^ shizuoka^ prefecture attracts gamera whose arrival witnessed young boy named eiichi gamera climbs volcano research team dispatched volcano find gamera study effects eruption meanwhile chuo^ expressway corporation building roadway^ nearby local villagers refuse leave research team helicopter destroyed sonic beam emitted^ cave mountains reporters informed bodies found culprit gamera volcanic eruption announcement made soon one reporters okabe leaves site roadcrew^ foreman shiro tsutsumi arrive protest area simultaneously okabe
(Generated Titles, higher is better) HEADS:
8.712603449821472 midnight
8.734845966100693 serena
10.19324865937233 night annihilates^
14.785994216799736 cat dream
15.782405376434326 happy jackson

'Mannix' is often generated as the title of the movie. A preliminary analysis reveals that 'Mannix' is the most dissimilar word to all others in the corpus, so the network decides to choose it when it cannot "think" of a good alternative.

All of the models / datasets / embeddings (+ figures + notebooks) can be downloaded from this link: https://drive.google.com/drive/folders/1Y_gj1PWEtBZbNSoc2VpmRhNXSalgD8sy?usp=sharing

A few RAM issues on training the model used.