andrewfm / mlfinal Goto Github PK

Read:
ICWSM – A Great Catchy Name: Semi-Supervised Recognition of Sarcastic Sentences in Online Product Reviews
http://www.aaai.org/ocs/index.php/ICWSM/ICWSM10/paper/viewFile/1495/1851

Sarcasm as Contrast between a Positive Sentiment and Negative Situation
http://www.cs.utah.edu/~riloff/pdfs/official-emnlp13-sarcasm.pdf

Irony and Sarcasm: Corpus Generation and Analysis Using Crowdsourcing
http://www.lrec-conf.org/proceedings/lrec2012/pdf/661_Paper.pdf

Semi-Supervised Recognition of Sarcastic Sentences in Twitter and Amazon
http://www.aclweb.org/anthology/W10-2914

Read: http://cs.stanford.edu/~quocle/paragraph_vector.pdf

In the Kaggle tutorial, they used a few basic methods to clean the input; they lowercased the text, stripped out any HTML and punctuation, and (optionally) removed all stopwords.

There other examples of things we can do to reduce the diversity of the input. We could use a stemmer (such as the Porter Stemmer) to reduce words to their root form. We could also compress synonymous words together using a thesaurus, such as WordNet.

The python library NLTK (http://www.nltk.org/) provides access to both WordNet and stemmers.

(To whoever chooses to work on this one: This one should be pretty quick and simple to implement, comparatively to the other tasks, so I also suggest doing some research and finding at least one other small improvement that can be made to the project, and also work on that)

Baseline Score: 0.84392

Increasing the number of data-points we train on may likely improve performance. (However, this should be the last thing we do, after we get everything else implemented and working)