An analysis of how the European refugee crisis is trending on Reddit

-- data collection (with PRAW) and analysis (with pandas) by Anca Chereches; data visualization (with D3) by Natalia Chetelat --

We're trying to understand public interest and public opinion in the European migrant crisis by analyzing Reddit activity on relevant subreddits. For now, we're focusing on /r/europe, but we hope to add other relevant subreddits to the analysis later.

grab-submissions.py harvests the raw dataset.

The current raw dataset, harvested on Sept. 21st 2015, contains all submissions to /r/europe created between Jan. 1st 2015, 00:00:00 GMT and Sept. 13th 2015, 00:00:00 GMT. This dataset is available on my website as a Pickle file. It can be loaded into Python like so:

>>> import pickle
>>> with ('PATH_TO_FILE/r-europe_posts.pkl','r') as f:
>>>     posts = pickle.load(f)

posts is an unordered list of praw objects.

>>> posts[0].title
>>> u'Greeks\u2019 view of the debt crisis: \u2018What lies ahead is great, great hardship\u2019'
>>> len(posts)
>>> 23732

preprocessing.py reads in the pickled raw dataset and creates a pandas DataFrame.

analysis.ipynb is an ipython notebook which loads the preprocessed dataset and does some data wrangling.

analysis.py loads the preprocessed dataset and writes to disk a number of .csv files which will be used for D3 visualizations.