Yahoo Hack Day 2013 - Dream chaser

We have cooked up a number of recipes for working with open data.

First we demonstrate using simple crawling tools to get large amounts of open data.

Then we show how to use YUI tools to explore open data.

Then we use YUI to perform simple schema integration.

Next we create a simple Python tool to modify the JSON data so it can be imported into Exhibit, a flexible data visualization tool. We demonstrate this approach for a number of data sets. We use Bootstrap to create a rich front end to enhance Exhibit.

Finally we demonstrate how some simple tools can be used to convert some JSON datasets to CSV, then import the data into a relational database, and then use the D2RQ tool to expose this data as RDF via SPARQL making the open data available to the Semantic Web. We explore data modelling and schema mapping issues.

We documented our work using Jekyll, a static web framework which is used as the basis of Github pages. The documentation is important because this provides a hacking cook book for working with open data.