imclab / data.okfn.org Goto Github PK

The code behind http://data.okfn.org.

Core is an ultra-lightweight Data Hub running off data packages.

Installing

Requires NodeJS (>= v0.8).

Install the requirements (in package.json):

npm install .

Run the web application locally:

node app.js

If you want to customize the catalog file you run off set the environment variable:

export CATALOG_URL={url-to-your-data-package-catalog}

A data package catalog should be a list of data package urls, one per line. Here's an example (used to run data.okfn.org):

https://raw.github.com/datasets/registry/master/datapackage-list.txt

Deployment to Heroku

Note that you will need to install the Heroku toolbelt to carry out the heroku commands below.

Create the application on Heroku

Skip this step if the application is already deployed

This is heavily based on this tutorial.

# replace {app-name} with the name of your app
heroku create {app-name}
git push heroku master
# now point at our config
heroku ps:scale web=1

To deploy (already created):

Push to heroku git repo:

git push heroku master

Note: if didn't do the create you will need to add the heroku remote:

git remote add heroku [email protected]:{app-name}.git

Adding collaborators:

heroku sharing:add [email protected]

Setting the domain name

Do the following:

heroku domains:add {your-domain-name}

Now CNAME your domain to {myapp}.herokuapp.com

Design and Implementation Notes

• Very simple. Just one or two simple web pages

• DataStore and API

  • May be optional for MVP?
  • SQL-backed (Use "SqlStore" stuff from CKAN?)
  • JSON RESTful API
  • Refine reconciliation support
  • Build on CKAN DataStore ...

• Web interface

  • Big list on the front page
  • per-datasets page /{dataset-name}-{id}/...
  • Scrollable table of the data with search (?)
  • Links to download of CSV and full API instructions
  • (?) URL for each element /{dataset}/id/{element-id}/
  • Add .json, .{...} for each item

How to implement?

• Pure JS (or Node or Flask?)

• Why Not Use CKAN?

  • We may use CKAN at some point BUT to start with ...
  • Want very simple structure (just a few pages)
  • Preference for JS only (lots of browser JS)
  • Don't need CKAN's full power
  • CKAN somewhat more complex to deploy