Giter Club home page Giter Club logo

azavea / open-data-catalog Goto Github PK

View Code? Open in Web Editor NEW
243.0 51.0 90.0 1.53 MB

Open Data Catalog is an open data catalog based on Django, Python and PostgreSQL. It was originally developed for OpenDataPhilly.org, a portal that provides access to open data sets, applications, and APIs related to the Philadelphia region. The Open Data Catalog is a generalized version of the original source code with a simple skin. It is intended to display information and links to publicly available data in an easily searchable format. The code also includes options for data owners to submit data for consideration and for registered public users to nominate a type of data they would like to see openly available to the public.

Home Page: http://www.opendataphilly.org/

License: MIT License

Python 72.63% ASP 0.02% JavaScript 19.45% CSS 7.89%

open-data-catalog's Introduction

Prep your dev environment

Install virtualbox: https://www.virtualbox.org/
Build a Linux VM from Ubuntu 12.04: http://www.ubuntu.com/

Installing the Open Data Catalog

Non-Python Dependencies

    sudo apt-get update
    sudo apt-get install git sendmail postgresql python postgresql-plpython-9.1 python-pip libpq-dev python-dev libxml2-dev libxslt1-dev libgeos-c1 mercurial meld

Python Dependencies

    sudo pip install virtualenv

Setting up the App

Create Virtual Env

Create the shell of the virtual env and "activate" it

    virtualenv opendatacatalog
    cd opendatacatalog
    source bin/activate

Grab the source

At this point you can grab the source from github:

    git clone git://github.com/azavea/Open-Data-Catalog.git
    cd Open-Data-Catalog

Or fork the code, make it better and send us a pull request!

Grab the python dependencies and do some final setup:

    pip install -r requirements.txt
    cd OpenDataCatalog
    mkdir media
    chmod 755 media
    ln -s ../../lib/python2.7/site-packages/django/contrib/admin admin_media

Setting up the database

Create a new postgres db

    sudo su postgres # Become the postgres user
    createuser -P odc-user
    psql template1 -c "CREATE DATABASE opendata OWNER \"odc-user\";"
    createlang plpythonu opendata
    psql -d opendata -f etc/pycsw_plpythonu.sql
    exit # Exit out of the postgres user's shell

Note that running the unit tests requires that your postgres user be a super user (since it drops/creates databases). To create a user as a superuser simply do:

    createuser -Ps odc-user

You can verify the connection with:

    psql opendata odc-user -h localhost

Update settings

Copy local_settings.py.example to local_settings.py

Update the database settings in local_settings.py. You'll probably have to update "name", "user", "password", and "host". It should look similar to:

    DATABASES = {
       'default': {
           'ENGINE': 'django.db.backends.postgresql_psycopg2',
           'NAME': 'opendata',
           'USER': 'odc-user',
           'PASSWORD': 'PASSWORD',
           'HOST': 'localhost',
           'PORT': '',
       }
    }

Open Data Catalog supports the [OGC Catalogue Service] (http://www.opengeospatial.org/standards/specifications/catalog) specification (CSW) using pycsw. CSW settings can be set/modified in settings.CSW. As well, settings.SITEHOST and settings.SITEPORT must be set accordingly for your deployment environment.

Creating the database scheme

To create the scheme we use django "syncdb" command

    python manage.py syncdb --migrate

Running a server

We installed gunicorn as part of the installation process. All you need to do now is start it:

    gunicorn_django

Customizing your installation

Settings

You should read OpenDataCatalog/local_settings.py for a list of configurable values, such as TWITTER_USER and SITE_ROOT. Feel free to play with these settings as much as you want.

Style

In local_settings.py you should specify a LOCAL_TEMPLATE_DIR that is the full path to your template location. As you want to change the style, you copy files out of OpenDataCatalog/templates and into your local overly directory and make the changes there. That way you can keep your repository in sync with the upstream more easily while still maintaining your own style.

Deploy to Heroku

For a quick and free deployment you can deploy directly to heroku (http://heroku.com). First make an account on the heroku website and then do the following:

Edit the .gitignore file and remove the following line:

    local_settings.py

Ths will allow your personal settings from being pushed to heroku. Then:

    sudo gem install heroku       
    heroku create --stack cedar --buildpack https://github.com/cirlabs/heroku-buildpack-geodjango/
    git push heroku master       
    heroku run python OpenDataCatalog/manage.py syncdb

Apache

Django can run via mod_wsgi on Apache as well. Add the following to a new Apache site:

    WSGIScriptAlias /hidden /<project location>/odp.wsgi
    Alias /media /<project location>/media
    Alias /static /<project location>/static

    create /<project location>/odp.wsgi >
    import os, sys
    sys.path.insert(0, '/home/azavea/NPower_OpenDataPhilly')

    import settings

    import django.core.management
    django.core.management.setup_environ(settings)
    utility = django.core.management.ManagementUtility()
    command = utility.fetch_command('runserver')

    command.validate()

    import django.conf
    import django.utils

    django.utils.translation.activate(django.conf.settings.LANGUAGE_CODE)

    import django.core.handlers.wsgi

    application = django.core.handlers.wsgi.WSGIHandler()

open-data-catalog's People

Contributors

ahinz avatar andrewbt avatar copelco avatar groovecoder avatar jimcraner avatar johndavidback avatar tecknix avatar timwis avatar tomkralidis avatar xtoddx avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

open-data-catalog's Issues

Initial data migration

There is a bit of data required to seed the database in a meaningful way. Now that South is installed, it would be great for a migration to add those bits of data. The things I've come across are:

  • flatpages [/about/, /faq/, ...]
  • update frequencies [daily, weekly, monthly, quarterly, ...]
  • url types [API, application, data, documentation, image]
  • Data types / formats [CSV, CSV with header, JSON, Shapefile, XML]

There may be others.

createlang plpython fails

Can Open-Data-Catalog be run on MacOSX? I'm trying to use it with Postgres.app, and getting this error:

$ createlang plpython
createlang: language installation failed: ERROR:  could not open extension control file "/Applications/Postgres.app/Contents/MacOS/share/extension/plpython.control": No such file or directory

Response from /data.json endpoint is not in standard format.

Base on the code changes made here 1ab5825

I see the endpoint just returns an array of datasets as oppose following the structure specified here https://project-open-data.cio.gov/v1.1/schema/ . That breaks implementation on our side cos we're getting a json array instead of the expected json object. Can we make changes to follow the standard structure :

{
  "@context": "https://project-open-data.cio.gov/v1.1/schema/catalog.jsonld",
  "@id": "https://data.chattlibrary.org/data.json",
  "@type": "dcat:Catalog",
  "conformsTo": "https://project-open-data.cio.gov/v1.1/schema",
  "describedBy": "https://project-open-data.cio.gov/v1.1/schema/catalog.json",
  "dataset": []
}

Or if changing will break impl for existing clients, We can add a new endpoint that follows the above format.

source files for icons?

Hey Azavea -- thanks a ton for this project! It is very nicely designed. I'm wondering if you have the PSDs or equivalent source files for the various nicely designed buttons and icons in the project as they would help when rebranding for other cities.

Cheers,

Max

plpythonu support

With most hosting providers not supporting plpythonu, is there a way to get a different sql function in place?

OpenDataPhilly site name still used on various pages

There are a few places that still use OpenDataPhilly as the site name.

/ # (the Twitter account)
/accounts/register/
/ideas/
/opendata/nominate/
...

Maybe use the Sites framework and pass some site_name variable to the templates to help white-label the platform.

Typo in Philly api-doc

I don't think this typo has made its way to the Open-Data-Catalog codebase,
but I'm posting this here just in case. I've also emailed [email protected].

http://opendataphilly.org/api-doc/ provides conflicting routes
for the API search endpoint; the two endpoints are

http://www.opendataphilly.org/api/resources/search?qs={keyword}
http://www.opendataphilly.org/api/resource/search/?qs={keyword}

The first of these is correct, and the second is not;
"resources" must be plural, and there may be no trailing slash.

PIL not supported anymore

requirements.txt has PIL (unsupported) package.

[localhost] out: Collecting PIL==1.1.7 (from -r requirements.txt (line 2))
[localhost] out: Could not find a version that satisfies the requirement PIL==1.1.7 (from -r requirements.txt (line 2)) (from versions: )
[localhost] out: No matching distribution found for PIL==1.1.7 (from -r requirements.txt (line 2))

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.