Giter Club home page Giter Club logo

codeforafrica / hurumap Goto Github PK

View Code? Open in Web Editor NEW
8.0 15.0 7.0 13.99 MB

HURUmap is an interactive web platform that allows citizens and journalists to explore, visualise, and download census data. This gives them the power to give context to stories that was otherwise hard to spot. Accessible at https://hurumap.org

Home Page: https://hurumap.org

License: GNU General Public License v3.0

Python 26.20% HTML 34.80% CSS 3.25% Makefile 0.27% Dockerfile 0.76% JavaScript 34.06% Shell 0.66%
hurumap africa geo-datasets geographies kenya nigeria south-africa tanzania zambia uganda

hurumap's Introduction

HURUmap

HURUmap is an interactive web platform that allows citizens and journalists to explore, visualise, and download census data. This gives them the power to give context to stories that was otherwise hard to spot. Accessible at https://hurumap.org

Goals

Instead of re-inventing the wheel, HURUmap relies on the excellent Wazimap and Census Reporter projects and aims to do the following:

  1. Extend the features available e.g download SVG / PNG
  2. Provide a platform of shared data - tables and geographies (using mapit) that can be re-used with the different apps
  3. Provide shared styling, design, attribution, and other engagement content preferences

Usage

TODO: How to build HURUmap-powered apps using opinionated Docker.

Install HURUmap Django App / package

  1. Install the package

    pip install -e git+https://github.com/CodeForAfrica/HURUmap.git@master#egg=hurumap
    
  2. Add the app to installed apps

        INSTALLED_APPS = [
            ...,
            'hurumap'
        ]
  3. Add these to your settings.py

  4. Run migrations

        python manage.py migrate
  5. Create an admin user to access the CMS

       python manage.py createsuperuser
  6. Read the full Wazimap documentation to get started.


Contributing

To ease contribution, our key principles are borrowed from contrib where multiple programming languages and approaches can live together in harmony.

To this end, we've created a contrib folder that contains the following:

  1. Docker scripts for the Django apps powering majority of the backend
  2. Yarn implementation for the frontend pieces (coming soon?)
  3. Makefile in the root directory to tie it all together with easy tasks

For more details on contributing, please check out CONTRIBUTING.md.

Development

Django Apps

Quickly get started with HURUmap by running the following:

# 1. Build services:
make build

# 2. Start web server:
make web

# 3. In a new terminal window, load data:
make loaddata

Visit http://localhost:8000 to view website.

HURUmap Dashboard

HURUmap comes packaged with built-in CMS using Wagtail. To work with the dashboard:

# 1. Target and build docker image
export DOCKERFILE_TARGET="hurumap-dashboard"
make build

# 2. Start web server:
make web

# 3. In a new terminal window, create your first user:
make createsuperuser

Visit http://localhost:8000/dashboard to log into the dashboard.

To remove dashboard workings, stop all services and run;

unset DOCKERFILE_TARGET
make build
make web

NOTE: Unless you remove the database container, all the database tables and their data will remain intact.

Frontend UI

Hide Empty sections on the profile page

Use LOCATION_NOT_FOUND_DIST in HURUmap profile templates to either hide or call to action whenever there's a missing data.

If our data distribution is called budget, then our code that hides the viz whenever data is missing:

  {% if not budget.is_missing %}
    <article class="clearfix">
      <header class="section-contents">
        <h1>Budget</h1>
      </header>
      <div class="section-container">
        {% if not budget.government_expenditure_dist.is_missing %}
          <section class="clearfix stat-row">
            <h2 class="header-for-columns">Government Spending</h2>
            <div class="column-full"
                 id="chart-grouped_column-budget-government_expenditure_dist"
                 data-stat-type="scaled-percentage">
            </div>
          </section>
        {% endif %}
      </div>
    </article>
  {% endif %}

Multiple Chart Qualifiers Support

This version adds the ability to have more than one qualifier. Due to the HTML limitation of single data-qualifier attribute, the value of this attribute would need to be delimited to support multiple values. \n is used as delimiter.

... data-qualifier="GF: Global Fund\nADF: African Development Fund" data-...

By adding multiple qualifiers support, one can use qualifiers to add a legend to those charts, like histogram, that do not have a legend by default. This means shorter names can be used on the charts themselves leading to clean and easy to read charts.

Adding Data

Datasets and Releases

This HURUmap release takes advantage of datasets and releases for versioning data. A dataset is a collection of related data tables, such as a national census. A dataset can be updated with new releases every few years. Not all data data tables will always be updated in every release, so HURUmap lets you link data tables to releases individually.

Sometimes a release has a different name to the original dataset. For example, South Africa conducts a full census every decade, but releases a community survey in between each full census. A community survey is a statistical sampling and is not a full census, so it would be incorrect to call them both "census". The results of the community survey are very similar to the census and are directly comparable. We consider census and community surveys to be different releases of the same dataset.

You must add at least one dataset and one release before you can add any data tables. See below for details on how to do this.

Create a Dataset and Release
  1. Go to the Django admin section at http://localhost:8000/admin and log in.
  2. Under Wazimap, click the Add button alongside Datasets.
  3. Give your dataset a name.
  4. Under Releases, fill in the name and the year of your first release. For example, you could use Census and 2019.
  5. Click Save.
Configuring Tables

Datasets, releases and data tables are configured through the Django admin interface, at http://localhost:8000/admin.

Once you have told HURUmap about your tables, it'll ensure that they exist in the database. You can then import the raw data from CSV.

Adding custom styles to your embeds

You can customize the styles of your iframe embed by creating a .css file in your static directory and you append it to the iframe link like this <iframe ... src="... &stylesheet=/static/css/embed_test.css" ... ></iframe>

Deployment

HURUmap Basic

TODO

HURUmap Dashboard

TODO

Static and Media Files deployment

Set these environment variables:

USE_S3 = True
AWS_ACCESS_KEY_ID = ...
AWS_SECRET_ACCESS_KEY = ...
AWS_STORAGE_BUCKET_NAME = ...
AWS_S3_REGION_NAME = ...

License

GNU GPLv3

Copyright (C) 2018 Code for Africa

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

hurumap's People

Contributors

davidlemayian avatar khadijamahanga avatar kilemensi avatar phillipahereza avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

hurumap's Issues

[FEATURE] Hide / Give CTA in empty sections

Is your feature request related to a problem? Please describe.
When data doesn't exist, the section looks empty.

Describe the solution you'd like
We should either hide or give a clear call to action (CTA) to the user.

Describe alternatives you've considered
<TODO: A clear and concise description of any alternative solutions or features you've considered.>

Additional context
40227649-c42cd904-5a97-11e8-9961-1db0840830f7

Import Data Management Command

Write a management command to replace cat takwimu/sql/*.sql | psql -U takwimu -W takwimu to simplify running this command in docker deployments

Add a blog template

Description

With a CMS, we'll be able to publish blogs. We should therefore create a template for the articles being published through there.

Tasks

  • #2
  • Create a blog template to align with HURUmap style

[FEATURE] Add filter functionality to profile page

Is your feature request related to a problem? Please describe.
When looking at a country profile, there are a lot of data indicators / topics / fieldtables making for a long page.

Describe the solution you'd like
There should be an easy way to filter the view of the profile page with data indicators so that I'm looking at the ones most relevant.

Perhaps should also consider a way to "suggest" filters according to choices made.

Describe alternatives you've considered
<A clear and concise description of any alternative solutions or features you've considered.>

Additional context

Misleading License Information in setup.py

Describe the bug
setup.pylists HURUmap license as MIT but GPLv3 is the one stated in the LICENCE as well as README files

To Reproduce
Steps to reproduce the behavior:

  1. Open setup.py, and
  2. Scroll down to license section.

Expected behavior
Should be GPL

Add Elasticsearch

Is your feature request related to a problem? Please describe.
I'm always frustrated when I can't easily fuzzy match through the site's content.

Describe the solution you'd like
A way to fuzzy search content on the site.

Describe alternatives you've considered
Elasticsearch

Additional context
This might be most powerful for "related" content tab on HURUmap.

Write Tests + Implement TDD

Is your feature request related to a problem? Please describe.
Currently we only do basic tests - security, deployment checks, and some unit tests.

Describe the solution you'd like
We should write more comprehensive unit tests and implement test driven development - where we write the tests first and then write the code.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
TO DO

Additional context

Light reading on the same:

We'll also need the technologists to go through quick courses on how to do this well in Django.

Initial Update

The bot created this issue to inform you that pyup.io has been set up on this repo.
Once you have closed it, the bot will open pull requests for updates as soon as they are available.

Setup Static Page Template

Description

Create a base page template to ease the creation of static pages on HURUmap

Tasks

  • Create the template
  • Create About page using this template

[Feature] Download Geography Level Data

Is your feature request related to a problem? Please describe.
It's sometimes desirable or even necessary to get a copy of all data within a given geography level i.e. country, county or province, region, district, or ward.

Describe the solution you'd like
I would like to have an option to download all the data with a given geography level that I'm currently browsing in an offline, printable format like PDF.

[BUG] Missing javascript files error with WhiteNoise

Describe the bug

While using WhiteNoise in a django app, I get a value error on profile page that breaks my site.

ValueError: The file 'js/affix.js' could not be found with <whitenoise.storage.CompressedManifestStaticFilesStorage object at 0x7f37a7c4e1d0>.

profile_detail.html has scripts that source from missing java-script files

 {% block profile_javascript_libs %}
    {{ block.super }}
    <script src="{% static 'js/affix.js' %}"></script>
    <script src="{% static 'js/profiles.js' %}"></script>
  {% endblock %}

Add Google Oauth

Description

To make it easier for authentication, we should allow for Google sign in.

Tasks

  • Create Oauth application on Google Console
  • Add Django All Auth to Django framework
  • Test

Add Content Management System

Description

After initially considering Django CMS, we will instead integrate Wagtail into HURUmap to offer a way to manage the content published on this platform including country profiles, blogs and site pages.

Some of the reasons choosing Wagtail over Django CMS include:

  1. StreamField offers interesting functionality similar to proto's card + stream approach to content.
  2. Being more recent, could offer newer techniques of problem solving.

Read more on choice here: http://www.djangopaths.com/why-did-i-choose-wagtail/#Integrating%20custom%20models%20and%20templates

Links

Tasks

  • Add wagtail to requirements
  • Write tests
  • Write documentation
  • User test
  • Delete ft_wagtail_cms branch on TakwimuAfrica/TAKWIMU

Write Documentation + Examples

Is your feature request related to a problem? Please describe.
HURUmap builds on @OpenUpSA/Wazimap, by adding features and extensions that are neither in Wazimap nor CensusReporter.

Describe the solution you'd like
Documentation of all features and settings specific to HURUmap only.

[IDEA] Use a Graph DB and add GraphQL API

Is your feature request related to a problem? Please describe.
Currently the API revisions and changes (of the structure and the data) don't maintain well such as not to be disrupted with major feature change.

Describe the solution you'd like
To provide a more robust API language and data structure for later processing and presentation (visualisation or otherwise), we should provide the data in a node/graph network structure.

Also our FieldTables may offer themselves better to a Graph Database.

Describe alternatives you've considered
A good start would be https://graphql.org/ before moving to a graph DB for even better query abilities on large survey data.

Additional context
https://www.youtube.com/watch?v=GekQqFZm7mA

Review HURUmap as a re-usable Django App

Description

Review HURUmap as a re-usable Django app and create relevant issues / tasks on this issue to fix this.

Tasks

  • Review according to https://docs.djangoproject.com/en/1.11/intro/reusable-apps/
    • Project folder structure
    • Documentation (#29)
    • Tests (#20)
    • Community
      • CONTRIBUTORS/AUTHORS/CONTRUBUTING file
    • Uses the Broadest Requirements Specifiers Possible (since this is a library)
    • README
    • LICENSE
      • Mentioned in the README
      • Separate LICENSE file for easier discover-ability by tools
    • CHANGELOG
    • setuptools
      • setup.py
      • Application information in hurumap/__init__.py imported into setup.py
      • MANIFEST.in
        • include README
        • include LICENSE
        • recursive include
          • hurumap/static *
          • hurumap/template *
          • docs *
  • v0.1.0 - Package(?) and create release
    • Published on PyPI

Switch to master branch

Is your feature request related to a problem? Please describe.
Switch to master branch as the main/release branch of the repository

Describe the solution you'd like
master as the default branch and all open.

Describe alternatives you've considered
Github Flow.

Validate Geography files (geojson, topojson)

Is your feature request related to a problem? Please describe.
Creating geojson and topojson files is prone to errors

Describe the solution you'd like
Create a management command that validates that the geography files match the format required by HURUmap

Describe alternatives you've considered
GeoJsonLint - Works well for small files but fails for significantly sized files

Additional context
None

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.