Giter Club home page Giter Club logo

opendataday's Introduction

opendataday

Build & Deploy to GH Pages

Open Data Day is a gathering of citizens in cities around the world to write applications, liberate data, create visualizations and publish analyses using open public data to show support for and encourage the adoption of open data policies by the world's local, regional and national governments.

You're invited. Again. The next Open Data Day will take place on Saturday 5 March 2022.

Check http://opendataday.org for public website.

Editing / contributing

  • This site is built with Lektor.
  • Install dependencies: make install
  • Start dev server: make serve
  • grunt will watch for changes to your SCSS files, and also icons (see svgstore).
  • lektor deploy will deploy to the gh-pages branch, putting changes live, if you have deploy permissions.

Translations

To translate the site we need to duplicate the contents.lr file, and save with the language code in the file name, like contents+de.lr.

We also need to add translations to:

If we are adding new language to the site we also need to:

Running the site locally

Requirements:

  • Python >= 3.6
  • Node JS >= 10
  • NPM >= 7

Setup:

  • Create a python >=3.6 virtual environment
  • Base requirements are defined in the requirements.in file (to ejecute pip-compile it's required to update requirements.txt)
  • pip install -r requirements.txt
  • npm ci
  • lektor server
  • http://127.0.0.1:5000/

Updating the Site

Events are populated using a very lightweight import script which:

  • Imports data from a google sheet
  • Does some cleaning and checks on it
  • Writes it out to a JSON file
  • Generates .lr files for any event report
  • Harvests any images we want to use on the site
  • Commits the JSON file, event reports and images back to the repo (remember to git pull)
  • Builds and deploys the site

This is run in a github action with a workflow_dispatch trigger, which allows our content manager to maintain the events in a spreadsheet update the site via a button in github.

screenshot

In general, the import process is designed to be lax about missing fields and unexpected values. Some minimal validation is performed on the fields we require for the site to build correctly. Any errors will fail the action and be displayed in the GitHub actions log. e.g:

marshmallow.exceptions.ValidationError: {'Latitude': ['Not a valid number.'], 'Longitude': ['Not a valid number.']}

These will need to be resolved by editing the input spreadsheet.

There are three bits of setup we need to do each year:

Before we start, we will need a Google Form and a Google Sheet to collect the event data.
This is the 2023 form: https://docs.google.com/forms/d/e/1FAIpQLSc5wiXeUur8GHMKmi1TQIMxwk107eMJVN6UVoibIfUwGZfc7A/viewform
The Google Sheet identificators must be added to the /etl/202x.py file.
This script will attempt to read the data from the Google Sheet so you need to ensure this sheet is public.

  • 1. Create a new import job:
    • Create ./etl/202x.py based on a previous year's file
    • Fill in SPREADSHEET_ID, SHEET_ID and THIS_YEAR
    • Update schema as applicable if the spreadsheet has changed, but we want to keep the JSON output the same if possible as every year's events page uses the same template to render the JSON to HTML
    • Replace the GitHub workflow update-data.yml with the new job tu run python ./etl/202X.py. We don't need to keep running last year's script.
    • Once we've run the job for the first time it will create a /databags/202x.json file. This is consumed by the events list page and the events map
    • We can run python ./etl/202x.py locally as well as triggering it with GitHub actions
  • 2. Update the map:
  • 3. Create a new events page:
    • Create ./content/events/202x/contents.lr based on a previous year's file
    • Fill in the fields
    • Create a stub ./content/events/202x/contents.<lang>.lr for each supported alternate language
    • It is not necessary to manually create the ./content/events/202x/reports dir or edit any content under it. This will be automatically generated by the import job.
    • Add the new 202x events page to the menu:
      • Add /events/202x to the list of events pages (main-nav.json).
      • Update the 'Search events' link to the current year (main-nav.json).
      • Repeat for each supported alternate language

For reference, you can check the PR for the 2022 update

Previous version

The previous version of the site has been moved to the v1 branch.

opendataday's People

Contributors

aivuk avatar amercader avatar augusto-herrmann avatar avdata99 avatar bact avatar chris48s avatar clkao avatar danfowler avatar dependabot[bot] avatar givemetarte avatar hatochan avatar hcorinna avatar jjalcolea avatar kant avatar loleg avatar lucpretti avatar lwinfree avatar morchickit avatar nikeshbalami avatar pdelboca avatar pwalsh avatar roll avatar sadless74 avatar shubhamghimire avatar smth avatar supaplextw avatar tlacoyodefrijol avatar trevineju avatar tuukka avatar zaymes avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

opendataday's Issues

map marker popups. linkify the urls

This should be an easy one.
The map display on the homepage spits out fields into the marker popups. This includes 'url' ... which is displayed without being linkified!

map is broken

probably gdoc api changed and recline is getting recordCount = 0

Link URLs in the map

In the map, make the URL for each event clickable, so I can access more directly to each city's event.

I think the fix should be in the file
js/map.js

Adding in line 23

var linkedUrl = "<a href=\""+record.get("url")+"\"">"+record.get("url")+"</a>";
record.set("url",linkedUrl);

I can't test it so I don't know if it works. The risk is that it displays the html in text instead of showing the link, that depends on how does the leaflet plugin handle each record, so I hope someone more familiar to the code could continue this.

Thanks.

Moscow OpenDataDay

Hi! We will organize open data day in Moscow. Is it possible to add place on map?

Display translations of homepage prominently

This issue is connected to issue #85 . Currently, the sidebar does link to different translations but it is not presented well on the site (user have to scroll down the sidebar to get to the appropriate language).

Map of Events

Had one prepped on my hard disk and needed to push (done now).

Add events to this google doc: https://docs.google.com/a/okfn.org/spreadsheet/ccc?key=0AqR8dXc6Ji4JdFNQRExkVjI3Q3RtRVZuU1hXRmhlTVE#gid=0

I have locked first row to avoid b0rking the map (it needs certain column names ...)

Note that Lon / Lat are not essential - if not provided we try to guess but better if they are there. Easiest way to look up is: http://open.mapquestapi.com/search?q=berlin and there is also a google app script plugin we could use

Allow to add individual event pages

This is for user story 11 - (issue #95 ) : Some Open Data Day orginisers will need their own event name (maybe something like opendataday.org/year/location). I assume that since this is a github pages site, this should be easy.
We need to -

  1. Allow users to add their event page to the site in an easy manner.
  2. Document how to add the event page.
  3. Have a page name "events" that is a table of contents to all the events that were added.

Can we add Extension:HTML tags? (Also need access to LocalSettings.php)

On the main page of the wiki http://wiki.opendataday.org/Main_Page there's this request:

Help wanted: Add the map to the wiki Can you install (http://www.mediawiki.org/wiki/Extension:HTML_Tags) to add the link to the map. <iframe style="width: 100%; border: none; height: 400px;" src="http://opendataday.org/map/" height="240" width="320" scrolling="no"></iframe>. Add your fork to [open data day github https://github.com/okfn/opendataday.org] to update this.

Unfortunately, just forking and adding the module is not enough - a setting has to be changed on LocalSettings.php: http://www.mediawiki.org/wiki/Extension:HTML_Tags

I'm not sure who has access to the Local Settings file!

Can't access all items in sidebar

The sidebar position is fixed so when I scroll it follows me around. I can see by looking at the source code of the site that I'm missing out on three items in the list which I can't access because of this.

get rid of the wiki. Merge content onto the main wiki.okfn.org

I think it could be good to think about getting rid of the opendataday wiki. Merge content onto the the main wiki.okfn.org (where necessary. There's actually not much very worthwhile content on the open data wiki anyway. Lots of pages documenting past events in an untidy confusing way. The only really useful thing is a single page list of events happening in 2015, which would very easily moved onto a wiki.okfn.org page)

Why do that? because a wiki community is a critical mass game. When it comes to things like getting people tidying things up and despamming and carefully restructuring things... you need a few active users forming a community, and watching the recent changes page, and feeding off eachothers activity. It's like trying to start a fire. Often activity fizzles out quickly, but if you can get it to grow...

So trying to kindle two different fires (or more. Are there other mediawiki's still being used by OKFN?) makes it less likely to succeed.

Fix map

This is connected to user story 3 (issue #87 ). The map component helps to users to identify events in their area. Currently, the map is broken, and I am gathering all map related issues under this issue.
The dataset for the map is located in this spreadsheet- https://docs.google.com/spreadsheets/d/1cV43fuzwy2q2ZKDWrHVS6XR4O8B01eLevh4PD6nCENE/edit#gid=1172400195

So far, it has been easy to add events with the spreadsheet, so we should keep it.

  • make the URL for each event clickable see #53.

  • The tiles in the map do not display anymore and point instead to this blog post. Make map to show events. See #84

Map is mostly 2014 events?

The map featured very prominently on the open data day homepage is mostly listing 2014 events I think. Can't always tell, so it's rather a confusing thing to show!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.