We need to provide a data dictionary that describes the indicators.

• Physical Exposure
• Social Vulnerability
• Multi-hazard Risk
• PSRA
• DSRA

Add Provincial/Territorial PSRA datasets to site. Note, metadata is in development so we should put some placeholders in for now.

We need to have metadata for the PSRA datasets (Sauid, and Building aggregations) prior to sending them to FGP.

Template here: https://github.com/OpenDRR/data/blob/main/metadata/Metadata_Minimum_Partner_Information_Form_v2.xlsx

@jvanulde to handle the translation.

{{site.github.releases[0].tag_name}} returns an empty string on GitHub Pages of forks of OpenDRR/data

As first reported by @jvanulde in OpenDRR/data#54, the release tag (e.g.v1.2) is missing in the release assets download URLs on GitHub Pages of forks of OpenDRR/data, as can be seen on, for example:

• https://jvanulde.github.io/data/en/nhsl.html
• https://anthonyfok.github.io/data/en/nhsl.html

The release tags do exist in the forks (see https://github.com/jvanulde/data/releases and https://github.com/anthonyfok/data/releases), but somehow {{site.github.releases[0].tag_name}} just isn't picking that up... Don't know why yet.

Fortunately, the assets download URLs are OK on https://opendrr.github.io/data/en/nhsl.html with the release tag intact.

Interestingly, the release tag is also intact on assets download URLs on e.g. http://127.0.0.1:4000/data/ served by bundle exec jekyll serve running locally on my computer.

See also PR OpenDRR/data#49 and commit 7bbfdf2ca7d465703912a89158840456d050bc96 where the Jekyll plugin jekyll-github-metadata which might offer some clue. Perhaps try other site.github.* variables(?) and see if they are picked up in GitHub Pages of forks?

In PR OpenDRR/data#49, I wrote:

I ended up using {{ site.github.releases[0].tag_name }} instead of {{ site.github.latest_release.tag_name }} because the latter does not recognize pre-releases.

That said, in jekyll/github-metadata#80, which incidentally came up as the second Googld search result of "jekyll get latest github release tag"), the original poster wrote:

Sometimes a release is tagged for in the past for historical purposes. Grabbing all of the releases and hoping that the first one is the latest doesn't work. Instead, it could query /repos/:owner/:repo/releases/latest which will give the release marked latest.

So, perhaps it would indeed be good to switch to {{ site.github.latest_release.tag_name }} in pages/{en,fr}/nhsl.md when we start having official releases (i.e. not pre-releases) probably in Q4 2021 or Q1 2022.

Turn the following:

curl -s https://api.github.com/repos/OpenDRR/data/releases \
    | egrep '"name"|"download_count"'

into a download statistics page somewhere in our GitHub Pages site. With tables? With barcharts? Try to collect data hourly/daily/weekly/monthly etc. to see trend? (but how to store the statistics? Using artifacts perhaps?

Could start with Hugo's download statistics:

curl -s https://api.github.com/repos/gohugoio/hugo/releases \
    | egrep '"name"|"download_count"'

for interesting results over a longer period of time.

Find out how to implement this with Jekyll, similar to how Hugo can get remote data with getJSON. Would the following Jekyll plugin work?

• brockfanning/jekyll-get-json: Import remote JSON data into the data for a Jekyll site

See also, probably for interests only:

• GitHub: Getting the download count for a release · Mark Needham
• Add download_count next to each release artifact · Issue #670 · sindresorhus/refined-github (Chrome extension)

As we tried out PR OpenDRR/data#49 after merging, we saw the release assets for v1.2 available, but the download links from e.g. https://opendrr.github.io/data/en/nhsl.html#physical_exposure still pointed to v1.1, i.e., the previous (pre-)release:

• https://github.com/OpenDRR/data/releases/download/v1.1/nhsl_social_fabric_all_indicators_s.zip (Not Found)

instead of the real:

• https://github.com/OpenDRR/data/releases/download/v1.2/nhsl_social_fabric_all_indicators_s.zip

We think it is due to GitHub Pages build does not get triggered by the mere creation of a release tag, i.e. without new commit push, it may be a good idea to follow https://stackoverflow.com/questions/24098792/how-to-force-github-pages-build

Notes:

• https://docs.github.com/en/rest/reference/repos#request-a-github-pages-build (POST request)
• https://docs.github.com/en/rest/reference/repos#get-latest-pages-build (GET request)

Existing GitHub action to trigger rebuild for ALL GitHub Pages sites of a user (or organization too?):

• https://github.com/marketplace/actions/jekyll-update-github-pages-without-new-commit

Define thresholds and color guides for symbolizing risk metrics that require a standard symbology. Layer definitions to be documented using QGIS MXD layer file format.

• Physical Exposure
• Social Vulnerability
• [ ] Multi-hazard Risk
• PSRA
• DSRA

We need to have metadata for the DSRA datasets prior to sending them to FGP.

Template here: https://github.com/OpenDRR/data/blob/main/metadata/Metadata_Minimum_Partner_Information_Form_v2.xlsx

@jvanulde to handle the translation.

As per discussions with Joost on Mon 2021-05-10 and Tue 2021-05-11, we would like to recompress the current ZIP archives which store the GeoPackage (*.gpkg) files with another compression tool (e.g. 7-Zip or xz) that offers higher compression rate.

Joost also suggested the use of GitHub Actions for automatic recompression

Notes:

There are only 9 ZIP files (out of 734) that are over 100MB, and fewer yet after recompression with 7z or xz (or even zpaq):

foka@debian-e431:OpenDRR/data$ find assets -size +100M -exec ls -lh '{}' \; | sed -e 's/.*users/*/'
* 174M May 11 11:17 assets/data/national_human_settlement_layers/nhsl_social_fabric_all_indicators_s.zip
* 110M May 11 11:17 assets/data/national_human_settlement_layers/nhsl_physical_exposure_all_indicators_b.zip
* 125M May 11 11:16 assets/data/national_human_settlement_layers/nhsl_hazard_threat_all_indicators_b.zip
* 191M May 11 11:16 assets/data/national_human_settlement_layers/nhsl_hazard_threat_all_indicators_s.zip
* 187M May 11 11:17 assets/data/national_human_settlement_layers/nhsl_physical_exposure_all_indicators_s.zip
* 234M May 11 11:17 assets/data/seismic_risk/on/psra_on_all_indicators_b_.zip
* 175M May 11 11:18 assets/data/seismic_risk/qc/psra_qc_all_indicators_b_.zip
* 244M May 11 11:18 assets/data/seismic_risk/psra_all_indicators_s.zip
* 404M May 11 11:18 assets/data/seismic_risk/psra_all_indicators_b.zip

So perhaps we will modify .gitattributes to match only these handful of files for Git LFS, and keep the other 700+ files inside the git repository.

Reference:

• Git LFS: support tracking by file size? git-lfs/git-lfs#282 (some users come up with creative solutions)

This is a follow-up to Issue OpenDRR/opendrr-data-store#42 and PR OpenDRR/opendrr-data-store#43 (review), especially Joost's (@jvanulde) recommendation:

to see a new issue created to track the requirement of S3 and/or a PR against the data repo with the new zip files.

This will need to be done for both:

1. the existing Jekyll site at https://opendrr.github.io/data/
2. the future Gatsby site at the same URL