tgve / app Goto Github PK

Template Repo

License: MIT License

HTML 18.18% CSS 7.67% JavaScript 41.29% Jupyter Notebook 12.73% Python 6.58% R 8.83% Shell 4.73%

app's Introduction

This is the R package for the TGVE front end npm package tgve. The R package is developed to facilitate interactive geospatial analysis and visualization in R, use R’s echo-system to drive advanced data processing, and facilitate deployment of geospatial web applications in production.

Install

Install tgver from CRAN

install.packages("tgver")

You can use the latest from GitHub using devtools::install_github("tgve/tgver")

Use

Overall, the package takes advantage of how TGVE can be used (see npm package for documentation) and provides options to R users.

For instance, this document is a Markdown (GitHub) document generated using an Rmarkdown (Rmd) document, if the Rmd is rendered to a HTML output, then using knitr::include_url we should see an instance of the TGVE embedded in the document. Please see the live examples in the vignette which is rendered to HTML.

To do this we can go:

# this is the most basic use
# tgver::tgve()
# to embed in a html rendered Rmd
html.file = tgver::tgve(browse = FALSE)
knitr::include_url(html.file)

The first function tgver::tgve(browse=FALSE) prepares an instance of the TGVE but does not run/open it, it returns its path (a tempdir() path). The second line is knitr function to embed the first line’s output.

That was the simplest way of running an instance of TGVE on the local machine. The more advanced but similar function of this package, with the back-end as a plumber API and serving the same instance, would be like:

# start a tgve instance before embedding it
ps = tgver::tgve_server(background = TRUE)
#> Attempting to serve TGVE instance from: /tmp/Rtmpevq9TT/tgve
#> Running plumber at: http://127.0.0.1/8000
knitr::include_url("http://127.0.0.1:8000")
# kill the process returned from underlying `callr`
ps$kill()
# or use the public one
# knitr::include_url("https://tgve.github.io/app/")

Explore `sf` objects

For these purposes, the package relies on the plumber API to serve data. So, again within HTML outputs we can explore sf objects like so:

# using sf
nc = sf::st_read(system.file("shape/nc.shp", package="sf"))
tgver::explore_sf(nc)

For more see the vignette.

Preview

app's People

Contributors

Stargazers

Watchers

app's Issues

nbval test

~~README.md~~
move notebook to tgve/app
nbval test ❌ doesn't really work (see below)

Unfortunately, the IFrame encapsulates the image data, and in particular hides whether the URL actually resolves. Obviously there's more we could do here, but since accessing from Python isn't a priority use case, I'll consider those options out of scope.

Merge `app` and `full-app`

app and full-app are identical, except that full-app has the following additional files:

nodejs/server.js
R/plumber.R
run.R
Dockerfile
tgve.py and tgve-cors.py

Notably, package.json and gh-pages.yml are identical, suggesting that the additional files could be harmlessly included into app. (Someone uninterested in running via Docker can ignore the Dockerfile, someone uninterested in Python can ignore the Python files, etc.)

Including app into full-app as a submodule wouldn't work particularly well because we need the content at the root of app to also be at the root of full-app.

migrate Python/jupyter-tgve.pynb to app
move remaining files
extract relevant parts of full-app/README to app/notes/hosting-data.md
link to hosting-data.md from app/README
move reproducible.Rmd from tgvejs to app
delete full-app

Executable docs for R app hosting

The instructions for hosting in R (via Plumber) should follow the same pattern as the Python instructions, i.e. be presented as reproducible scripts.

Move R instructions from notes/serving-data.md to R/README.md
rename Dockerfile to Dockerfile.R
where appropriate move documentation from R/README.md to plumber.R, run.R or Dockerfile.R
remove CORS documentation (by analogy with Python)?
~~factor Dockerfile.R through run.R?~~ delete run.R -- subsumed by instructions in Docker.R
#44
simplify deploy step of GitHub Pages action
docker.yml workflow to validate Docker scripts

Consolidate Flask documentation

When consolidating the documentation, I found information about how to run things via Flask in at least two different places (Wiki, plus full-app README). These have been merged into the first section of serving-data.md, but it's not clear how they are related. These need to be further consolidated, at least if we want to continue supporting Python (if not we can just delete).

reproducible Dockerfile to validate Flask instructions
#31
move Flask instructions to README in Python folder
merge documentation from README into Dockerfile, serve.sh and tgve.py
extract separate cross-origin.md instructions

Migrate away from geoplumber

Fix e2e testing

Sync `homepage` with `history` API

The GH deployment is stable. The only issue so far seems to be pushing to history does not consider the homepage entry in package.json.

Image snapshot testing with Puppeteer

Snapshot (approval) testing is useful for easy refactoring, and well supported by Jest. It's not quite trivial to set up for us because the map view renders asynchronously, and so waiting on "Nothing to show" doesn't seem to be enough for deterministic behaviour. The following diff illustrates different states that both satisfy expect(..).toContain("Nothing to show"):

Maybe we can find a UI element to wait on that only appears when the map is fully loaded.

install jest-image-snapshot
import toMatchImageSnapshot from jest-image-snapshot and extend Jest’s expect API
~~condition that over-approximates map being fully loaded~~
- ~~waitUntil: "networkidle0"~~ -- probably not very reliable
~~revisit snapshot tests in tgvejs (maybe no longer needed)~~
set visibility of map and loader component to hidden (to avoid problem with async map content)
change Puppeteer viewport size to include more of the side desk
export __snapshort folder as build artifact
set error tolerance in screenshot diffs to 0.5%

See:

Snapshot testing with Puppeteer
Puppeteer options for detecting navigation complete
Puppeteer wait until page is completely loaded
Puppeteer to listen for map.on('load') from within Node

Add data source to last build in actions

Seems we have removed the env vars from the last build. App is deployed with no data source

Webserver produced by Flask script fails with syntax error in `[...].chunk.js` file

Running the Flask instructions starts a webserver at http://127.0.0.1:5000 but going to that page fails:

See the Dockerfile for a reproducible example.

The obvious choices are to either:

support Python properly and fix the instructions/Dockerfile
drop Python support and drop the instructions/Dockerfile

fix problem in current Flask configuration (as captured in Dockerfile.Python)
~~validation script for cross-origin.md instructions~~

Move to `full-app`

Delete this repo, moving the notebook example to full-app along with the other Python examples.

move Python (Flask) examples in full-app to new python folder
move jupyter-tgve.ipynb to same folder
move run.R to R folder
migrate issues to full-app
delete jupyter repo

Delete or verify `reproducible.Rmd`

An RMarkdown file called reproducible.Rmd should do what it says on the tin. Set up some CI to verify it, or delete it if no longer relevant.

rmarkdown::render("notes/reproducible.Rmd")
- there is no package called RCurl
  - install.packages("RCurl")
~~remove geoplumber dependency~~
give up and delete

Fix repo rename issues & update docs

Hosted instance workflow

The “dev” instructions in the README are incomplete and inconsistent with the “dev” instructions in eAtlas, which also explains how to use eatlas-template. We should consolidate, and ideally provide an executable example. We should also determine whether there need to be instructions for setting up just an application instance, with no dev capability.

move “dev” documentation to tgvejs, link to from app
eAtlas README says file: instead of link: in package.json – which is correct?
~~build automation to verify/replace instructions (see checkout repos side-by-side instructions)~~ – overkill
make a pass over Hosting your own instance instructions in README.md
manually verify instructions after instantiating template
- sed -i "s/tgve/${a[0]}/" step fails (shouldn't match "@tgve/tgvejs or tgver script name)
  - use json tool instead of sed to update homepage only

UI testing strategy

Test that app loads with a data URL. Arose from tgve/tgvejs#133.

Pushing to branch doesn't trigger build

GitHub workflow is configured to only run on release, which makes it hard to validate e.g. changes to package.json.

have build sep run on all branches
have deploy step run only on release
build on pull_request too
same changes for full-app