tgve / tgvejs Goto Github PK

• write actions yml
• sync with docker.yml

The .env and then exposing it as process.env as per React docs may not be suitable for the production.
The deployment pipelines means we cannot add the MAPBOX access_token to the prebuilt image on DockerHUb.

Seems passing both REACT_APP_MAPBOX_ACCESS_TOKEN and MAPBOX_ACCESS_TOKEN is not picked up by the JS bundle served.

Looking for a solution.

@griff-rees

Currently semi-hardcoded. Cautiously increasing the API values.

Current release of the npm package 0.0.211 functions without any issues/warnings but current dev work hit warnings which are related to this.

mapbox-gl-js is no longer under the 3-Clause BSD license. By upgrading to this release, you are agreeing to Mapbox terms of service. Refer to LICENSE.txt for the new licensing terms and details. For questions, contact our team at https://support.mapbox.com.

Options are limited, clearly for those who love Mapbox map tiles, they can continue as usual. For the rest, which includes the way eAtlas is used in geoplumber R package, must consider mapblibre-gl-js.

- [ ] Arc
- [ ] Line
- [ ] path

The approach is as follows:

1. Define the type of settings required.
2. Define the visualization based on (1)
3. Define passing the values from (2) in generating the layers.

That does not mean much because it is a functional approach the example is this for a heatmap layer:

1. Define a variable called getWeight as a column based settings
2. Generate a dropdown to select a column to be assigned to (1)
3. Use the value obtained in (2) to generate the layer and also use it in the triggers defined in DeckGL.

Some docs is required see the Wiki pages.

e.g.

REACT_APP_GEOGRAPHY_COLUMN_NAME: msoa11cd

means that both the data from REACT_APP_DEFAULT_URL and REACT_APP_GEOGRAPHY_URL must share a common column name, which might require manual editing of the data.

Would it be possible to specify this as a dict, e.g.

REACT_APP_GEOGRAPHY_COLUMN_NAME: msoa11cd: MSOA

(apologies if my yaml syntax is a bit off)

then I could keep the original column name ("MSOA") in the data?

Thanks!

Make sure add data takes priority

The idea comes from SPENSER and now PIGSUSTAIN. Provide a CSV with geography column and eAtlas detects geographies.

• eAtlas -> tgvejs
• eatlas-template -> app
• full-template -> app-full
• use notation tgve/app, tgve/full-app in docs
• revisit tgve- prefix for app, app-full
• search and replace all occurrences of eAtlas
• rename main/master branches in app and spenser to release
• references to eatlas-data
• wip.R refers to ~/Desktop/recent/eAtlas (2+ years old, delete?)
• package.json in full-app identifies package as eatlas-template
• package.json in spenser identifies package as eatlas
• rename eatlas-data
• tgve.R refers to eatlas-template
• rename npm component
• gh-pages.yml in app and full-app refers to layik
• https://badge.fury.io/js/eatlas.svg
• get_quant.R, get_spenser.R, plumber.R and Header.jsx in spenser refer to layik/eAtlas
• rollup.config.js in tgvejs refers to (generates?) dist/eatlas.js
• update links to npm package

For clarification, the branch was the main dev repo for eAtlas/TGVE and has now grown out of its purpose.

• master branch is now way behind branch npm
• branch npm is the most important part, npm-dev is where I do work and create PRs on npm.
• Wiki is the main docs for "the project"
• Dockerfile is based on "old" approach of building the front-end using JS tools (npm)

Must be done:

• README to serve as pointer to Wiki/docs and other repo docs etc
• master branch => npm branch?
• etc

Consider doing:

• ??

A development workflow for working on tgvejs and running it via app.

• testing instructions
  • yarn test in tgvejs
    • Cannot find module 'react' from 'ReactSixteenAdapter.js' -- add to devDependencies
• experiments on master branch
  • build Docker image
  • yarn install/yarn start
• experiments on npm branch
  • build Docker image ❌ Error: Cannot find module 'webpack/lib/node/NodeTemplatePlugin'
  • yarn install/yarn start ❌ Could not find a required file. Name: index.html
  • npm install/npm start ❌ Webpack 4.42.0/4.46.0 dependency issue
• resolve #92
• tgve/app#4 (?)
• identify key users:
  • TGVE contributor
  • Node.js/React developer
  • R developer
  • Python developer (building Flask app)
• native Mac development
  • get to the point where we can run geoplumber::gp_plumb() in the R console
• Docker-based development
  • move Dockerfile from eAtlas/npm to app
  • locally modify package.json to point to file:../eAtlas (not link:!)
• identify key use cases
  • contribute to tgvejs
  • build custom app based on TGVE?
  • deploy my own copy of TGVE (e.g. locally to work with sensitive data)?
  • use the publicly available deployment of TGVE
• move change log to separate .md file
• #100
• commit to yarn (vs. npm) to avoid warnings about package-lock.json
• locally modified package.json is a bit of a pain
  • npm link ../tgvejs seems to achieve same effect but without changing package.json
• dev workflow instructions in README.md for tgvejs contributors
  • clone tgvejs and app repos to local machine
  • locally modify @tgve/tgvejs dependency in app/package.json to point to link:../tgvejs
  • yarn watch to incrementally update dist dir of tgvejs?
    • app doesn't pick this up -- need to delete node_modules/@tgve/** and then yarn again

Understand how to install and run the app on localhost.

• remove references to “the main repo” in eAtlas
• check in with @layik re. unpublished commits on master
• rename master -> old
• delete old

Done/dropped:

• make npm the default branch
• delete build and start scripts from package.json
• delete wip branch
• #107
• delete letsbeatcovid and covid19
• rename npm to (e.g.) release and npm-dev to develop

This is mainly to keep using GH functionality such as automatic issue closing etc.

The npm development is the way forward. However, the original fork is useful for people who might want to use it the way the original work was done. Now is the time for a new master/main branch.

Imported from #96:

For clarification, the branch was the main dev repo for eAtlas/TGVE and has now grown out of its purpose.

• master branch is now way behind branch npm
• branch npm is the most important part, npm-dev is where I do work and create PRs on npm.
• Wiki is the main docs for "the project"
• Dockerfile is based on "old" approach of building the front-end using JS tools (npm)

Must be done:

• README to serve as pointer to Wiki/docs and other repo docs etc
• master branch => npm branch?
• etc

Consider doing:

• ??

1.1.0-beta.2 feedback

Currently, the Welcome component does quite a lot for a good reason. It is now time to do less. There is a branch to deal with this wip with main problem being a standard React issue of "mirroring state and props" for viewport.

The new component should only rerender if "data" or "layers" has changed, but it also needs to change "viewport" if user interacts or location changes. The latter would be managed by the parent component. But when providing that viewport there is kind of an attempt of mirroring props and state which didn't work on a full day attempt.

As well as #35, application should now sync functionality from #30: allow separate files (like API/settings).

@layik Looks like this differs from npm by a single commit (5b3535d) which removes the plotly dependency:

Need to figure out if this still needed and if so, whether there is a better solution to having another active branch.

React-vis does not have one, I am not sure if I can contribute upstream, but for now this can get us a nice D3 boxplot for continuous numerical columns.

Or using the React-vis candlestick example
https://github.com/uber/react-vis/blob/master/showcase/examples/candlestick/candlestick.js

Hooks is used at some of the components in the application as they can be used interchangeably with classes. The aim is to cover the whole application to hooks in future (talking of strategy, had I known typescript, should have written it ts).

Hooks do not rerender if state does not change. I cannot get the component to update state here for example
https://github.com/layik/eAtlas/blob/master/src/components/DeckSidebar/HexPlot.jsx#L8

In #38 we have a method to define DeckGL layer properties and use them either in generating layers or indeed in setting their values or assigning values from columns.

For this issue:

• one source of truth where Layer is defined and used consistently throughout
• define current set of layer props

For the latter, we can have:

1. number (numeric)
2. column (value from column), this is turned into a function on the fly
3. colour (array) is used in DeckGL layers

This could do most of what eAtlas needs to do.

The best course of actions is to ignore the defaultURL when data is provided. Or it could be even ignoring both if data is provided. Either will do.

Do the domain generation only once.

#12

Given:

• year as time column
• column name as layer column
• data on client side (standalone npm case)

Generate: tooltip graph.

Our other showcases rely on custom code to pull data on tooltip hover. Is it possible to detect at least year + current column used to generate say an area thematic map.

One thing I have not been able to work out is being able to switch themes. Currently the major hurdle is "Slider" from BaseUI itself. When using styled or styletron functions to override css, it complains about some unmounted state issue which definitely does not happen when I remove the overrides affecting Slider component.

Pushed it into a new branch for future ref.

There are two imminent DeckGL layers in the horizon. With the WP bar vis almost ready to put on npm js.

1. scoped? Yes!
2. How? don't know, options are:

• @eatlas/eatlas, @eatlas/deckgl-barlayer etc
• @layik/barlayer ... dont like this
• ?

• arc - requires integrating color picking in layer definitions. Done in #151
• ?

For current (future) releases we must simplify how eAtlas handles date/time. For now:

• ask users to bring ISO dates
• continue using luxon to parse them using .fromISO

uber/react-vis#1303

This is an issue as the project now has grown beyond a quick README. It does need a structure and some good write up.

Hi Layik,

Thanks for making this open source! I had a few questions about your implementation, apologies if these questions would be better suited to the geoplumber repository:

It seems to me that the R backend / geoplumber is used to read geospatial data, convert it to geojson if necessary, and then serve the geojson via API. However, why not cut out the middleman, and get the browser to load geojson directly, from GitHub pages for example? People who want to reuse this wouldn't need to run a server / write R code for each new dataset, and the geojson could be cached by the browser / compressed / served over a CDN. R / plumber / geoplumber operate single threaded - so if one user is requesting some data that takes a long time to download, it can slow down the experience for other users. This can be mitigated by running multiple docker containers and load balancing across them - but this seems like over-engineering when there are freely available CDNs that would be more performant.

For example, with my wired gigabit connection here in Auckland, New Zealand:

time curl http://eatlas.geoplumber.com/api/covid19 > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 43.2M  100 43.2M    0     0   324k      0  0:02:16  0:02:16 --:--:--  501k

real	2m16.520s
user	0m0.186s
sys	0m0.431s

It takes 2 minutes and 16 seconds to load 43MB from your geoplumber.com server, probably due to the 300ms ping.

Cheers,
Nick Young
Centre for eResearch, University of Auckland

README.Rmd is generated from README.md using markdown::render. Since we are regularly updating README.md with installation/setup instructions, this is a bit of a pain.

• generate .md when .Rmd modified?
  • new .git.hooks folder for storing commit hooks in repo
  • create .git.hooks/pre-commit and make executable
  • symlink to .git.hooks/pre-commit from .git/hooks
  • install rmarkdown into RStudio from CRAN
  • install pandoc ❌ package ‘pandoc’ is not available for this version of R
  • lazy installation of required packages in pre-commit
• [alternatively] delete README.Rmd

So far TGVE has been relying on hover. Having both options will be useful to variety of projects.

The data from QUANT is literally huge. Thanks to Dr Richard Milton from UCL for providing the data.

That means DeckGL layers might not have built in layers to assist. This ticket is a place-holder for that.

• clean up and branch here
• tested locally
• minified as one file (not easy due to composite layer)
• other?

On 1.3.1-beta-0 release, geojson layer does not trigger stroke etc layer options

Particularly when considering #36 etc.

Just a friendly issue after we picked up that for example icons only show once you zoom in enough which should perhaps aggregate to a nearer icon.