atfutures / upthat Goto Github PK

Plus a small to translate from file names to city names.

See below, any ideas @mpadge ?

You up for taking a look @mpadge ?

I see:

To be replaced with:

Heads-up @mpadge

https://atfutures.github.io/uta/

Any ideas @mpadge ? This is the log from shiny-server, not sure if this contains the issue:

Listening on http://127.0.0.1:37321
Warning in force(expr) :
  strings not representable in native encoding will be translated to UTF-8
Warning in force(expr) :
  input string '��' cannot be translated to UTF-8, is it valid in 'UTF-8' ?
Warning in force(expr) :
  input string '��' cannot be translated to UTF-8, is it valid in 'UTF-8' ?
Warning in force(expr) :
  input string '��' cannot be translated to UTF-8, is it valid in 'UTF-8' ?
No mapdeck token found on system. Trying environment variable MAPBOX
Warning in readRDS(system.file("net-kathmandu.Rds", package = "upthat")) :
  strings not representable in native encoding will be translated to UTF-8
2 files found
Registered S3 method overwritten by 'jsonify':
  method     from    
  print.json jsonlite
2 files found
Warning in readRDS(system.file("net-kathmandu.Rds", package = "upthat")) :
  strings not representable in native encoding will be translated to UTF-8
Warning in readRDS(system.file("net-kathmandu.Rds", package = "upthat")) :
  strings not representable in native encoding will be translated to UTF-8

Age-gender are most important variables.

Baseline level vs new level -> Marginal METs, one way of measuring all physical.

to modify

• exposure - change to direct pollution exposure, instead of mortality effect thereof
• vehicular flow layer - should be in same units (trips per day) as pedestrian flow

to add

• layer of pedestrian exposure to vehicular flows
• exposure of pedestrians to vehicular crash risk
• exposure to green (via satellite NDVI

This list should be viewed as potential additional layers, to be moved to own issues if and when we agree to proceed @Robinlovelace please indicate your thoughts on the latter two in particular, and suggest any other layers you can think of

To prevent the whole map reloading with each interaction: https://rstudio.github.io/leaflet/shiny.html

@Robinlovelace I've been through all of your excellent code for adapting the PCT approach to scenario modelling, and am pretty confident that it all adapts fairly directly to network-level analyses. I also (finally) understand the way that scenarios are constructed, and the way they depend on the fundamental distribution of bicycle trip distances. And so this issue is to ask whether it might be possible for you to use the NTS data you used for the analysis of walking trips per hour to generate an equivalent distribution of either times or lengths (or, even better, both!)?

Given that, i should be able to fairly quickly adapt the general PCT approach to modelling cycling propensity and demand to pedestrian behaivour, and generate realistic scenarios for changes in walking behaviour in response to the addition of bus stops.

I get a crash on loading the master version. Any ideas what's causing this @mpadge ? Tried to debug but failed so far. Reprex (let me know if you can reproduce this):

devtools::install_github("atfutures/upthat")
#> Skipping install of 'upthat' from a github remote, the SHA1 (15d75686) has not changed since last install.
#>   Use `force = TRUE` to force installation
upthat::runUpthat()
#> Loading required package: shiny
#> 
#> Listening on http://127.0.0.1:6853
#> Reading this matching file: /home/robin/R/x86_64-pc-linux-gnu-library/3.6/upthat/net.Rds
#> Warning: Error in if: argument is of length zero

#> Warning: Error in if: argument is of length zero

^{Created on 2019-12-17 by the reprex package (v0.3.0)}

The latest version of the reproducible spatial interaction model (SIM) testing script, using only the Euclidean distance of the desire line and distance of origin and destination from the city as predictors, all of which can be calculated for any city when divided into an appropriate zoning system (more on that soon in relation to work with @mtennekes), can explain ~50% of the variability in commute numbers at the OD level based on example data from Bristol. Results of the actual data (left), unconstrained SIM (middle) and 'production constrained' SIM (right) shown below.

We can estimate mode shift under different scenarios at the OD level. The question is, how to integrate this with the SNA approach? Calculating the route for each OD pair for each mode of transport could give an indication of physical activity and, provided an air pollution exposure layer is available, air pollution, for all travellers travelling along each desire line.

An idea I'm keen on is treating 'working from home' as a mode of travel, with resulting reduction in energy use and exposure for long trips that are more likely to switch to telecommuting, although appreciate that at present this is more relevant for high income nations at the moment (although that may change).

Heads-up @mpadge and @thiagoherickdesa - this is work in progress but runs super fast, is easy to generalise, has minimal data requirements and should generate plausible, testable, geographically differentiated estimates of health impacts under a range of scenarios. My aim is to generate results for Bristol, Accra and (for sanity checking) Leeds.

At the moment, data are dumped into main directory, but it would be good if upthat only ever looked in inst for data. I'm uneasy with the current procedure of just taking first file if copies are both in main dir and inst.

As discussed, heads-up @mpadge

• New Demo City with tiny size but same structure for testing

• Kathmandu

• NYC

• Bristol

For some reason there are 2 travis CI outputs from this:

Is one of them linked to your account @mpadge ? I think we should turn one off.

@SymbolixAU A mapdeck issue for you, if i may. This repo is a shiny app that uses mapdeck, and has a toggle to switch between cities. Each switch then auto-moves to mapdeck a bunch of paths. The problem is something is aggregating in memory, so each toggle progressively eats up memory. The cause is this (from the "Memory Dominators" report in Firefox):

This seems to my naive understanding to suggest that the array objects that are eating it are the deck.gl data. Any advice or help would be greatly appreciated!

Here is the server script, and you can reproduce the issues by locally loading big enough data (presuming you're in the root dir of this repo):

f_accra <- "https://github.com/ATFutures/who-data/releases/download/v0.0.5-moveability/accra-moveability-sf.Rds"
f_kath <- "https://github.com/ATFutures/who-data/releases/download/v0.0.5-moveability/kathmandu-moveability-sf.Rds"
download.file (f_accra, file.path (here::here(), "inst", "accra-moveability-sf.Rds")) 
download.file (f_kath, file.path (here::here(), "inst", "kath-moveability-sf.Rds"))

Then just upthat::runUpthat(). (Oh, and you'll first have to ensure your mapbox token is called just "MAPBOX", then it's auto-found so this whole thing can auto-deploy from docker.) Your help in reclaiming the eaten memory would be hugely appreciated! Thanks

Currently we have this:

See here for code and here for great keynote. This could lead to a step change in reproducibility. Thoughts @mpadge ?

Quick one: the results in the new health tab (which overall look great btw!) should I think:

• Respond to changes in the case study city made on the interactive map
• Not duplicate the dropdown menu in the map tab
• Currently returns error message

Thoughts @mpadge ? See below a GIF that highlights the impact of these issues for the user:

Currently we have:

net = readRDS(system.file("net.Rds", package = "upthat"))
head(net)
#>        from_id      to_id edge_id from_lon from_lat   to_lon   to_lat
#> 38  2589407293 5174857024 a246623 85.31030 27.71481 85.31023 27.71390
#> 94  1875597194 1968417633 a246544 85.31218 27.71264 85.31218 27.71161
#> 410 5533222482 5662991935 a246042 85.31002 27.71588 85.30925 27.71545
#> 436 3487459912 5093800713 a246004 85.31055 27.71481 85.31068 27.71428
#> 485  724316510  993710686 a245932 85.30959 27.70607 85.30819 27.70639
#> 490 1275284544 6193720260 a245921 85.30759 27.71200 85.30657 27.71336
#>             d   object_      highway oneway.bicycle oneway:bicycle
#> 38  101.33715 133051041 unclassified           <NA>           <NA>
#> 94  114.35734  24683729  residential           <NA>           <NA>
#> 410 119.90618  56688294  residential           <NA>           <NA>
#> 436  82.04076 523171220      footway           <NA>           <NA>
#> 485 142.90122  58425711     cycleway           <NA>           <NA>
#> 490 182.68254 112117466  residential           <NA>           <NA>
#>     d_weighted     time time_weighted      flow component
#> 38   101.33715 36.48137      36.48137 113495846         1
#> 94   114.35734 41.16864      41.16864 116615984         1
#> 410  119.90618 43.16622      43.16622 109072960         1
#> 436   82.04076 29.53467      29.53467 100494246         1
#> 485  142.90122 51.44444      51.44444 108080837         1
#> 490  182.68254 65.76572      65.76572 119321239         1
#>                                                                                                                                                                                                                                                                                                                                                                                       geometry
#> 38                                                                                                                                                                                                                                                                      85.31030, 85.31027, 85.31024, 85.31024, 85.31023, 85.31023, 27.71481, 27.71459, 27.71425, 27.71416, 27.71411, 27.71390
#> 94                                                                                                                                                                                                                                                  85.31218, 85.31215, 85.31215, 85.31215, 85.31215, 85.31216, 85.31218, 27.71264, 27.71225, 27.71204, 27.71194, 27.71184, 27.71172, 27.71161
#> 410                                         85.31002, 85.31002, 85.31001, 85.31001, 85.30999, 85.30980, 85.30962, 85.30961, 85.30961, 85.30960, 85.30960, 85.30959, 85.30947, 85.30931, 85.30930, 85.30930, 85.30925, 27.71588, 27.71586, 27.71582, 27.71581, 27.71581, 27.71582, 27.71583, 27.71583, 27.71582, 27.71571, 27.71569, 27.71569, 27.71570, 27.71570, 27.71570, 27.71569, 27.71545
#> 436 85.31055, 85.31054, 85.31052, 85.31051, 85.31050, 85.31049, 85.31049, 85.31050, 85.31048, 85.31048, 85.31053, 85.31057, 85.31063, 85.31067, 85.31070, 85.31070, 85.31069, 85.31070, 85.31068, 27.71481, 27.71478, 27.71475, 27.71471, 27.71467, 27.71462, 27.71460, 27.71457, 27.71448, 27.71445, 27.71445, 27.71445, 27.71445, 27.71444, 27.71444, 27.71439, 27.71435, 27.71431, 27.71428
#> 485                                                                                                                                                                                                                                                                                         85.30959, 85.30909, 85.30902, 85.30827, 85.30819, 27.70607, 27.70621, 27.70624, 27.70635, 27.70639
#> 490                                                                                                                                                                                                         85.30759, 85.30747, 85.30732, 85.30722, 85.30712, 85.30700, 85.30687, 85.30668, 85.30657, 27.71200, 27.71213, 27.71226, 27.71236, 27.71249, 27.71268, 27.71285, 27.71315, 27.71336

^{Created on 2019-07-18 by the reprex package (v0.3.0)}

There are some columns we don't need in there. There are also some columns we should add.

When I load the data for New York City, and click on the menu, it takes a few seconds to load, during which time the main screen looks like it normally does. This has the effect of making it seem - for those brief few seconds - like my selection has done nothing. It'd be good to add a spinny widget that loaded as soon as something is clicked, to give immediate visual feedback that something is happening.

@Robinlovelace I just encountered sandstorm - it looks amazing, and is programmed by the guy who made Protocol Buffers (which is how i found out about it). Free to host a local instance, and has a really different architecture that could be really useful for serving upthat. Just a thought?

@Robinlovelace I've revisited flow layers, so this should now work (presuming run from in the flowlayers directory)

devtools::load_all ("../m4ra", export_all = FALSE) # or from whereever you are
devtools::load_all (".", export_all = FALSE)
plot_flows (city = "kathmandu", from = "residential", to = "bus")

Heads-up @mpadge can you try this please:

piggyback::pb_download("net-accra.Rds")

I think the result is a route network for Bristol.

Currently the flow numbers are bamboozlingly big. We could modify them in-app but I think it would be better if the input data were updated so that they provide counts of trips per day. You up for doing that @mpadge for the Accra and Kathmandu as you have for NYC?

ATFutures/nyped#7 (comment)

To mapdeck or another mapping package such as tmap? One to discuss with @mtennekes...

Leading to outputs like this:

Linked to #21 but opening a new issue as it affects usability and reproducibility. You up for tackling this one @mpadge ?

@Robinlovelace FYI the current Kathmandu layers look something like this:

... where the blob down on the lower left side is because Kathmandu officially stops there, and so the street network does not include Patan across the river. I need to expand it to include Patan as well, which will then enable those flows to disperse across the bridge as they obviously should, and we'll be back to the monocentric pattern we expect there.

It's currently not, so it's not possible to simply switch between viz layers based on user interactions. Easy to fix by switching current static app serving to shinyReactive form

Switch to a layer other than Pedestrian flows, then change city to observe current behaviour of:

1. New city data is loaded 👍
2. Pull-down box for selected layer remains at previous value 👍 ; but ...
3. Only the default layer of flows is shown. To see the desired layer requires first switching to anther layer and then back again to reset the plot. 👎

As .csv file with variable name and description.