reichlab / covid-19-hub-vizualization Goto Github PK

There is some duplicated code for calculating data for plot layers representing 50% and 95% intervals and point forecasts here. Let's reduce duplication by extracting the relevant code into two functions we can call -- one that calculates and returns layers representing point forecasts and one that calculates and returns a layer representing interval forecasts.

Since the hospitalization data are updated daily, would it be possible, for the current week only, to update the ground truth data daily so that it shows the most recent truth data?

Screenshot below from full-screen image on MacBook Pro 13 inch.

• change "Target variable" to "Outcome"
• layout of truth data both on same line doesn't seem to be working. Reduce size and keep on separate lines?
• make font size of labels of drop-downs and and drop-down menu text smaller
• spacing of disclaimer text is weird - only one word on third line. Can we make the horizontal space for the text larger? reduce font-size?

Last two of these I see as "nice-to-have" but first two I think need to be updated before publicizing.

(COVIDhub-4_week_ensemble, KITmetricslab-...). Is there a logic that needs to be updated in the data retrieval code? Also, maybe those models are there - but why isn’t the list in alphabetical order?

• Matt: We should pass an initial_as_of property as an option to predtimechart.
• Evan: This should be calculated as the most recent as_of date for which deaths forecast data are available. This should be loaded from a new file saved in static/data/initial_as_of.json with contents

{
    "initial_as_of": "<yyyy-mm-dd>"
}

This will involve a change here

“Most forecasts have failed to reliably predict rapid changes in the trends of reported cases and hospitalizations. Due to this limitation, they should not be relied upon for decisions about the possibility or timing of rapid changes in trends.”

Change "As Of" to "As of".

Should Truth data be a radio button perhaps?

Example (reproduce by unselecting truth data and using right/left cursor keys to navigate to the date):

Don't know if this might be related this closed issue but looking at Oklahoma deaths when they came up at MIDAS today I noticed KITmeticslab ensemble not looking right for at least the last few weeks.

• In the layers for the point prediction add a new value to the beginning of the x and y arrays with the most recent observed data point from Truth As Of.
• Similar for the intervals

Header from https://covid19forecasthub.org/
Yuxin will find old code that was used to add this to an old viz

Ensemble hospitalization forecasts with only 14 days ahead show points. The forecasts with 28 days show just a line. For consistency, I think the ensemble forecast should be just the line. If there is an adjustable threshold for how many "steps ahead" to show before not showing points, I think it should be set at 7.

• "Current truth" to "Truth as of <last observed truth date>"
• "Truth As Of" to "Truth as of <selected reference date>"

manual update of the site involves

• updating lines of code that call the first file shown
• run R script to get forecasts
• run R script to get truth values

currently, every 10th model will be assigned the same color. A use case for the shuffle colors button will be a situation where two models are assigned the same color, and we want to disambiguate them. To support this, let's more directly associate a color with each model, and shuffle those assignments across all models when shuffling colors.

When you shuffle the colors, the color of the initial line segment that connects the model with the observed data changes to a different color than the main line.

Before shuffling:

After shuffling a few times

Because the truth data ends on Saturday, there is a few day period where daily observations that were available to models are not shown. Could this be adjusted so that the last ground truth observation for each hospitalization "as of" date is from Monday?

we have hard coded the initial data that are loaded in at

Instead, could we load it from whatever the latest available file is, maybe using something like the below?

import current_truth from '~/static/data/truth/case_US_' +
    available_as_ofs.case[available_as_ofs.case.length - 1] + '.json';
import forecasts from '~/static/data/forecasts/case_US_' +
    available_as_ofs.case[available_as_ofs.case.length - 1] + '.json';

We can eliminate a dependency on covidEnsembles by replacing the call to covidEnsembles::load_covid_forecasts_relative_horizon here with calls to covidHubUtils::load_forecasts and covidHubUtils::align_forecasts.

• currently initialized to "selected", though not all models are selected initially. Suggest changing so that initially it is not selected
• if we select all models while viewing a particular outcome variable (e.g. deaths), only models with forecasts for that variable are selected. this results in unselected models when viewing other variables. Suggest either changing so that "select all" selects all models, regardless of whether or not they provided forecasts for the current target, or updating its status to unselected (without changing model selections) if the outcome variable is changed and this results in available models that are not selected.

When you load the page and switch to a new outcome, all models are selected. not just the ensemble.

The desired behavior in this situation is to:

• unselect the individual model the user clicked on (but leave it in the list of models used to populate the legend)
• unselect the "select all" box (but not change any other model selections)

There are two goals here:

1. track text for plot labels in json rather than calculating it on the fly in javascript. This will make the code more transferrable to other projects, and simplify adding cumulative deaths to this project later.
2. reduce the number of different ways we are referring to the same quantity

We currently refer to the names of the outcome variables in three different ways:

1. the variable names in the drop down menu for "Target Variable". This comes from this json file, and is one of "Cases", "Deaths", or "Hospitalizations"
2. the description used in the plot title. This is calculated in the javascript here, and is one of "weekly COVID-19 cases", "daily COVID-19 hospitalizations", or "weekly COVID-19 deaths"
3. the description used for the vertical axis label in the plot. This is calculated in the javascript here, and is one of "Incident weekly cases", "Incident daily hospitalizations", "Incident weekly deaths"

Let's:

• decide on a single naming scheme for use in the plot labels -- I prefer the second format, e.g. "Incident weekly cases"
• Add the selected text as entries in the json file for target variables in a new field, e.g. "plot_text"
• Use this text in both parts of the plot label

Here is an unexpected behavior with the "select all" button.

• I load the page. select and deselect a given model. then click select all. This selects all models EXCEPT the single model that I have deselected. It's as though my manual "deselection" has turned it off for good.

Note the difference in the tab icon.

currently, the GitHub button at the top links to the main forecast hub repo, which is consistent with main header, so this should stay. But somewhere else on the page we should have a link to the "source code for visualization". Maybe small text, bottom left of page? small text bottom of legend column?

• add statcounter code #6
• fix issue #36
• push latest changes
• get app up and running on subdomain viz.covid19forecasthub.org
• change links in menu (in main bootstrap theme and in custom header in viz itself, and link on main page and other places)

there are currently some blank lines and inconsistencies in the use of spaces and tabs in the javascript. Let's add a linter to help standardize.

e.g. The JHUAPL-SLPHospEns model is submitted every week to the hub, but they often miss the 8am deadline on Tuesday morning for submissions of ensembles to be included in the visualization build. It appears that then those forecasts are not added in subsequent weeks. Is there a simple fix to ensure that forecasts like this, that are validly submitted after we update the viz, are included in future weeks?

Note that the point that we have added to manufacture the full polygon that originates from the last observed datapoint should not be present in the tooltip when hovering on that point, as it isn't part of the forecast. Ideal behavior would be:

• there should be a line from the end of the truth data to the end of the prediction points, going through each point. This line already exists in the viz
• There should only be actual points at the "point" prediction values, but there should not be a colored point at the "manufactured" point at the last truth value.
• The tooltip should show the point estimates for all values of the truth data (it does this already), and for the forecasted points that are shown (it does this already), but not for the initial ghost point of the models (this would need to be changed).

To do this, we need to:

• Add the analytics.js file back in
• Add the html referencing this file to index.html

to reproduce:

• make some model selections
• change selected location

We should keep the existing model selections in place when we change locations.

We should add a function to the store/index.js file in this repository that returns the config parameters object for the nuxt module. After loading relevant json files currently at the top of the nuxt.config.js file (i.e., we need to move to loading of those json files over from nuxt.config.js to store/index.js), basically just returns what's currently specified for module options:

  async forecastViz_options(_) {
    return {
        target_variables: target_variables,
        init_target_var: 'death',
        locations: locations,
        init_location: 'US',
        intervals: ['0%', '50%', '95%'],
        init_interval: '95%',
        available_as_ofs: available_as_ofs,
        init_as_of_date: available_as_ofs.death[available_as_ofs.death.length - 1],
        current_date: available_as_ofs.death[available_as_ofs.death.length - 1],
        models: models,
        default_models: ['COVIDhub-ensemble'],
        all_models: false,
        disclaimer: 'Most forecasts have failed to reliably predict rapid changes in the trends of reported cases and hospitalizations. Due to this limitation, they should not be relied upon for decisions about the possibility or timing of rapid changes in trends.'
      }
  }

Then, remove those module option specifications from nuxt.config.js.

The goal is to extract the main functionality for the plot and UI selection into a separate place that can then be pulled into other projects, e.g. a flu viz or a Zoltar viz. I'm not totally sure, but I think the best way to achieve this is to create a module and/or plugin that contains the plot, options selector, and interactions with the data store. These could then be imported into the current project and other projects as needed. More information at these links:

• https://nuxtjs.org/docs/directory-structure/modules
• https://nuxtjs.org/docs/directory-structure/plugins
• https://nuxtjs.org/tutorials/creating-a-nuxt-module/
• https://medium.com/carepenny/creating-a-nuxt-module-1c6e3cdf1037
• I think this might be a reasonable example to follow, similiar to ours in that there is a .vue component to make available: https://github.com/nuxt-community/recaptcha-module/tree/master/lib Note the line Vue.component('Recaptcha', () => import('./recaptcha.vue')) in lib/plugin.js.

related to issue #77 , but this is issue relates to doing a one-time backfill to make sure that all forecasts that have been missed are included.

right now, you have to get right on the check-box. Can we make it so you can click on the name or the color button to select/deselect a model?

I think it might help us to generalize the functionality (e.g. support Zoltar or other vis efforts) if we:

• extract the navigation menu into its own component
• add it to the page by using a layout

Rather than having the outcome menu be
Cases
Deaths
Hospitalizations

Let's change the order to be
Deaths
Cases
Hospitalizations

With Deaths as the default choice that displays when the page first loads.

I think currently this includes models_to_add and interval_level

• We should sort the models alphabetically and then put the COVIDhub models at the beginning when the json file is created, here.

• We should not sort them in the javascript here.