Comments (5)
Hi There,
Thanks for your message. At the moment we are not planning on making this update. The numbers reported by BC CDC and Alberta Gov't are reported using internal report dates-- so we are not exactly sure which days specifically the cases are added (as these dates do not align with our variable definition of public report dates). We are in the process of setting up some data linkages, if these are successful we hope to include this information.
We realize that Monday's are hyper-inflated and note that in our dashboard for Mondays. We also recommend that individuals using this data look at trends and 7-day averages as opposed to single-day counts to better understand current trajectories.
Hope that helps!
from covid19canada.
Using the dates from the BCCDC CSV would result in distorted real-time data as well.
Take this Monday's dataset: https://github.com/jeanpaulrsoucy/covid-19-canada-gov-data/blob/master/bc/case-data/BCCDC_COVID19_Dashboard_Case_Details_2020-10-05_23-02.csv
BC reported 358 cases on Monday. But only 11 of them have the date of 2020-10-05 (Monday), with the rest having dates earlier than this (mainly the weekend). This is not unique to Mondays - this happens every single day. I believe there have even been days when every single case added is given a date earlier than the current date.
Of course, this distortion is corrected over time as cases are "backfilled"...but using the dates from the BCCDC CSV (internal reporting dates) will result in constant real-time distortion of the most recent dates, mainly the current date.
Date a case was reported to the public is the only date variable that is consistent across all provinces.
Our on-going data linkage project should allow us to provide an alternate date column for cases in certain provinces (ON, BC, AB). Stay tuned for updates on this.
I have also considered including the provincial CSVs as alternate datasets in the repository. It may be possible to further integrate this into our API by providing an option to use our dataset (report date) or the official date (internal report date) when returning the time series. This would give the best of both worlds and is technologically feasible.
Let me know if you have any further questions.
from covid19canada.
Suggestion: Have a column for crowd-sourced numbers. All it would take is one person to enter the weekend numbers reported on Monday. The repo is a great resource and it would be helpful not to have these weekend discontinuities. Even with 7-day averaging they distort the curves.
from covid19canada.
@gauss256 I don't think that would work with our process, unfortunately. All of our time series are completely re-generated daily from cases.csv and mortality.csv (using update_data.R: https://github.com/ishaberry/Covid19Canada/blob/master/scripts/update_data.R). BC frequently edits old cases - removing/moving them - which we take into account by comparing old and new datasets. These are then reflected in our updated time series. At the moment, we have no way to feed in crowd-sources data to our private Google Sheets that are used to generate cases.csv and mortality.csv each day. The good news is that our data linkage script (prototype for ON here: https://github.com/ishaberry/ON_data_link) should mainly solve this problem, and be robust to changes in historical data, since it can be periodically re-run on the entire dataset. We just need to adapt it for BC, which is on my to-do list.
Perhaps a stop-gap alternative would be writing a script that pulls in the official BC dataset each day and formats it like our dataset - creating a drop-in alternative for our BC dataset.
from covid19canada.
Perhaps a stop-gap alternative would be writing a script that pulls in the official BC dataset each day and formats it like our dataset - creating a drop-in alternative for our BC dataset.
That's approximately what I'm doing now.
Anyway, thanks for all your good work on this. Looking forward to data linkage for BC.
from covid19canada.
Related Issues (20)
- Replace NT testing time series with PHAC time series
- Updates of NT sub-hr data must be more robust
- RE April 15 changes .. not clear if old dataset remains in current Github repo and will no longer be updated? Or will it be deleted entirely? HOT 1
- Third doses HOT 2
- Update population values HOT 1
- Question about versioned data for number of cases HOT 1
- Question about Alberta-Central health region case numbers from 05-02-2020 to 06-09-2020 HOT 1
- Rewrite MB vaccination time series HOT 2
- Add NT data update to workflow
- Temporary manual data collection
- Donations? HOT 1
- NT sub-region update: Add retry HOT 1
- NS vaccine 2nd and 3rd dose data
- PEI appears to count deaths in the recovered category HOT 2
- Update NT sub-hr data
- Validation script: Use proper tables for metric summaries
- Halimand-Norfolk HOT 1
- Requirements to replace Ontario PHU datasets with the Ontario Ministry of Health dataset
- Saskatchewan HOT 1
- NT sub-hr data not being updated
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from covid19canada.