How do we handle different data formats? about covid-api HOT 5 CLOSED

I haven't thought about this very much, but seeing as the reports won't be consistent between organisations, maybe we could simplify it to just /itapc or jh instead of categorising them further with national-reports or world-reports?

Then if you query the root of the api we could return a list of all the reports available.

So something like

GET /api/v1/ ->
[
{
   "source": "Protezione Civile",
   "reports_url": "/api/v1/itapc/"
},
{
  "source": "John Hopkins CSSE",
  "reports_url": "/api/v1/jh/"
}
]

GET /api/v1/jh/ -> what's now /daily-reports

GET /api/v1/itapc/ -> all the data from the Protezione Civile dataset

I think it makes sense to model each source independently and reuse column names from the original dataset, rather than trying to map each report to a common vocabulary, because the exact meaning of each metric will depend on how its collected/recorded.

from covid-api.

I think separate endpoints for each data source is a great way to go. It will make the payload size smaller for your consumer since they can be more granular in their fetch.

from covid-api.

Perhaps adding an abstraction layer will make it easier to add in new datasources.

def john_hopkins_data():
  return fetched_data()

source_map = {
  "john_hopkins": john_hopkins_data,
  "another_source": another_source
}

def get_data(source_map, source_name):
  getter = source_map[source_name]
  return getter()

Doing something like this would make it easy to add new sources without updating your api. Serious downside though is this requires magic strings.

from covid-api.

Perhaps adding an abstraction layer will make it easier to add in new datasources.

def john_hopkins_data():
  return fetched_data()

source_map = {
  "john_hopkins": john_hopkins_data,
  "another_source": another_source
}

def get_data(source_map, source_name):
  getter = source_map[source_name]
  return getter()

Doing something like this would make it easy to add new sources without updating your api. Serious downside though is this requires magic strings.

Oh I see! So the end user would only call GET /api/v1/daily-report

and the response would contain something like:

{
    "john_hopkins": {
        .... (data from JH)
    },
    "protezione_civile": {
        .... (data from Italian PC)
    }
}

and it would be up to the user to pick the one they want, right?

We could even include the possibility to limit the returned source/sources or to exclude one we don't want.

Do we agree that each data source should have their own models? Cheers

from covid-api.

I'm closing this since we agreed on a solution.

from covid-api.