crertel / cityxr-data-engine Goto Github PK

(check the config table is_disabled flag for where we might be able to store this)

The REST api needs to allow users to query a data source's values at a particular time.

Users should specify:

• What data source they want
• What fields they care about

We need a page to view the different run status and data values. This can live in the ingest server, which already has a lot of this.

This needs to:

• show what data sources are available
• show what data schema the data sources have
• show what their run status is and history is
• visualize the data

The REST api needs to allow users to query a data source's values between two times, on a closed-open interval.

The data server needs to be able to send updates to clients that are subscribed to a data source over websockets.

Subscribers are notified once a run completes. Historical data must be fetched using the API.

Process of subscription for websockets is:

1. Server maintains a list of active data sources and their subscribers
2. Client subscribes to a data source
3. On update from the database (NOTIFY perhaps? see example gist somebody used here), notify subscribers through websockets that new data is available.

Clients subscribe using this method:

{
  "ds": "<datasource id>",
  "a": "sub"
}

To unsubscribe:

{
  "ds": "<datasource id>",
  "a": "unsub"
}

Once subscribed, a client should expect messages of the form:

{
  "t": "<RFC3339 timestamp of sending time",
  "rid"
  "ds": "<datasource id>",
  "d" : [
   <data frames as objects whose keys are the column names and whose values are the data points> 
  ]
}