curran / data-reduction Goto Github PK

Use d3-time for aggregation over nice time intervals.

Instances of chiasm-dataset should be used as input and output from data-reduction.

There are really two subtasks here

• Use chiasm-dataset as input.
• Use chiasm-dataset as output.

Just a note that you should update the dependencies. d3-arrays and d3-scale especially. I'll do a PR later.

Develop a POC that computes the multidimensional aggregations using Crossfilter, and test the performance vs. the current implementation. Hypothesis: using Crossfilter will be much faster, because it's highly optimized.

Currently, only "nice interval" bins are supported. However, there are use cases to have a fixed number of bins that do not fall on nice intervals. An option should be added to support this.

The Datalib project found at https://github.com/vega/datalib looks like a great library that implements many of the features that data-reduction does. It needs to be evaluated in depth to see if data-reduction could potentially be entirely deprecated in favor of datalib.