Giter Club home page Giter Club logo

dataseries's Introduction

dataseries

License:MIT

data-series functions support for data-series and time-series.

rust

build codecov docs.rs crates.io crates.io (recent)

java

build codecov maven central javadoc

functions

union

Continuous time series union between 2 series. Left and right data can be absent (left and right only cases).

          1     3     10                 20
    Left: |-----|-----|------------------|-
          130   120   95                 160
                           12     15
   Right:                  |------|--------
                           105    110
          1     3     10   12     15     20
Expected: |-----|-----|----|------|------|-
          130,∅ 120,∅ 95,∅ 95,105 95,110 160,110

eventual consistency and conflict resolution for data-series

The crdt example provides an example of the conflict-free replicated data type resolution based on data-series union function and VersionedValue type to solve conflict with a timestamp (any variable supporting partially ordered set) for rust and java.

trade-offs

interval representation

Half-open interval                      Data-series (time-series/gauge)
(n value and 2n points/delta)           (n+1 datapoints)
                            \           
1    3    5                  \          1    3    5           +∞
[----[----[                   \         |----|----|------------
 100   120                    /         100  120  ∅
                             /
                            /
pros cons
half-open interval +same read and write model -illegal state representation
-requires global secondary index to support range queries
data-series/time-series +nosql and TSDB friendly
+less illegal states
+compatibility with time/data-series functions and visualization
+compact format (requires only n+1 datapoint intead of 2n))
+partitionning is trivial (only one dimension)
+updates are less complex (no need to update impacted points of interval)
-read and write models are different
-hole should be represented with a datapoint None value

An interval can be defined by using 2 points (upper and lower bound) with an associated value but it can be difficult to index those 2 points in nosql databases (Global secondary index) or simply using a TSDB (timeseries database).

Another approach consists of defining an intermediate model, a data-series with only one point and one value so that the datapoint fits really well with TSDB and is algorithm friendly.

It becomes also easy to avoid unwanted states; an interval can be defined with 2 points and the last point can be before the first one which is a bug in the domain. You can also define a point and a non negative offset which can work but requires more code.

For database support, interval reading requires 2 reads to compute the interval but the extra read can be hidden in easily in an Iterator.

implementation

rust and java implementation are provided in respective directories.

dataseries's People

Watchers

 avatar

dataseries's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.