Giter Club home page Giter Club logo

timel's Introduction

TimEL: Time-series Expression Language

License: LGPL v3 Build Status Test Coverage Javadocs

TL;DR

Pick a random example in the πŸ“Ί online console and try it yourself!

Why?

Monitoring, metering, IoT, pay-per-use billing: these are only few examples of applications that rely on time-series data! Often you want the final user to be able to manipulate and customize some results based on some time-series data - that's when TimEL comes in handy!

Keep your data: do not resample your time series

Many time-series systems automatically upscale your value to a certain interval (let's say a value every 60 seconds): this produces a lot of duplicates and also limits granularity: what if there is a value change that lasted for less than 60 seconds? The information will be somehow lost, as it will be blended with some other value to accommodate the 60 second boundary constraint.

TimEL won't force you to do that: it will operate directly using your sample timestamps, regardless of their form (aligned, misaligned, constant interval or variable one) or granularity (anything from years to milliseconds).

The granularity of your samples drives the computation, not the other way around!

Do the right thing: gauges vs integrals (counters)

Some time-series data come as a gauge, while some other come as a counter (integral). TimEL encodes this information directly within its typing system, so you do not have to worry.

Let's imagine you're summing two integral (counter) values: if they're not perfectly aligned, TimEL will automatically interpolate when needed.

What?

TimEL is a Java library to compile and evaluate TimEL expressions. TimEL expressions are written in a user-friendly language that allows time-series manipulation without the need of taking care about upscaling, downscaling or mixing up different time series intervals.

Let's see an expression to count the number of days:

scale(                                      // (3) and then downsample for the whole interval
    scale(
        uniform(1.0),                       // (1) let's take an integral value 1.0
        every(1, "DAY_OF_YEAR", "UTC")      // (2) repeat it every day
    )
)

If we evaluate this expression for an interval in the same day, let's say 06:00-18:00, it'll report 0.5 - that is half day. If we evaluate it for more days it will count how many days are contained in the interval. The function uniform here returns an integral, so TimEL knows how to interpolate it properly - that is handled by the interpreter so the user does not need to worry no more about time frames.

Features

With TimEL you can:

  • Mix multiple time frames - for example you can sum daily data with hourly data, or even non-regular data like monthly data;
  • Express easily recurrent quantities, like 10 units every hour;
  • Scale natively integral values (like consumptions) and averages;
  • Stream results without the need of having all the operands in memory;
  • Support integer, floating point and double expressions;
  • Extend with your own types and functions;
  • Evaluate expression securely - by default there is no way to access external JVM objects or methods that would expose you to a security risk.

TimEL requires Java 8 and will run in any J2SE or J2EE container. For complete project information, see TimEL's website.

Quickstart

To use TimEL you need to import the following dependency:

Maven

<dependency>
    <groupId>net.vleo.timel</groupId>
    <artifactId>timel-core</artifactId>
    <version>0.9.3</version>
</dependency>

Gradle

implementation 'net.vleo.timel:timel-core:0.9.3'

Now you're ready to go! Let's count how many days passed since (unix) epoch:

// Compile the expression
Expression<?> expression = TimEL
        .parse("scale(scale(uniform(1.0), every(1, \"DAY_OF_YEAR\", \"UTC\")))")
        .compile();

// Evaluate and print the number of days from epoch
Interval interval = Interval.of(Instant.ofEpochMilli(0), Instant.now());
System.out.println(TimEL.evaluate(expression, interval).next());

For a more detailed guide refer to the quickstart guide on TimEL's website.

Language

On TimEL's website you can find a list of available types and functions.

Extending the language

You can extend TimEL language by adding new types, conversions as well as functions. Refer to the extension page on the homepage.

timel's People

Contributors

aleofreddi avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

imperatorx

timel's Issues

Detect duplicate signatures as soon as they are added

Describe the bug
Detect duplicate signatures as soon as they are registered to the function registry.

Actual behavior
FunctionRegistry registry = ...; registry.add(function); registry.add(function);
does not detect the duplicated.

Expected behavior
The code above should throw IllegalArgumentException on the second add.

Runtime
Any

Additional context
/

Instantiating an invalid type leads to RE

Describe the bug
Compile (Integer<1>) 1

Actual behavior
Throws a ParseException wrapped in a RuntimeException.

Expected behavior
Throws a ParseException.

Runtime

  • TimEL 0.9.3-SNAPSHOT (14e3ae9574c5d66e73f6e51554349219b7761c34)

Additional context
/

Add boolean constants

Is your feature request related to a problem? Please describe.
TimEL should support true and false constants.

Describe the solution you'd like
True and false should be constants so that things like
when(a > 10, 10, b > 5, 5, 1==1, 0)
can be rewritten as
when(a > 10, 10, b > 5, 5, true, 0)

Describe alternatives you've considered
/

Additional context
/

Parse identifier unicode escape sequences

Describe the bug
TimEL's grammar supports unicode escape sequences as \uHHHH and \UHHHHHHHH to be used in identifiers, but these are not handled correctly.

Actual behavior
Running hello\U0001f602 = 1 defines a variable called hello\U0001f602

Expected behavior
Running hello\U0001f602 = 1 defines a variable called helloπŸ˜‚

Runtime
Any

Additional context
/

The uniform function is dangerous when used with variables

Describe the bug
uniform and the current implementation of variables don't play well together.

The problem arises due to the temporally-unbound nature of uniform, because the result varies semantically (not only by resolution) on the evaluation time frame.

Actual behavior
For the interval 07/06/2019 10:09 PM β†’ 07/07/2019 12:00 PM, evaluate the following:

a = extract("DAY_OF_WEEK", "Europe/Rome") == 7 ? uniform(1.0) : uniform(0); 

scale(
    a, 
    every(1, "DAY_OF_MONTH", "Europe/Rome")
);

The expected result would be 0.5, actual 0.75! That one is due to the uniform(1.0) being interpreted differently during the first evaluation and the second one.

Expected behavior
Such inconsistency should not happen, how it is still unclear.

Runtime
Any

Additional context
Maybe forcing uniform to be time-bounded will do? To be investigated.

Parse hex constants

Describe the bug
TimEL is not able to parse hexadecimal constants, like 0x1.

Actual behavior

java.lang.NumberFormatException: For input string: "0x1"
	at java.base/java.lang.NumberFormatException.forInputString(Unknown Source)
	at java.base/java.lang.Integer.parseInt(Unknown Source)
	at java.base/java.lang.Integer.parseInt(Unknown Source)
	at net.vleo.timel.impl.parser.ParserTreeAdapter.visitTerminal(ParserTreeAdapter.java:255)
	at net.vleo.timel.impl.parser.ParserTreeAdapter.visitTerminal(ParserTreeAdapter.java:47)

Expected behavior
0x1 (as well 0X1) is interpreted correctly.

Runtime
Any

Additional context
/

Support long constants (IntegerSuffix)

Is your feature request related to a problem? Please describe.
TimEL should support long constants via the L suffix

Describe the solution you'd like
Add IntegerSuffix to the grammar, so that maybe one day one can use it also for other data types (bigdecimals?):

fragment
IntegerConstant
    :   DecimalConstant IntegerSuffix?
    |   OctalConstant IntegerSuffix?
    |   HexadecimalConstant IntegerSuffix?
    ;

fragment
IntegerSuffix
    :   LongSuffix
    ;

fragment
LongSuffix
    :   [lL]
    ;

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Calling a function with non matching signature results in NPE

Describe the bug
Compile version(1) or any other invalid function call that doesn't match a valid signature

Actual behavior

Aug 28 20:54:57 timel-demo java[23887]: java.lang.NullPointerException: null
Aug 28 20:54:57 timel-demo java[23887]: at net.vleo.timel.function.FunctionRegistry.lookup(FunctionRegistry.java:145) ~[timel-core-0.9.3-SNAPSHOT.jar!/:na]
Aug 28 20:54:57 timel-demo java[23887]: at net.vleo.timel.impl.intermediate.SyntaxTreeAdapter.visit(SyntaxTreeAdapter.java:97) ~[timel-core-0.9.3-SNAPSHOT.jar!/:na]
Aug 28 20:54:57 timel-demo java[23887]: at net.vleo.timel.impl.intermediate.SyntaxTreeAdapter.visit(SyntaxTreeAdapter.java:49) ~[timel-core-0.9.3-SNAPSHOT.jar!/:na]
Aug 28 20:54:57 timel-demo java[23887]: at net.vleo.timel.impl.parser.tree.FunctionCall.accept(FunctionCall.java:48) ~[timel-core-0.9.3-SNAPSHOT.jar!/:na]
Aug 28 20:54:57 timel-demo java[23887]: at net.vleo.timel.impl.intermediate.SyntaxTreeAdapter.lambda$visit$1(SyntaxTreeAdapter.java:119) ~[timel-core-0.9.3-SNAPSHOT.jar!/:na]
Aug 28 20:54:57 timel-demo java[23887]: at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[na:1.8.0_151]
Aug 28 20:54:57 timel-demo java[23887]: at java.util.LinkedList$LLSpliterator.forEachRemaining(LinkedList.java:1235) ~[na:1.8.0_151]

Expected behavior
ParseException should be thrown instead, including the suggested signatures.

Runtime

  • OS: Any
  • JVM version: Any
  • TimEL version: 0.9.3-SNAPSHOT (14e3ae9574c5d66e73f6e51554349219b7761c34)

Additional context
/

Accept a semicolon to close the last statement as well

Currently the last statement should not be semicolon terminated, which is extremely annoying for the users.

Change the behavior so that the last expression can be compiled with a closing semicolon without affecting the semantic

Implement `EvaluationException` instead of unchecked exceptions

Describe the bug
Many Functions throw RuntimeException when an evaluation exception happens. These should be migrated to a specific checked exception EvaluationException.

Actual behavior
Evaluation of every(1, "x", "x") throws java.lang.RuntimeException: Invalid field specified: x

Expected behavior
Evaluation of every(1, "x", "x") throws EvaluationException.

Runtime
Any

Additional context
/

Support octal integer constants

Is your feature request related to a problem? Please describe.
For the sake of compability with the rest of the world, TimEL should be able to understand octal constants.

Describe the solution you'd like
Add octal support:

fragment
IntegerConstant
    :   OctalConstant

fragment
OctalConstant
    :   '0' OctalDigit*
    ;

Describe alternatives you've considered
/

Additional context
/

Support string escape sequences

Is your feature request related to a problem? Please describe.
Currently one can't escape characters in a string.

Describe the solution you'd like
The grammar and the parser should be evolved to support

fragment
EscapeSequence
    :   SimpleEscapeSequence // Standard C-like escapes
    |   OctalEscapeSequence
    |   HexadecimalEscapeSequence
    |   UniversalCharacterName

Describe alternatives you've considered
/

Additional context
/

Add unary operator+

Describe the bug
TimEL doesn't support unary operator+ so far.

Actual behavior
Evaluating +1 causes a Parse error at line 1:0: Failed to lookup function +

Expected behavior
Evaluating +1 returns 1.

Runtime
/
Additional context
/

Unary operator-

Describe the bug
TimEL lacks unary operator- implementation

Actual behavior
Parsing "-1" leads to:
Parse error: net.vleo.timel.ParseException: Cannot resolve function -(Integer)

Expected behavior
"-1" is parsed correctly

Runtime
/

Additional context
/

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.