Giter Club home page Giter Club logo

seddryck / expressif Goto Github PK

View Code? Open in Web Editor NEW
1.0 2.0 0.0 22.74 MB

Expressif is the variable substitution syntax combined with a library of predicates and functions. Initially designed for NBi.io, Expressif allows you to define variables, transformations of these variables (with expressions and functions), in plain text, which can then be interpreted by the engine. Same can be done with predicates.

License: Apache License 2.0

C# 96.37% PowerShell 3.52% Batchfile 0.06% Shell 0.04%
predicate predicate-functions variable-binding variable-substitution

expressif's Introduction

Expressif

Expressif is the variable substitution syntax, initially designed for NBi.io.

Expressif allows you to define variables and transformation of these variables (functions), in plain text, which can then be interpreted by the engine. The syntax for the definition of the expression transforming the variable is similar to:

@myVariable | text-to-lower | text-to-pad-right(@myCount, *)

Logo

About | Quickstart | Installing | Functions and predicates

About

Social media: website twitter badge

Releases: nuget licence badge

Dev. activity: GitHub last commit Still maintained GitHub commit activity

Continuous integration builds: Build status Tests CodeFactor codecov FOSSA Status

Status: stars badge Bugs badge Features badge Top language

Quickstart

Expression

Expressif provides a class named Expression to define a chain of functions applied to a value. The class is expecting the textual representation of the chained functions in its constructor.

var expression = new Expression("lower");
var result = expression.Evaluate("Nikola Tesla");
Assert.That(result, Is.EqualTo("nikola tesla"));

Some functions require parameters, you can specify them between the parenthesis following the function name. Note that literal textual values don't required quotes surronding them.

var expression = new Expression("remove-chars(a)");
var result = expression.Evaluate("Nikola Tesla");
Assert.That(result, Is.EqualTo("Nikol Tesl"));

You can chain the functions with the operator pipe (|). The functions are executed from left to right.

var expression = new Expression("lower | remove-chars(a)");
var result = expression.Evaluate("Nikola Tesla");
Assert.That(result, Is.EqualTo("nikol tesl"));

It's possible to use variables as function parameters. The name of the variables must always start by an arobas (@)

var context = new Context();
context.Variables.Add<char>("myChar", 'k');

var expression = new Expression("lower | remove-chars(@myChar)", context);
var result = expression.Evaluate("Nikola Tesla");
Assert.That(result, Is.EqualTo("niola tesla"));

In addition to the variables that must be scalar values (text, numeric, dateTime ...), you can also add a property-object to the context. A property-object must be a pure C# object, an IDictionnary, an IList, or a DataRow. You can access the properties of the property-object based on the property's name with the syntax [property-name].

var context = new Context();
context.CurrentObject.Set(new { CharToBeRemoved = 't' });

var expression = new Expression("lower | remove-chars([CharToBeRemoved])", context);
var result = expression.Evaluate("Nikola Tesla");
Assert.That(result, Is.EqualTo("nikola esla"));

or based on its position with the syntax #index (where index is positive number).

var context = new Context();
context.CurrentObject.Set(new List<char>() { 'e', 's' });

var expression = new Expression("lower | remove-chars(#1)", context);
var result = expression.Evaluate("Nikola Tesla");
Assert.That(result, Is.EqualTo("nikola tela"));

It's also possible to use the result of function as the value of a parameter for another function. To achieve this the function as a parameter must be surrounded by curly braces {...}.

var context = new Context();
context.Variables.Add<int>("myVar", 6);
context.CurrentObject.Set(new List<int>() { 15, 8, 3 });

var expression = new Expression("lower | skip-last-chars( {@myVar | subtract(#2) })", context);
var result = expression.Evaluate("Nikola Tesla");
Assert.That(result, Is.EqualTo("nikola te"));

Predication

Expressif provides a class named Predication to define a combination of predicates applied to an argument. The class is expecting the textual representation of the predicates in its constructor.

var predication = new Predication("lower-case");
var result = predication.Evaluate("Nikola Tesla");
Assert.That(result, Is.False);

Same than for expressions, some predicates require parameters, you can specify them between the parenthesis immediately following the predicate name. More specifically, some predicates require an interval as parameter. The parameter can be define with the help of square brackets or parenthesis.

var predication = new Predication("within-interval([0;20[)");
var result = predication.Evaluate(15);
Assert.That(result, Is.True);

Other predicates require a culture as parameter. To specify a culture just use the textual representation of the culture composed of the two letter ISO code of the language then the two letters ISO code of the country separated by a dash i.e. fr-be for Belgian French, nl-be for Belgian Dutch or de-de for German.

var predication = new Predication("matches-date(fr-fr)");
var result = predication.Evaluate("28/12/1978");
Assert.That(result, Is.True);

Any predicate can be negated to return the opposite result. To negate a predicate just put the exclamation mark (!) in front of the predicate name.

var predication = new Predication("!starts-with(Nik)");
var result = predication.Evaluate("Nikola Tesla");
Assert.That(result, Is.False);

You can combine the predicates. Each predicate will accept the same argument and will be evaluated separatly. The results of the combination is dependening on the combinational operator used. To specify the name of the combinational operator use the pipe operator (|) immediately followed by the name of the operator. The following operators are valid |AND, |OR, |XOR.

Take into account that when possible, the operators are implementing a short-circuit. If the two predicates are combined with the operator |AND and the first is returning false, the second will not be evaluated. Following the same reasoning, if the two predicates are combined with the operator |OR and the first is returning true, the second will also be ignored.

var predication = new Predication("starts-with(Nik) |AND ends-with(sla)");
var result = predication.Evaluate("Nikola Tesla");
Assert.That(result, Is.True);

By default, the predicates are combined from left to right. If you've three predicates, the two firsts will be combined and then the result of this combination will be combined with the third predicate. To alter this order, you must group the predicates with the help of curly braces {...}. Each predicate inside a group is evaluated from left to right and then the result of the group is combined with another group or predicate also from left to right.

var predication = new Predication("{starts-with(Nik) |AND ends-with(sla)} |OR {starts-with(ola) |AND ends-with(Tes)}");
var result = predication.Evaluate("Nikola Tesla");
Assert.That(result, Is.True);

var withoutGroupsPredication = new Predication("starts-with(Nik) |AND ends-with(sla) |OR starts-with(ola) |AND ends-with(Tes)");
var secondResult = withoutGroupsPredication.Evaluate("Nikola Tesla");
Assert.That(result, Is.Not.EqualTo(secondResult));

Installing

Install in the usual .NET fashion:

Install-Package Expressif

Supported functions and predicates

Functions

Scope Name Aliases
IO creation-datetime file-to-creation-datetime, file-to-creation-dateTime
IO creation-datetime-utc file-to-creation-datetime-utc, file-to-creation-dateTime-utc
IO directory path-to-directory
IO extension path-to-extension
IO filename path-to-filename
IO filename-without-extension path-to-filename-without-extension
IO root path-to-root
IO size file-to-size
IO update-datetime file-to-update-datetime, file-to-update-dateTime
IO update-datetime-utc file-to-update-datetime-utc, file-to-update-dateTime-utc
Numeric absolute numeric-to-absolute
Numeric add numeric-to-add
Numeric ceiling numeric-to-ceiling
Numeric clip numeric-to-clip
Numeric cube-power numeric-to-cube-power
Numeric cube-root numeric-to-cube-root
Numeric decrement numeric-to-decrement
Numeric divide numeric-to-divide
Numeric floor numeric-to-floor
Numeric increment numeric-to-increment
Numeric integer numeric-to-integer
Numeric invert numeric-to-invert
Numeric multiply numeric-to-multiply
Numeric nth-root numeric-to-nth-root
Numeric null-to-zero
Numeric oppose numeric-to-oppose
Numeric power numeric-to-power
Numeric round numeric-to-round
Numeric sign numeric-to-sign
Numeric square-power numeric-to-square-power
Numeric square-root numeric-to-square-root
Numeric subtract numeric-to-subtract
Special any-to-any
Special neutral Special-to-neutral
Special null-to-value
Special value-to-value
Temporal age date-to-age
Temporal backward dateTime-to-backward, dateTime-to-subtract
Temporal ceiling-hour dateTime-to-ceiling-hour
Temporal ceiling-minute dateTime-to-ceiling-minute
Temporal change-of-hour dateTime-to-change-of-hour
Temporal change-of-minute dateTime-to-change-of-minute
Temporal change-of-month dateTime-to-change-of-month
Temporal change-of-second dateTime-to-change-of-second
Temporal change-of-year dateTime-to-change-of-year
Temporal clamp dateTime-to-clamp, dateTime-to-clip
Temporal datetime-to-date dateTime-to-date
Temporal day-of-month dateTime-to-day-of-month
Temporal day-of-week dateTime-to-day-of-week
Temporal day-of-year dateTime-to-day-of-year
Temporal first-in-month dateTime-to-first-in-month
Temporal first-of-month dateTime-to-first-of-month
Temporal first-of-year dateTime-to-first-of-year
Temporal floor-hour dateTime-to-floor-hour
Temporal floor-minute dateTime-to-floor-minute
Temporal forward dateTime-to-forward, dateTime-to-add
Temporal hour dateTime-to-hour
Temporal hour-minute dateTime-to-hour-minute
Temporal hour-minute-second dateTime-to-hour-minute-second
Temporal hour-of-day dateTime-to-hour-of-day
Temporal invalid-to-date
Temporal iso-day-of-year dateTime-to-iso-day-of-year
Temporal iso-week-of-year dateTime-to-iso-week-of-year
Temporal iso-year-day dateTime-to-iso-year-day
Temporal iso-year-week dateTime-to-iso-year-week
Temporal iso-year-week-day dateTime-to-iso-year-week-day
Temporal last-in-month dateTime-to-last-in-month
Temporal last-of-month dateTime-to-last-of-month
Temporal last-of-year dateTime-to-last-of-year
Temporal length-of-month dateTime-to-length-of-month
Temporal length-of-year dateTime-to-length-of-year
Temporal local-to-utc
Temporal minute-of-day dateTime-to-minute-of-day
Temporal minute-of-hour dateTime-to-minute-of-hour
Temporal month dateTime-to-month
Temporal month-day dateTime-to-month-day
Temporal month-of-year dateTime-to-month-of-year
Temporal next-business-days temporal-to-next-business-days, next-business-day, add-business-days
Temporal next-day dateTime-to-next-day
Temporal next-month dateTime-to-next-month
Temporal next-weekday dateTime-to-next-weekday
Temporal next-weekday-or-same dateTime-to-next-weekday-or-same
Temporal next-year dateTime-to-next-year
Temporal null-to-date
Temporal previous-business-days temporal-to-previous-business-days, previous-business-day, subtract-business-days
Temporal previous-day dateTime-to-previous-day
Temporal previous-month dateTime-to-previous-month
Temporal previous-weekday dateTime-to-previous-weekday
Temporal previous-weekday-or-same dateTime-to-previous-weekday-or-same
Temporal previous-year dateTime-to-previous-year
Temporal second-of-day dateTime-to-second-of-day
Temporal second-of-hour dateTime-to-second-of-hour
Temporal second-of-minute dateTime-to-second-of-minute
Temporal set-time dateTime-to-set-time
Temporal set-to-local
Temporal set-to-utc
Temporal utc-to-local
Temporal year dateTime-to-year
Temporal year-of-era dateTime-to-year-of-era
Text after-substring text-to-after-substring
Text append text-to-append
Text append-new-line text-to-append-new-line
Text append-space text-to-append-space
Text before-substring text-to-before-substring
Text clean-whitespace text-to-clean-whitespace
Text collapse-whitespace text-to-collapse-whitespace
Text count-distinct-chars text-to-count-distinct-chars
Text count-substring text-to-count-substring
Text empty-to-null
Text filter-chars text-to-filter-chars
Text first-chars text-to-first-chars
Text html-to-text
Text last-chars text-to-last-chars
Text length text-to-length, count-chars
Text lower text-to-lower
Text mask-to-text
Text null-to-empty
Text pad-center text-to-pad-center
Text pad-left text-to-pad-left
Text pad-right text-to-pad-right
Text prefix text-to-prefix
Text prefix-new-line text-to-prefix-new-line
Text prefix-space text-to-prefix-space
Text prepend text-to-prepend
Text prepend-new-line text-to-prepend-new-line
Text prepend-space text-to-prepend-space
Text remove-chars text-to-remove-chars
Text replace-chars text-to-replace-chars
Text replace-slice text-to-replace-slice
Text retain-alpha text-to-retain-alpha
Text retain-alpha-numeric text-to-retain-alpha-numeric
Text retain-numeric text-to-retain-numeric
Text retain-numeric-symbol text-to-retain-numeric-symbol
Text skip-first-chars text-to-skip-first-chars
Text skip-last-chars text-to-skip-last-chars
Text suffix text-to-suffix
Text suffix-new-line text-to-suffix-new-line
Text suffix-space text-to-suffix-space
Text text-to-datetime text-to-dateTime
Text text-to-html
Text text-to-mask
Text token text-to-token
Text token-count text-to-token-count
Text trim text-to-trim
Text upper text-to-upper
Text whitespaces-to-empty blank-to-empty
Text whitespaces-to-null blank-to-null
Text without-diacritics text-to-without-diacritics
Text without-whitespaces text-to-without-whitespaces

Predicates

Scope Name Aliases
Boolean false boolean-is-false
Boolean false-or-null boolean-is-false-or-null
Boolean identical-to boolean-is-identical-to
Boolean true boolean-is-true
Boolean true-or-null boolean-is-true-or-null
Numeric equal-to numeric-is-equal-to
Numeric even numeric-is-even
Numeric greater-than numeric-is-greater-than
Numeric greater-than-or-equal numeric-is-greater-than-or-equal
Numeric integer numeric-is-integer
Numeric less-than numeric-is-less-than
Numeric less-than-or-equal numeric-is-less-than-or-equal
Numeric modulo numeric-is-modulo
Numeric negative numeric-is-negative
Numeric negative-or-zero numeric-is-negative-or-zero
Numeric odd numeric-is-odd
Numeric one numeric-is-one
Numeric opposite numeric-is-opposite
Numeric positive numeric-is-positive
Numeric positive-or-zero numeric-is-positive-or-zero
Numeric within-interval numeric-is-within-interval
Numeric zero numeric-is-zero
Numeric zero-or-null numeric-is-zero-or-null
Special null is-null
Temporal after dateTime-is-after
Temporal after-or-same-instant dateTime-is-after-or-same-instant
Temporal before dateTime-is-before
Temporal before-or-same-instant dateTime-is-before-or-same-instant
Temporal business-day dateTime-is-business-day
Temporal contained-in dateTime-is-contained-in
Temporal in-the-future dateTime-is-in-the-future
Temporal in-the-future-or-now dateTime-is-in-the-future-or-now
Temporal in-the-future-or-today dateTime-is-in-the-future-or-today
Temporal in-the-past dateTime-is-in-the-past
Temporal in-the-past-or-now dateTime-is-in-the-past-or-now
Temporal in-the-past-or-today dateTime-is-in-the-past-or-today
Temporal leap-year dateTime-is-leap-year
Temporal on-the-day dateTime-is-on-the-day
Temporal on-the-hour dateTime-is-on-the-hour
Temporal on-the-minute dateTime-is-on-the-minute
Temporal same-instant dateTime-is-same-instant
Temporal today dateTime-is-today
Temporal tomorrow dateTime-is-tomorrow
Temporal weekday dateTime-is-weekday
Temporal weekend dateTime-is-weekend
Temporal within-current-month dateTime-is-within-current-month
Temporal within-current-week dateTime-is-within-current-week
Temporal within-current-year dateTime-is-within-current-year
Temporal within-last-month dateTime-is-within-last-month
Temporal within-last-week dateTime-is-within-last-week
Temporal within-last-year dateTime-is-within-last-year
Temporal within-next-days dateTime-is-within-next-days
Temporal within-previous-days dateTime-is-within-previous-days
Temporal within-upcoming-month dateTime-is-within-upcoming-month
Temporal within-upcoming-week dateTime-is-within-upcoming-week
Temporal within-upcoming-year dateTime-is-within-upcoming-year
Temporal yesterday dateTime-is-yesterday
Text any-of text-is-any-of
Text contains text-contains
Text empty text-is-empty
Text empty-or-null text-is-empty-or-null
Text ends-with text-ends-with
Text equivalent-to text-is-equivalent-to
Text lower-case text-is-lower-case
Text matches-date text-matches-date
Text matches-datetime text-matches-datetime
Text matches-numeric text-matches-numeric
Text matches-regex text-matches-regex
Text matches-time text-matches-time
Text sorted-after text-is-sorted-after
Text sorted-after-or-equivalent-to text-is-sorted-after-or-equivalent-to
Text sorted-before text-is-sorted-before
Text sorted-before-or-equivalent-to text-is-sorted-before-or-equivalent-to
Text starts-with text-starts-with
Text upper-case text-is-upper-case

expressif's People

Contributors

dependabot[bot] avatar github-actions[bot] avatar seddryck avatar

Stargazers

 avatar

Watchers

 avatar  avatar

expressif's Issues

Parse an interval of numeric or dateTime

An interval is defined by two values and two open-closed bounds.

Should support the European notation with a semi-column as the separator of the two values

[a;b]
[a;b[
]a;b[
]a;b]

Combination of predicates

Predicates can be chained but they require a combination operator to stipulate how they should be chained. The following three logical operators can be used: AND, OR, XOR. Regarding the Expressif notation the logical operators must be preceded by a pipe |.

123 |? !equal-to(125) |OR even 

If you want to group some predicates and apply some priorities to these predicates, you can use curly braces to group them.

123 |? {!equal-to(125) |AND even} |OR {greater-than-or-equal-to(450) |AND odd}

Additional dateTime functions to extract a portion of the actual dateTime

It should be possible to extract a portion (Year, Month, Day, Hour, Minute, Second) of a date/time value. It's returning a numeric value.

  • dateTime-to-extract-year
  • dateTime-to-extract-month
  • dateTime-to-extract-day
  • dateTime-to-extract-hour
  • dateTime-to-extract-minute
  • dateTime-to-extract-second

Text function to return the substring after or before a given character or string

text-to-after returns text that occurs after given character or string passed as parameter. If the given char/string is not found then the function returns null. If the given char/string are the first (before) or last (after) chars then the function returns empty

"Joe, it's me!" | text-to-after(", ")

returns it's me!

"Joe, it's me!" | text-to-before(", ")

returns Joe

Color function translating a color name to Hex

color-to-hex returns an Hex notation, obtained by combining color values for red, green, and blue from a color name. If the color name is unknown returns null

string ToHex(this Color c) => $"#{c.R:X2}{c.G:X2}{c.B:X2}";

Text function to substitute a char by another

  • When you have just one specific character or word to replace, you can use replace-char function. For example, the sentence โ€œcake it easyโ€ here is misspelled, and I can correct it easily to โ€œtake it easyโ€ instead of โ€œcake it easyโ€ using replace-char(c, t) function.

Replicate text predicates supported by nbi.io

Following text predicates are currently supported by nbi.io

  • equivalent-to (case-sensitive)
  • sorted-after (case-sensitive)
  • sorted-after-or-equivalent (case-sensitive)
  • sorted-before (case-sensitive)
  • sorted-before-or-equivalent (case-sensitive)
  • null
  • empty
  • null-or-empty
  • starts-with (case-sensitive)
  • ends-with (case-sensitive)
  • contains (case-sensitive)
  • lower-case
  • upper-case
  • matches-regex (case-sensitive)
  • matches-numeric (culture)
  • matches-dateTime (culture)
  • matches-date (culture)
  • matches-time (culture)
  • any-of

Document Quickstart for functions and predicates based on tests

Use content of files PredicationTest.cs and ExpressionTest.cs to respectively build the documentation of the quick-start for predication and expression.

  • Based on the File/TestName information provided extract the few lines of code and include them in the documentation file.
  • Include the content of these files in the ReadMe file

Text function returning a specific amount of left/right tokens

The function text-to-tokens-left returns the text containing the specified number of words in the input text, counting from the left (or right). The parameter is specifying how many words are returned, if the initial value contains fewer words than expected to be returned then all words are returned. The delimitation between words is specified by an optional second parameter when the delimiter is not specified then any whitespace is the default words' separator.

Document existing predicates and functions

  • Document text functions
  • Document numeric functions
  • Document temporal functions
  • Document special functions
  • Document IO functions
  • Document text predicates
  • Document numeric predicates
  • Document temporal predicates
  • Document special predicates
  • Document boolean predicates

Numeric function to transform a numeric value to a bucket

The numeric-to-bucket functions expect a first parameter specifying the minimal value (included) of the first bucket, the second parameter being the maximal value (excluded) of the last bucket and the third parameter is how many equally spaced buckets are defined between these two values. The function returns in which bucket the initial value is positioned.

12500 | numeric-to-bucket(5000, 20000, 3)

The range [5000;20000[ is divided into three buckets [5000;10000[ , [10000;150000[ , [15000;20000[ and 12500 is situated into the second range, so the function returns 2.

If the input value is outside the range defined by the two first parameters, the function returns null
If the amount of bucket is zero or less, the function returns null

A 4th parameter optional set to true (defaulted to false) is creating a bucket 0 for all values less than the first parameter and a bucket n+1 for all values equal or greater than the input value.

22500 | numeric-to-bucket(5000, 20000, 3, true)

returns 4

2500 | numeric-to-bucket(5000, 20000, 3, true)

returns 0

Implement functions for special values manipulations

Following functions are already supported in nbi.io and should be migrated to the current solution

  • null-to-value: if the current content of the cell is null replace the content by (value)
  • any-to-any: replaces the content of each cell by (any)
  • value-to-value: if the cell's value is not null will replace the content by (value)

Combination of predicates with multiple inputs

Expressif should support the following predication having more than a single argument (multivariate). In this basic version, the argument of each predication should be a IParameter.

{123 |? null |OR {is-even |AND is-greater-than(100) }} |AND {@foo |? is-after(@bar)} |AND {[bar] |? starts-with(K)}
  • Parser
  • Factory

Parse another function as a parameter of a function

It should be possible to parse another function as a function's parameter. The syntax of a sub-function is a function definition surrounded by curly braces {}

text-to-function ({ text-to-other-function(param1, param2) })

Serialize an expression from an ExpressionBuilder

The builder should also support to serialize an Expressif expression:

var builder = new ExpressionBuilder()
                         .Chain<Lower>()
                         .Chain<FirstChars>(5)
                         .Chain<PadRight>(7, '*');
var expression = builder.ToString();
Assert.That(expression, Is.EqualTo("lower | first-chars(5) | pad-right(7,*)"));

Additional text functions for capitalization

The list of native functions should also support the following functions for text values, playing on the capitalization of letters:

  • text-to-swapcase: Transform the string's lowercase characters to uppercase and uppercase characters to lowercase.
  • text-to-capitalize: Capitalize the first character of the input string.
  • text-to-title: Converts the input string into titlecase. Capitalize the first character of each word in the input string except for articles (a, an, the). The capitalization rules are defined according to a style.

Numeric function transforming the value into a currency representation

The numeric-to-currency-before function converts a number to text using currency format, with the decimals rounded to the number of places you specify. It uses the $#,##0.00 number format, although the currency symbol that is applied depends on the parameter specified. The decimal parameter is optional and defaults to 2.

123.4567 | numeric-to-currency-before("$")

returns $123.46

57123.4567 | numeric-to-currency-after("โ‚ฌ", 0)

returns 57,123โ‚ฌ

By default the decimal separator is apoint . and the thousand separator is a coma , but tou can also specify them as parameters.

57123.4567 | numeric-to-currency-after("EUR", 2, ",", ".")

returns 57.123,46EUR

In case of negative value the default behaviour is to precede the string representation by a negative sign. A last parameter can also be defined with a single char representing the negative symbol or with two parameters representing symbol to put at the beginning or end of the string if the value is negative.

-57123.4567 | numeric-to-currency-after("EUR", 2, ",", ".", "-")

returns -57.123,46EUR

-57123.4567 | numeric-to-currency-after("EUR", 2, ",", ".", "()")

returns (57.123,46EUR)

Negation of predicates

Users should be able to select the negation of a predicate. The negation of a predicate is noted with an interrogation mark ! in front of the predicate.

123 |? !equal-to(125)
123 |? !{ equal-to(125) }

Builder should support instance of a function

The builder should support to not receive a type but an instance

var builder = new ExpressionBuilder()
                               .As(new TextToLower())
                               .Chain(new TextToFirstChars(new LiteralScalarResover<int>(5)))
                               .Chain(new TextToPadRight(
                                        new LiteralScalarResover<int>(7)
                                        , new LiteralScalarResover<char>('*')
                               );
var expression = builder.ToString();
Assert.That(expression, Is.EqualTo("text-to-lower | text-to-first-chars(5) | text-to-pad-right(7,*)"));

Map a function's name and its parameters in a plain text description to the corresponding constructor of a function type

  • When parsing text-to-function(param1), it should be able to instantiate an instance of the type TextToFunction using a constructor with a single parameter and the value param1.

  • Function type could have multiple constructors with different counts of parameters, it's needed to match the constructor and the count of parameters submitted in the plain text.

  • Parameters of the function should correctly been evaluated (at runtime).

Parse predicates

A predicate is a special case of a function returning a boolean. Predicates can also support parameters with the same type as functions (Literal, Variable, ObjectProperty, ObjectIndex, and InputExpression). Predicates are single-member expressions, so the pipe operator to chain predicates is not supported. Another kind of combination of predicates will be discussed later.

The parsing of predicates names should support the usage of the contracted version

equal-to(125)

and the fully qualified version (including the namespace and the "is" keyword).

numeric-is-equal-to(125)
  • Predication

Within an input expression; you must stipulate that your expression is a predicate with the help of the operator |?

123 |? equal-to(125)
  • InputPredication

Implement functions for text manipulations

  • blank-to-empty: if the current text is blank (zero or many spaces) replace the content by empty
  • blank-to-null: if the current text is blank (zero or many spaces) replace the content by null
  • empty-to-null: if the current text is empty (length=0) replace the content by null
  • null-to-empty: if the current content of the cell is null replace the content by (empty)
  • text-to-without-diacritics: if the current text contains any accents or diacritics, they are removed
  • text-to-without-whitespaces: removes blanks from anywhere within the text. If the text is null, it returns null but if empty or blank, it returns empty.
  • text-to-remove-chars(char): removes the defined char from anywhere within the text. If the text is null, it returns null and if empty, it returns empty. If the original value is blank and the character to remove is not a whitespace it returns blank else empty.
  • text-to-upper: returns a copy of this text converted to uppercase
  • text-to-lower: returns a copy of this text converted to lowercase
  • html-to-text: decodes the html to text
  • text-to-html: encodes the text to html
  • text-to-trim: removes blanks from the beginning and end of the text.
  • text-to-length: returns the length of the text value of the text. If the text is null or empty, it returns 0.
  • text-to-token-count and text-to-token-count(char): returns the count of tokens in the original text. If no sperator is specified (char), it uses the whitespace tokenizer that is considered as one or more letter or digit or hyphen seperated by one or more whitespace. If a separator is specified, a token is identified as a serie of any character separated by one or more instance of the separator. If the current value is null or empty or blank, it returns 0.
  • text-to-token(index) and text-to-token(index, char): returns the token at the position of the index. The first token as an index of 0. This function uses the same tokenizer than defined in text-to-token-count. If the requested token doesn't exist (index greater or equal to the tokens' count), it returns null. The initial value null and empty returns null for any separator and any index. In general, the value blank returns blank for the first token except if the separator is a whitespace.
  • text-to-prefix(string): Append the value of string before the current value. If the current value is null, the result will be null.
  • text-to-suffix(string): Append the value of string after the current value. If the current value is null, the result will be null.
  • text-to-first-chars(length): if the text is longer than the specified length, take the first characters.
  • text-to-last-chars(length): if the text is longer than the specified length, take the last characters.
  • text-to-skip-first-chars(length): returns the text except the length first characters. If the text's length is less than the specified length returns an empty string.
  • text-to-skip-last-chars(length): returns the text except the length last characters. If the text's length is less than the specified length returns an empty string.
  • text-to-pad-left(length, character): if the text is shorter than the specified length, add the specified character at the beginning of the text until the length of this text is equal to the expected length.
  • text-to-pad-right(length, character): if the text is shorter than the specified length, add the specified character at the end of the text until the length of this text is equal to the expected length.
  • text-to-dateTime(format) and text-to-dateTime(format, culture) returns a dateTime from the text value after parsing it with the format provided as argument. If the format includes day or month names, it could be useful to specify the culture.
  • text-to-mask(format) returns a formated text based on a mask. The mask is a text where the character '' will be replaced by a glyph from the original text, other characters of the mask are unreplaced. As an example the text 12345678 with a mask BE-***.***.*** will return the textual value BE-123.456.78. If the mask is expecting less charachters than the original value then the remaining characters are dropped. If the mask is expecting more components than the original value the last '' characters won't be not replaced.
  • mask-to-text(format) returns a unformated text extracted from a text on which a mask was previously applied. The mask is a text where the character '*' has been replaced by a glyph from the original text, other characters of the mask are not substitued. As an example the masked text BE-123.456.78 with a mask BE-***.***.*** will return the textual value 12345678. In case this transformation can't happen because the masked value and the mask don't match, the retruned result is (null).

Support for a special value named Whitespace

It should be possible to define a special value Whitespace that is equal to any string containing only whitespace chars. A string that is empty is also a whitespace string. The literal representation of this value is (whitespace).

Builder should support instance of a predicate

The builder should support to not receive a type but an instance of a predicate

var builder = new PredicationBuilder()
                               .Chain(new LowerCase())
                               .Chain(new AndOperator(), new StartsWith(new LiteralScalarResover<string>("Nik")));
var predicatation = builder.Build();
Assert.That(predicatation.Evaluate("Nikola Tesla"), Is.False);

Support for special value named Empty

It should be possible to define a special value Empty that is equal to an empty string (a string of 0 chars length) and the literal representation (empty)

Special values for temporal functions

When using Expressif, especially with temporal data, it would be convenient to get access to a few special values.

  • (now) representing the current date/time
  • (today) representing the current date at midnight
  • (yesterday) representing the date of the previous day at midnight
  • (tomorrow) representing the date of the next day at midnight

Alternative operator to support null (or missing) property/index

This feature is useful when you've some transformations but you are not sure that some of the columns used in the transformation always exist or you fear that the value could be (null)

The syntax [foo] ??= [bar] ??= default | text-to-upper will uppercase the value of the column named foo except if this column doesn't exist or is null where the column bar will be used, again if this column is missing or null then the value default will be uppercased.

Implement functions for numeric manipulations

The following functions are already supported in nbi.io and should be migrated to the current solution

  • null-to-zero: if the cell is null or empty or blank, it replaces the content by 0.
  • numeric-to-floor: returns the largest integral value less than or equal to the specified number. If the cell is null or empty or blank, it returns null.
  • numeric-to-ceiling: returns the smallest integral value greater than or equal to the specified number. If the cell is null or empty or blank, it returns null.
  • numeric-to-integer: rounds a value to the nearest integer. If the cell is null or empty or blank, it returns null.
  • numeric-to-round(integer): rounds a value to the specified number of fractional digits.
  • numeric-to-clip(numeric, numeric): Clip a value such as if smaller than the first argument then it will return the first argument or if larger than the second argument then will return the second argument. If the original value is between the first and second argument then the original value is returned.
  • numeric-to-increment: add 1 to the current value
  • numeric-to-decrement: subtract 1 to the current value
  • numeric-to-add(incr): add incr to the current value
  • numeric-to-add(incr, times): add incr to the current value several times. If times is zero, return current value. times must be an integer value.
  • numeric-to-subtract(decr): subtract decr to the current value
  • numeric-to-subtract(decr, times): subtract decr to the current value several times. If times is zero, return current value. times must be an integer value.
  • numeric-to-multiply(factor): multiply the current value by factor
  • numeric-to-divide(factor): divide the current value by factor
  • numeric-to-invert: invert the current value (equivalent to 1/current value)

Support aliases for functions

It should be possible to use the aliases of a function in a script

e.g

"FOO bar" | lower

Should be equivalent to the canonical form

"FOO bar" | text-to-lower

Create a context class to specify additional information that can be used in expression

When calling an expression you can define the value to be transformed by the expression in the parameter of the function Evaluate. Often, it's useful to have another bunch of values that can be used in parameters of the expression. To achieve this, Expressif support variables and the CurrentObject.

  • Variables can be defined with the method context.Variables.Add("variableName", object)
  • CurrentObject can be defined with the method context.CurrentObject.Set(object)

Implement functions for temporal manipulations

Following functions are already supported in nbi.io and should be migrated to the current solution

  • date-to-age: returns the age according to the input value value of the cell at the moment of execution of the test.
  • dateTime-to-date: remove information about the time (equivalent to set the input value to midnight)
  • dateTime-to-first-of-month: returns the first day of the month where the input value lies in.
  • dateTime-to-first-of-year: returns the first day of the year where the input value lies in.
  • dateTime-to-last-of-month: returns the last day of the month where the input value lies in.
  • dateTime-to-last-of-year: returns the last day of the year where the input value lies in.
  • dateTime-to-next-day: returns the next day, at the same time.
  • dateTime-to-previous-day: returns the previous day, at the same time.
  • dateTime-to-next-month: returns a dateTime corresponding to one month after the given date.
  • dateTime-to-previous-month: returns a dateTime corresponding to one month before the given date.
  • dateTime-to-next-year: returns a dateTime corresponding to one year after the given date.
  • dateTime-to-previous-year: returns a dateTime corresponding to one year before the given date.
  • dateTime-to-floor-hour: returns a dateTime rounded down to the nearest boundary of an hour.
  • dateTime-to-ceiling-hour: returns a dateTime rounded up to the nearest boundary of an hour.
  • dateTime-to-floor-minute: returns a dateTime rounded down to the nearest boundary of a minute.
  • dateTime-to-ceiling-minute: returns a dateTime rounded up to the nearest boundary of a minute.
  • dateTime-to-clip(dateTime, dateTime): Clip a value such as if smaller than the first argument then it will return the first argument or if larger than the second argument then will return the second argument. If the original value is between the first and second argument then the original value is returned.
  • dateTime-to-set-time(timeSpan): Set the hours, minutes, second of a dateTime to the specified value without changing the date part. The timespan should be defined with the format hh:mm:ss such as 07:00:00.
  • dateTime-to-add(ts): add ts (a timeSpan) to the current value
  • dateTime-to-add(ts, times): add ts (a timeSpan) to the current value several times. If times is zero, return the current value. times must be an integer value.
  • dateTime-to-subtract(ts): subtract ts (a timeSpan) to the current value
  • dateTime-to-subtract(ts, times): subtract ts (a timeSpan) to the current value several times. If timesis zero, return the current value. times must be an integer value.
  • utc-to-local(timeZone): returns the dateTime converted from UTC to the local time of the specified time zone
  • local-to-utc(timeZone): returns the dateTime converted from the local time of the specified time zone to utc. If the local time was ambiguous (at the moment of the switch between summer and winter the same local time occurs twice) then the first occurrence is selected.
  • null-to-date(date): returns the date provided in the parameter if the input value was null, else return the initial value.
  • invalid-to-date(date): returns the date provided in the parameter if the input value is not a valid date, else returns the initial value.

Serialize a predication from a PredicationBuilder

The builder should also support to serialize an Expressif predication:

var builder = new PredicationBuilder()
                .Chain<StartsWith>("Nik")
                .Chain<AndOperator, EndsWith>("sla");
var str = builder.Serialize();
Assert.That(str, Is.EqualTo("starts-with(Nik) |AND ends-with(sla)"));

ExpressionBuilder to programatically build an expression

We should be able to build an expression using in a pure C# approach.

var builder = new ExpressionBuilder()
                         .Chain<Lower>()
                         .Chain<FirstChars>(5)
                         .Chain<PadRight>(7, '*');
var expression = builder.Build();
Assert.That(expression.Evaluate("Nikola Tesla"), Is.EqualTo("Nikol**"));

Implement functions for path and file (IO) manipulations

Following functions are already supported in nbi.io and should be migrated to the current solution

The following transformations will consider the location of the test-suite as the base path when facing a relative path.

  • path-to-filename: returns the file name and extension of the specified path string. The characters after the last directory separator character in path. If the last character of path is a directory or volume separator character, this method returns empty.
  • path-to-filename-without-extension: Returns the file name of the specified path string without the extension. The text returned by path-to-filename, minus the last period (.) and all characters following it.
  • path-to-extension: Returns the extension (including the period ".") of the specified path string. The extension of the specified path (including the period ".") or (empty).
  • path-to-root: Gets the root directory information of the specified path including a directory separator character at the end.
  • path-to-directory: returns the directory information for the specified path string.

For these transformations, the input must corresponds to an existing file. If it's not the case an exception will be generated and the test will fail.

  • file-to-size: Gets the size, in bytes, of the file.
  • file-to-creation-dateTime: Gets the creation time of the file.
  • file-to-creation-dateTime-utc: Gets the creation time, in coordinated universal time (UTC), of the file.
  • file-to-update-dateTime: Gets the time that the current file was last written to.
  • file-to-update-dateTime-utc: Gets the time, in coordinated universal time (UTC), that the current file was last written to.

Additional text functions

The list of native functions should also support the following functions for text values

  • count-substring(substring): returns the count of non-overlapping occurrences of a substring defined as a parameter in the argument value.
  • count-distinct-chars: returns the count of distinct chars in the argument value.
  • replace-slice(start, length, substring): Replace a slice of the argument value with a substring. A specified length of characters will be deleted from the input string beginning at the start position and will be replaced by a new substring. A start value of 0 indicates the first character of the input string. If start is greater than the length of the input string, the substring is appended at the end. If 'start' is negative, the substring is appended at the beginning. If 'length' is zero, inserting of the new string occurs at the specified start position and no characters are deleted. If length is greater than the input string, deletion will occur up to the last character of the input string.
  • pad-center(length, char): Center the input string by padding the sides with a single char until the specified length of the string has been reached. If the length will be reached with an uneven number of padding, the extra padding will be applied to the right side. if the length of the argument value is greater than the length parameter, the argument value is returned.

Replicate dateTime predicates supported by nbi.io

The following dateTime predicates are currently supported by nbi.io

  • same-instant
  • after
  • after-or-same-instant
  • before
  • before-or-same-instant
  • contained-in-interval (interval)
  • on-the-day
  • on-the-minute
  • on-the-second

Support for special values named Any and Value

It should be possible to define special values named Any and Value that are respectively equal to any value (including null) and any non-null value. The literal representation are (any) and (value)

PredicationBuilder to programmatically build a predicate

We should be able to build a predication using in a pure C# approach.

var builder = new PredicationBuilder()
                .Chain<StartsWith>("Nik")
                .Chain<AndOperator, EndsWith>("sla");
var predication = builder.Build();
Assert.That(predication.Evaluate("Nikola Tesla"), Is.True);

Pay attention that some predicates are coming with an Operator and some are not (the first one)

Parse function and it literal parameters

It should be possible to parse a function name and its parameters.

  • Function's name is composed of one or more tokens separated by dashes -
  • Function's parameters are contained between braces () and separated by commas ,
text-to-function(param1, param2)

Literal parameters can also be surrounded by quotes "..." (optional)

Alternative operator to support missing property/index

This feature is useful when you've some transformations but you are not sure that some of the columns used in the transformation always exist.

The syntax [foo] ?= [bar] | text-to-upper will uppercase the value of the column named foo except if this column doesn't exist where the column bar will be used.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.