Giter Club home page Giter Club logo

dentaku's Introduction

Dentaku

Join the chat at https://gitter.im/rubysolo/dentaku Gem Version Build Status Code Climate Coverage

DESCRIPTION

Dentaku is a parser and evaluator for a mathematical and logical formula language that allows run-time binding of values to variables referenced in the formulas. It is intended to safely evaluate untrusted expressions without opening security holes.

EXAMPLE

This is probably simplest to illustrate in code:

calculator = Dentaku::Calculator.new
calculator.evaluate('10 * 2')
#=> 20

Okay, not terribly exciting. But what if you want to have a reference to a variable, and evaluate it at run-time? Here's how that would look:

calculator.evaluate('kiwi + 5', kiwi: 2)
#=> 7

To enter a case sensitive mode, just pass an option to the calculator instance:

calculator.evaluate('Kiwi + 5', Kiwi: -2, kiwi: 2)
#=> 7
calculator = Dentaku::Calculator.new(case_sensitive: true)
calculator.evaluate('Kiwi + 5', Kiwi: -2, kiwi: 2)
#=> 3

You can also store the variable values in the calculator's memory and then evaluate expressions against those stored values:

calculator.store(peaches: 15)
calculator.evaluate('peaches - 5')
#=> 10
calculator.evaluate('peaches >= 15')
#=> true

For maximum CS geekery, bind is an alias of store.

Dentaku understands precedence order and using parentheses to group expressions to ensure proper evaluation:

calculator.evaluate('5 + 3 * 2')
#=> 11
calculator.evaluate('(5 + 3) * 2')
#=> 16

The evaluate method will return nil if there is an error in the formula. If this is not the desired behavior, use evaluate!, which will raise an exception.

calculator.evaluate('10 * x')
#=> nil
calculator.evaluate!('10 * x')
Dentaku::UnboundVariableError: Dentaku::UnboundVariableError

Dentaku has built-in functions (including if, not, min, max, sum, and round) and the ability to define custom functions (see below). Functions generally work like their counterparts in Excel:

calculator.evaluate('SUM(1, 1, 2, 3, 5, 8)')
#=> 20

calculator.evaluate('if (pears < 10, 10, 20)', pears: 5)
#=> 10
calculator.evaluate('if (pears < 10, 10, 20)', pears: 15)
#=> 20

round can be called with or without the number of decimal places:

calculator.evaluate('round(8.2)')
#=> 8
calculator.evaluate('round(8.2759, 2)')
#=> 8.28

round follows rounding rules, while roundup and rounddown are ceil and floor, respectively.

If you're too lazy to be building calculator objects, there's a shortcut just for you:

Dentaku('plums * 1.5', plums: 2)
#=> 3.0

PERFORMANCE

The flexibility and safety of Dentaku don't come without a price. Tokenizing a string, parsing to an AST, and then evaluating that AST are about 2 orders of magnitude slower than doing the same math in pure Ruby!

The good news is that most of the time is spent in the tokenization and parsing phases, so if performance is a concern, you can enable AST caching:

Dentaku.enable_ast_cache!

After this, Dentaku will cache the AST of each formula that it evaluates, so subsequent evaluations (even with different values for variables) will be much faster -- closer to 4x native Ruby speed. As usual, these benchmarks should be considered rough estimates, and you should measure with representative formulas from your application. Also, if new formulas are constantly introduced to your application, AST caching will consume more memory with each new formula.

BUILT-IN OPERATORS AND FUNCTIONS

Math: +, -, *, /, %, ^, |, &, <<, >>

Also, all functions from Ruby's Math module, including SIN, COS, TAN, etc.

Comparison: <, >, <=, >=, <>, !=, =,

Logic: IF, AND, OR, XOR, NOT, SWITCH

Numeric: MIN, MAX, SUM, AVG, COUNT, ROUND, ROUNDDOWN, ROUNDUP, ABS, INTERCEPT

Selections: CASE (syntax see spec)

String: LEFT, RIGHT, MID, LEN, FIND, SUBSTITUTE, CONCAT, CONTAINS

Collection: MAP, FILTER, ALL, ANY, PLUCK

RESOLVING DEPENDENCIES

If your formulas rely on one another, they may need to be resolved in a particular order. For example:

calc = Dentaku::Calculator.new
calc.store(monthly_income: 50)
need_to_compute = {
  income_taxes: "annual_income / 5",
  annual_income: "monthly_income * 12"
}

In the example, annual_income needs to be computed (and stored) before income_taxes.

Dentaku provides two methods to help resolve formulas in order:

Calculator.dependencies

Pass a (string) expression to Dependencies and get back a list of variables (as :symbols) that are required for the expression. Dependencies also takes into account variables already (explicitly) stored into the calculator.

calc.dependencies("monthly_income * 12")
#=> []
# (since monthly_income is in memory)

calc.dependencies("annual_income / 5")
#=> [:annual_income]

Calculator.solve! / Calculator.solve

Have Dentaku figure out the order in which your formulas need to be evaluated.

Pass in a hash of {eventual_variable_name: "expression"} to solve! and have Dentaku resolve dependencies (using TSort) for you.

Raises TSort::Cyclic when a valid expression order cannot be found.

calc = Dentaku::Calculator.new
calc.store(monthly_income: 50)
need_to_compute = {
  income_taxes:  "annual_income / 5",
  annual_income: "monthly_income * 12"
}
calc.solve!(need_to_compute)
#=> {annual_income: 600, income_taxes: 120}

calc.solve!(
  make_money: "have_money",
  have_money: "make_money"
}
#=> raises TSort::Cyclic

solve! will also raise an exception if any of the formulas in the set cannot be evaluated (e.g. raise ZeroDivisionError). The non-bang solve method will find as many solutions as possible and return the symbol :undefined for the problem formulas.

INLINE COMMENTS

If your expressions grow long or complex, you may add inline comments for future reference. This is particularly useful if you save your expressions in a model.

calculator.evaluate('kiwi + 5 /* This is a comment */', kiwi: 2)
#=> 7

Comments can be single or multi-line. The following are also valid.

/*
 * This is a multi-line comment
 */

/*
 This is another type of multi-line comment
 */

EXTERNAL FUNCTIONS

I don't know everything, so I might not have implemented all the functions you need. Please implement your favorites and send a pull request! Okay, so maybe that's not feasible because:

  1. You can't be bothered to share
  2. You can't wait for me to respond to a pull request, you need it NOW()
  3. The formula is the secret sauce for your startup

Whatever your reasons, Dentaku supports adding functions at runtime. To add a function, you'll need to specify a name, a return type, and a lambda that accepts all function arguments and returns the result value.

Here's an example of adding a function named POW that implements exponentiation.

> c = Dentaku::Calculator.new
> c.add_function(:pow, :numeric, ->(mantissa, exponent) { mantissa ** exponent })
> c.evaluate('POW(3,2)')
#=> 9
> c.evaluate('POW(2,3)')
#=> 8

Here's an example of adding a variadic function:

> c = Dentaku::Calculator.new
> c.add_function(:max, :numeric, ->(*args) { args.max })
> c.evaluate 'MAX(8,6,7,5,3,0,9)'
#=> 9

(However both of these are already built-in -- the ^ operator and the MAX function)

Functions can be added individually using Calculator#add_function, or en masse using Calculator#add_functions.

FUNCTION ALIASES

Every function can be aliased by synonyms. For example, it can be useful if your application is multilingual.

Dentaku.aliases = {
  round: ['rrrrround!', 'округлить']
}

Dentaku('rrrrround!(8.2) + округлить(8.4)') # the same as round(8.2) + round(8.4)
# 16

Also, if you need thread-safe aliases you can pass them to Dentaku::Calculator initializer:

aliases = {
  round: ['rrrrround!', 'округлить']
}
c = Dentaku::Calculator.new(aliases: aliases)
c.evaluate('rrrrround!(8.2) + округлить(8.4)')
# 16

THANKS

Big thanks to ElkStone Basements for allowing me to extract and open source this code. Thanks also to all the contributors!

LICENSE

(The MIT License)

Copyright © 2012-2022 Solomon White

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the ‘Software’), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

dentaku's People

Contributors

a5-stable avatar acuster77 avatar alavrard avatar alexeymk avatar dleavitt avatar flanker avatar fsateler avatar gitter-badger avatar ignatiusreza avatar jasl avatar jasoncrawford avatar joshuabates avatar krtschmr avatar legendetm avatar mestachs avatar ndbroadbent avatar prashantvithani avatar project-eutopia avatar rhys117 avatar rikkipitt avatar rthbound avatar rubysolo avatar schneidmaster avatar scudelletti avatar sliiser avatar thbar avatar tienduyvo avatar tmikoss avatar tobmatth avatar tristil avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dentaku's Issues

Operator to check if a String includes another. (LIKE or =~)

I need the operators for LIKE and NLIKE (or maybe =~) that I'd like to add which delegates to String#starts_with?, String#ends_with?, or String#include? depending on the wildcard placement.

Would you be open to adding or accepting a PR for this?

Support for % (percentage)?

2.1.0 :008 > rate = "0.5% * ((x - 500000)/2500000)" 
2.1.0 :010 > Dentaku::Calculator.new.evaluate(rate, :x => 1)
RuntimeError: no rule matched [:numeric, :operator, :operator, :numeric]
from /home/avril14th/.rvm/gems/ruby-2.1.0/gems/dentaku-0.2.13/lib/dentaku/evaluator.rb:13:in `evaluate_token_stream'
from /home/avril14th/.rvm/gems/ruby-2.1.0/gems/dentaku-0.2.13/lib/dentaku/evaluator.rb:7:in `evaluate'
from /home/avril14th/.rvm/gems/ruby-2.1.0/gems/dentaku-0.2.13/lib/dentaku/calculator.rb:30:in `block in evaluate'
from /home/avril14th/.rvm/gems/ruby-2.1.0/gems/dentaku-0.2.13/lib/dentaku/calculator.rb:50:in `store'
from /home/avril14th/.rvm/gems/ruby-2.1.0/gems/dentaku-0.2.13/lib/dentaku/calculator.rb:28:in `evaluate'
from (irb):10
from /home/avril14th/.rvm/gems/ruby-2.1.0/gems/railties-4.0.2/lib/rails/commands/console.rb:90:in `start'
from /home/avril14th/.rvm/gems/ruby-2.1.0/gems/railties-4.0.2/lib/rails/commands/console.rb:9:in `start'
from /home/avril14th/.rvm/gems/ruby-2.1.0/gems/railties-4.0.2/lib/rails/commands.rb:62:in `<top (required)>'
from bin/rails:4:in `require'
from bin/rails:4:in `<main>'

Dentaku calculations slowing down with time

Hi,

I noticed in my application that certain calculations got slower and slower with time. I profiled my code and found out that the offender was Dentaku. The version used in the application is 1.2.1. (1.2.4 behaves differently, more below).

Running Dentaku calculations i was able to find out that slowing down occurs when

  • there is a custom function, even if it is not used
  • the hash of variables submitted to calculator.evaluate as the second argument contains 2 copies of the same variable, downcased and lowercased. Example: {"R1"=>100000, "R2"=>0, "r1"=>100000, "r2"=>0, ...}. This is a way I made sure my variables were case-insensitive :-)

Both factors should be present to reproduce this behavior.

I fixed this problem in my app by removing lowercased characters, but still, this is such an interesting case, I thought you'd be interested. What is really amazing is that I am not talking about one Dentaku::Calculator object here performing all calculations, no, Dentaku::Calculator is created for every calculation. Does Dentaku store some data globally?

Here is my test script:

#!/usr/bin/env ruby

require 'dentaku'
require 'benchmark'

puts "Dentaku version #{Dentaku::VERSION}"
puts "Ruby version #{RUBY_VERSION}"

with_duplicate_variables = [
  "R1+R2+R3+R4+R5+R6",
  {"R1"=>100000, "R2"=>0, "R3"=>200000, "R4"=>0, "R5"=>500000, "R6"=>0, "r1"=>100000, "r2"=>0, "r3"=>200000, "r4"=>0, "r5"=>500000, "r6"=>0}
]

without_duplicate_variables = [
  "R1+R2+R3+R4+R5+R6",
  {"R1"=>100000, "R2"=>0, "R3"=>200000, "R4"=>0, "R5"=>500000, "R6"=>0}
]

def test(args, custom_function: true)
  calls = [ args ] * 100

  4.times do |i|

    bm = Benchmark.measure do

      calls.each do |formule, bound|

        calculator = Dentaku::Calculator.new

        if custom_function
          calculator.add_function(
            name:      :sum,
            type:      :numeric,
            signature: [:arguments],
            body:      ->(args){args.reduce(0){|a, b| a + b }}
          )
        end

        calculator.evaluate(formule, bound)
      end
    end

    puts "  run #{i}: #{bm.total}"
  end
end

case ARGV[0]
when '1'
  puts "with duplicate (downcased) variables, with a custom function:"
  test(with_duplicate_variables, custom_function: true)

when '2'
  puts "with duplicate (downcased) variables, without a custom function:"
  test(with_duplicate_variables, custom_function: false)

when '3'
  puts "without duplicate (downcased) variables, with a custom function:"
  test(without_duplicate_variables, custom_function: true)

when '4'
  puts "with duplicate (downcased) variables, without a custom function:"
  test(without_duplicate_variables, custom_function: false)
end

Output for version 1.2.1 is

$ ./test_dentaku.rb 1 && ./test_dentaku.rb 2 && ./test_dentaku.rb 3 && ./test_dentaku.rb 4
Dentaku version 1.2.1
Ruby version 2.2.1
with duplicate (downcased) variables, with a custom function:
  run 0: 0.88
  run 1: 2.43
  run 2: 3.9499999999999997
  run 3: 5.54
Dentaku version 1.2.1
Ruby version 2.2.1
with duplicate (downcased) variables, without a custom function:
  run 0: 0.13999999999999999
  run 1: 0.14
  run 2: 0.14
  run 3: 0.13999999999999996
Dentaku version 1.2.1
Ruby version 2.2.1
without duplicate (downcased) variables, with a custom function:
  run 0: 0.010000000000000002
  run 1: 0.009999999999999995
  run 2: 0.010000000000000009
  run 3: 0.01999999999999999
Dentaku version 1.2.1
Ruby version 2.2.1
with duplicate (downcased) variables, without a custom function:
  run 0: 0.010000000000000002
  run 1: 0.009999999999999995
  run 2: 0.010000000000000009
  run 3: 0.009999999999999995

As you see, only the first case shows some significant slowing down.

But version 1.2.4 works differently:

$ ./test_dentaku.rb 1 && ./test_dentaku.rb 2 && ./test_dentaku.rb 3 && ./test_dentaku.rb 4
Dentaku version 1.2.4
Ruby version 2.2.1
with duplicate (downcased) variables, with a custom function:
  run 0: 0.23
  run 1: 0.5599999999999999
  run 2: 0.87
  run 3: 1.2100000000000002
Dentaku version 1.2.4
Ruby version 2.2.1
with duplicate (downcased) variables, without a custom function:
  run 0: 0.060000000000000005
  run 1: 0.06
  run 2: 0.07
  run 3: 0.05999999999999997
Dentaku version 1.2.4
Ruby version 2.2.1
without duplicate (downcased) variables, with a custom function:
  run 0: 0.23
  run 1: 0.57
  run 2: 0.89
  run 3: 1.2000000000000002
Dentaku version 1.2.4
Ruby version 2.2.1
with duplicate (downcased) variables, without a custom function:
  run 0: 0.060000000000000005
  run 1: 0.06
  run 2: 0.06
  run 3: 0.06999999999999998

As you see

  • There is still some slowing down in certain cases
  • but it not that terrible as with 1.2.1
  • the hash of variables with lowercased and downcased copies of a variable has no effect
  • calculations slow down when there is a custom function, i.e. case 1 and 3.

I am very curious, could you please help me understand this behavior?

For now I am staying on 1.2.1, because I do need custom functions, and even modest slowing down is not good for my application.

Thank you

Unable to evaluate "-1"

Dear all,

I found your gem to be nothing short of lifesaver for my current project I'm working on. However, there is one gotcha I found recently. That is, Dentaku cannot evaluate "-1" to be a valid mathematical expression.

I can see why Dentaku can't evaluate "-1" as the error thrown here is:

RuntimeError: no rule matched [:operator, :numeric]

Any suggestion?

Recursive calculation

Is there a way to make something like this to work:

calculator = Dentaku::Calculator.new
calculator.evaluate('10 * 2 + x * y', x: 3, y: 'i + b', i:9, b:8)

thanks.

Money gem support

Hi great job on this gem. It's awesome!

I'd like to know if you have support planned for the money gem. I'd love to be able to use money objects as inputs to the formula.

I'd be more than happy to take a swing at implementing this. Is this something you'd consider including? Can you provide any pointers on how I should go about implementing this type of support?

Thanks!

Handle nil/empty values from store during comparisons

I just came across your project today and am really excited. It's just the project I was looking for. I'm running into an error when a value is nil or does not exist.

The Hash that I'm using for the calculator.store will sometimes have nil or non-existent values that the expression relies on. This causes "price > 15000" to error if 'price' does not exist or is nil. Error: undefined method `>' for nil:NilClass (this would be the same error for any operator that doesn't support nil)

How would you suggest handling this situation?

RuntimeError: parse error

RuntimeError: parse error at: '#met_values#exercise - #food_frequency#vegall )*2 - 11 + (#exercise_days *2) '
dentaku-2.0.4/lib/dentaku/tokenizer.rb:16:in `tokenize'

My formula was "(#met_values - #food_frequency )*2 - 11 + (#exercise_days *2) "

def validate_formula
calculator = Dentaku::Calculator.new
formula.scan(/#\S+/).each do |hash|
calculator.store(hash: 1)
end
errors.add(:formula, "is not valid") unless calculator.evaluate(formula)
end

Plz let me know if i m missing any thing

Question regarding "reserved" variables

Hi, we are using Dentaku for calculating statistics for sports. One of our default statistics has the acronym or and that appears to be read by dentaku as the logical operation first. These are customer entered and we are trying out dentaku as a replacement, so we already have a good deal of data entered and backtracking to have reserved words would be challenging at best.

I am looking at the code to see if I can figure out a fix but I was curious if you might have thoughts or something that I am not seeing?

Thanks!

Can we concatenate strings in dentaku?

I can make the evaluated expression return a string - like calc.evaluate("foo", foo: 'abc') returns the string "abc". Can I concat strings as well? I tried:

calc.evaluate("foo + bar", foo:"abc", bar: "def")

but it returns 0, and not "abcdef". Is there some other function that can do this?

I added a custom function for this - is it ok to use dentaku for string manipulations like this?

calc.add_function(:concat, :string,  ->(str1, str2) { str1 + str2 })
calc.evaluate("concat(foo, bar)", foo: 'abc', bar: 'def')
=> "abcdef"

Get rid of generic exception raising

Raising just a RuntimeError exception like in

raise "parse error at: '#{ input }'" unless TokenScanner.scanners.any? do |scanner|
makes it hard to fetch and identify them as Dentaku related ones. Named exceptions should be thrown ... like it is done at several other places in the code.

Just my 2 cents ;) .

When solving an expression with dependencies, are cyclic errors always raised before unbound variable errors?

Sorry for entering this as an issue - it is more of a clarification - I didn't find a contact link on the main page.

Is it guaranteed, that, given a hash of expressions without any stored values, solve! will always return cyclic errors before unbound variable errors? I checked and it seemed to work, but wanted to confirm, as that is critical to my scenario.

    calc = Dentaku::Calculator.new
    # case 1
    need_to_compute = {
      income_taxes: "annual_income / 5",
      annual_income: "monthly_income * 12"
    }

    # case 2
    calc.solve! need_to_compute #=> Unbound variable error

    need_to_compute = {
      income_taxes: "annual_income / 5",
      annual_income: "monthly_income * 12",
      monthly_income: 'annual_income'
    }

    calc.solve! need_to_compute #=> Cyclic error 
    # Is it always the case that cyclic error will be raised before 
    # unbound variable error in cases like case 2 above?

Divide by Zero in `solve!`

If any of the equations that you pass to solve! divides by zero, the whole method errors out. I see why that could be a preferred approach. We would like to be able to return nil instead of a full error so the rest of the equations can complete.

I was curious if you felt this might be a more desirable default as well or if you would prefer the current behavior. And if you would prefer the current behavior, perhaps passing in an optional error handler or something along those lines?

I am happy to submit a PR, but I thought it would be best to get an idea of direction from you all first.

Thanks! Geoff

Array parameter being flattened with others

The following function works well when the expression evaluates to two strings, but fails when haystack is an Array and needle is a single value.

calculator.add_function(
    :is_included,
    :logical,
    ->(haystack, needle) {
        haystack.include?(needle)
    }
)

The flat_map method is causing the first array parameter to be flattened with the 2nd string parameter. Is there a way this can be supported?

// lib/dentaku/ast/function.rb 
36  def value(context={})
37     args = @args.flat_map { |a| a.value(context) }
38     self.class.implementation.call(*args)
39  end

Handling variable values of wrong data type in numeric operations

calculator.evaluate("10 + 'abc'")
=> throws RuntimeError

calculator.evaluate("10 + x", x: "abc")
=> 10

Is this reasonable? It seems to me that the latter should also follow the pattern where it returns nil, with its evaluate! counterpart throwing an Exception.

License missing from gemspec

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can imagine, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.

Combinator fails on incomplete statements

Dentaku::Ast::Combinator#valid_node? seems to fail on incomplete logical statements:

pry(main)> Dentaku('true AND')
NoMethodError: undefined method `dependencies' for nil:NilClass
from [..]/dentaku-2.0.10/lib/dentaku/ast/combinators.rb:20:in `valid_node?'

% giving error using v 2.0.8

I wrote the lines in rails c
calculator = Dentaku::Calculator.new
calculator.evaluate("60000*67%")
The result comes:

Dentaku::ParseError: Dentaku::AST::Modulo requires numeric operands
from /Users/sulman/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/dentaku-2.0.8/lib/dentaku/ast/arithmetic.rb:11:in `initialize'

Improve performance of "if" function?

I noticed that the "if" function is quite a bit slower than some others such as arithmetic operations. I was wondering if the author sees any obvious performance bottlenecks in the implementation of that function that might be improved. I'm using the code below as a simple benchmark and noticing it take quite a bit longer than the equivalent ruby code.

10000.times { calc.evaluate('if (a > b and a > c, a, c)', a:1, b:2, c:3) }

I dug into the source a bit myself but didn't see anything obvious. Thanks in advance and thank you for this amazing library!

ArgumentError: negative argument

Hi, I'm getting ArgumentError: negative argument when I do:

calculator.evaluate!('-(F2)', 'F2' => '5')

Looks like it's only an issue when F2 is a string.. this does not cause the error:

calculator.evaluate!('-(F2)', 'F2' => 5)

Strings do appear to be supported though, since this doesn't throw an error either:

calculator.evaluate!('F2', 'F2' => '5')

Any thoughts on this? Thanks!

disable cache for certain formula

Hi, this is really an awesome library, I love it! especially your idea to cache the formula. However, is it possible to disable cache for certain formula? I've seen the code and it seems dentaku can't do that, but in case I'm wrong.
the use case for this such as, user want to validate their formula. Therefore no need to cache it, however when they are sure that the formula works of course we need to cache it.
If it's good idea I don't mind to create pull request about it

Calculator.dependencies raises `undefined method `flat_map' for nil:NilClass` when the expression contains a call to (some?) included string functions

Tested on commit 1b2036b

Example of failing unit tests:

    it "finds dependencies in a statement using CONCAT" do
      expect(calculator.dependencies("CONCAT(bob, dole)")).to eq(['bob', 'dole'])
    end

    it "finds dependencies in a statement using LEN" do
      expect(calculator.dependencies("LEN(bob)")).to eq(['bob'])
    end

Result, for both calls:

     NoMethodError:
       undefined method `flat_map' for nil:NilClass
     # ./lib/dentaku/ast/function.rb:11:in `dependencies'
     # ./lib/dentaku/calculator.rb:58:in `dependencies'
     # ./spec/calculator_spec.rb:70:in `block (3 levels) in <top (required)>'

Is there a way to parse == as = ?

First, thanks for your amazing work on that gem. It's on a very sweet spot, since it allows us not to write a full grammar, yet :-)

I am parsing iOS generated NSPredicate strings and these strings use == instead of =.

I looked at the code and I think for now I will have to pre-process the string, but I wonder: would it be hard to allow to tweak the tokenizer underneath the calculator a bit? Either to add synonyms for this case, or to post-process the result? Thanks!

Error when using function result with identifier arguments in further operation

I am having trouble using a function with identifiers as arguments and then using the result in a more complex equation:

c = Dentaku::Calculator.new
c.store("multi_color" => true, "number_of_sheets" => 5000, "sheets_per_minute_black" => 2000, "sheets_per_minute_color" => 1000)
c.evaluate('number_of_sheets / if(multi_color, sheets_per_minute_color, sheets_per_minute_black)')
#=> NoMethodError: undefined method `type' for #<Dentaku::AST::Identifier:0x007fc051849f90>

Using integers or floats as arguments to the if function works without problems. Also not using the computed result of the if function any further works, too (i.e. only evaluating the denominator). Since it calls type on identifier I assume it tries to get the type of the argument of the if function before the identifiers were actually resolved into concrete numbers. Or am I doing something wrong?

Allow non-String arguments for string functions

Currently all string functions fail with Value expression String can't be coerced into Fixnum if invoked with numeric arguments, for example:

concat(round(5.124827, 1),"%")

Would it be possible to change the implementation and allow the above to succeed by implicitly converting any non-String argument to string. With that the above example would result in:

5.1%

Rounddown can't handle itself

Should this work?

c = Dentaku::Calculator.new

c.evaluate 'rounddown(rounddown(4.5) + 0.5)'

I know it seems like a nonsense formula, but that's only because I've stripped it down from what I'm using. It seems whenever a rounddown exists inside another rounddown, I get a "RuntimeError: no rule matched {{rounddown ( 4.5}}"

I played around with it a bit. rounddown can't be inside round, but round can be inside rounddown, so it seems like rounddown is returning some mutated symbol that remains forever changed.

...or something.

Thanks for taking the time to read this,
flibbles

Different behavior with unbound variables between evaluate and solve!

When you execute, with x unbound

calculator.evaluate("x+1")
#=> nil
calculator.solve!(y: "x+1")
#> {"x" => 1}

I was curious if this was desired behavior? Personally, I would prefer the nil option but I wanted to see what you all thought. The ! would tend to indicate an error, but I was hoping to avoid those (see my other ticket #46).

Again, I am happy to submit a PR but wanted to run it through you all. Thanks for the gem, saved me a ton of time :)

Thanks! Geoff

expression not parsed

this one doesn't work
any idea?

[49] pry(main)> calculator.evaluate("(((695759/735000)^(1/(1981-1991)))-1)*1000")
ZeroDivisionError: divided by 0
from /home/atzorvas/.rvm/gems/ruby-2.2.1@cessda/gems/dentaku-2.0.2/lib/dentaku/ast/arithmetic.rb:64:in `**'

Undefined methods

I can build a calculator object and use it's evaluate, store and add_function methods.

However, when i try to use solve! or dependencies, I get some undefined methods errors.

<main>': undefined methodsolve!' for #Dentaku::Calculator:0x0000000167e5c0 (NoMethodError)

block in <main>': undefined methoddependencies' for #Dentaku::Calculator:0x00000001e7e298 (NoMethodError)

Installed with command 'gem install dentaku' and using ruby 2.0.0p576.

Latest version 2.0.1 Modulus (%) not supported

Utilizing the modulus function you get error:
KeyError: key not found: :mod
dentaku-2.0.1/lib/dentaku/parser.rb:127:in fetch' dentaku-2.0.1/lib/dentaku/parser.rb:127:inoperation'
dentaku-2.0.1/lib/dentaku/parser.rb:41:in parse' dentaku-2.0.1/lib/dentaku/calculator.rb:55:inblock in ast'
dentaku-2.0.1/lib/dentaku/calculator.rb:54:in fetch' dentaku-2.0.1/lib/dentaku/calculator.rb:54:inast'
dentaku-2.0.1/lib/dentaku/calculator.rb:36:in block in evaluate!' dentaku-2.0.1/lib/dentaku/calculator.rb:73:instore'
dentaku-2.0.1/lib/dentaku/calculator.rb:34:in evaluate!' dentaku-2.0.1/lib/dentaku/calculator.rb:28:inevaluate'

Implement a case statement

I have need of a case statement to handle looking up values. This is the way I would be inclined to implement it (I pass values in from a yaml file):

price: 100
sale: 'black friday'
price_after_sale:
  " CASE sale
       WHEN 'black friday':  "price - price * 0.25"
       WHEN 'christmas eve': "price - 5 - price * 0.35"
    END"

so that price_after_sale would be 75. I might have time to work on this but would like to know the preferred interface first and if you're interested in having this.

CONCAT with more than 2 arguments

Thanks for this very useful library! It's pretty neat, I'm considering using it to safely evaluate some user-entered functions.

The CONCAT function can take any number of arguments, but will only concatenate the first two arguments:

Dentaku::Calculator.new.evaluate('CONCAT(a, b, c)', a: 'a', b: 'b', c: 'c') #=> 'ab'

So to get full concatenation, I have to chain multiple concats:

Dentaku::Calculator.new.evaluate('CONCAT(a, CONCAT(b, c))', a: 'a', b: 'b', c: 'c') #=> 'abc'

I was thinking of opening a PR that would allow any number of arguments to CONCAT and combine them all.

Math expression correctness

When I evaluate a wrong math expression f.e. calculator.evaluate('c + - )'). It gives me different gem errors. I think it's better to show an error in calculate object for a particular reason.

Can't resolve dependency when variable declaration is in upper-case

Hello guys,

I've faced the issue which is illustrated here:

calc = Dentaku::Calculator.new
need_to_compute = {
  "Different_case_ref"=>1,
  "result"=>"different_case_ref*1",
}
calc.solve!(need_to_compute)

And here what I'm getting

calc.solve!(need_to_compute)
NoMethodError: undefined method `each' for nil:NilClass
from /Users/mehal/.rbenv/versions/2.1.2/lib/ruby/gems/2.1.0/gems/dentaku-1.2.4/lib/dentaku/dependency_resolver.rb:21:in `tsort_each_child'

If I would change Different_case_ref to different_case_ref

calc = Dentaku::Calculator.new
need_to_compute = {
  "different_case_ref"=>1,
  "result"=>"different_case_ref*1",
}
calc.solve!(need_to_compute)

Everything magically works!

General parse syntax, dependency resolution, and a bug

Wanted to give some feedback - I'm using Dentaku to allow non-technical admin users who are creating forms that end-users fill in to create formulas (IE, answer_c = answer_b + answer_a).

I've made a couple of modifications that I figured I'd share and see if you'd want to include them.

  1. More general parse syntax.
    I happened to need dots to be valid in identifier names, so I made a hacky change to the regex. Longer-term, allowing customizable overrides of the syntax might be nice.
  2. Dependency resolution.
    A user may have entered several formulas (in any order) which depend on one another in some way. When you want to compute the value of all of these, the order in which you compute them matters. To figure out the order, I used TSort and added a dependencies method on Calculator. One extension of my work might be to include dependency resolution as a first class feature, using something like a compute_many function which takes in multiple interdependent formulas.
  3. Quick bug.
    At the beginning of Calculator.store, you check if value, the assumption being that if the user only passed the first argument then they want to store multiple values (IE, they passed a hash). However, false is a valid value to want to store. Instead, the line should read if !value.nil? to distinguish between nil and false

Thanks for a tremendously helpful library.

Bitwise operators support

Hi,
This is an awesome gem! I was looking for 'safe_eval' and this gem works perfect :).

But, may I make a wish that to support the | and & operators?

calculator.evaluate('1 | 2') #=> 3
calculator.evaluate('3 & 2') #=> 2

In addition, the ^ operator stands for the power operator currently.
Shouldn't it be the bitwise xor operator to fit most programming languages?

Thanks :)

Update

I forked and try to add this feature, it seems works :)
master...david942j:master

IS Operator for IS BLANK or IS NOT BLANK

I would like support for an "IS operator where I can do "IS BLANK" or "IS NOT BLANK" to see if a particular value exists or not. In my own expression evaluator I'm using Object#blank? and Object#present? from Rails to check this condition.

Is this something you'd be willing to add or accept a PR for?

Use of to_sym can cause memory leaks

I noticed that string keys in the variables hash are converted to symbols prior to calculation. This can lead to memory leaks over time when the set of possible variable names is infinite (for example, when end users can create their own variables), as Ruby never garbage collects symbols. Is it possible to leave keys as strings to prevent this? A quick search of the codebase for "to_sym" will show a few places where this is problematic.

How to resolve a formula with conditional dependencies?

I'm stumped by a case where a conditional formula sometimes depends on another formula, that might not be resolvable all the time. A simplified case for example:

calc = Dentaku::Calculator.new

data = {
  what_i_want: "IF(a != 0, formula, 0)",  
  formula: "10/a",  
  a: 0  
} 

I'm looking for the value of what_i_want. However, when running calc.solve!(data), I'm faced with Dentaku::ZeroDivisionError, because it also tries to solve formula.

evaluate! works in this case:

calc.evaluate!("IF(a != 0, formula, 0)", { formula: "10/a", a: 0 } )
=> 0

However, it doesn't handle dependency formulas:

calc.evaluate!("IF(a != 0, formula, 0)", { formula: "10/a", a: 1 } )
=> "10/a"

I could go on fetching in all the dependencies, resolving them and storing in memory. However, I would still end up trying to evaluate 10/0 at one point and getting errors:

calc.store(a: 0)
=> #<Dentaku::Calculator:0x007f9f763802f8 @ast_cache={}, @memory={"a"=>0}, @tokenizer=#<Dentaku::Tokenizer:0x007f9f76380208>>
calc.dependencies("IF(a != 0, formula, 0)")
=> ["formula"]

Is there any way to handle this situation? Besides just evaluating the result of calc.evaluate! again and again, until it stops changing?

Complex expressions inside user-defined methods

Hello,
I've been using Dentaku for my project and it's been fantastic so far. Great work.

But I am experience what I believe is a bug.
The following commands reproduce it:

c = Dentaku::Calculator.new

c.add_function(
    name: :max,
    type: :numeric,
    signature: [:non_close_plus],
    body: ->(*args) { args.max }
)

c.evaluate 'MAX(5, 8)' # works fine
c.evaluate 'MAX(5, 8 / 2)' # causes ArgumentError: comparison of Symbol with 8 failed

Looking through the code, it looks like it's probably including the '/' as an argument, but I'm not positive. If I make the max function using signature: [:numeric, :numeric], it seems to work fine though.

Thanks & cheers,
flibbles

Formulas without any calculation return the original input (instead of casting to numeric type)

We got a bit of surprising output when trying to round the result of a formula calculation:

> Dentaku::Calculator.new.evaluate!(some_formula, some_variables).round(3)
NoMethodError: undefined method `round' for "52.0":String

It turns out Dentaku returns the original variable if no calculation is done:

> Dentaku::Calculator.new.evaluate!('a', a: '52.0').round(3)
NoMethodError: undefined method `round' for "52.0":String

This seems like really surprising behavior. I would expect evaluating a formula with Dentaku to always return a numeric type.

Is Dentaku safe for untrusted user input?

Since I could not find this information in the readme, I looked at the code (not in depth though) and my understanding is that Dentaku seems safe to process user entered input (untrusted) without risk of remode code execution (like the liquid template library).

Can you confirm that? If that is the case, maybe it's worth adding to the readme?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.