rodrigodesalvobraz / neuralpp Goto Github PK

Main components of Symbolic

• Expression
• Interpreter
• Eliminator
• Normalization
• Interval

Expression

• Make sure the ordering of the functions are the same within the classes
• Are the specific error classes still necessary? Can we reuse the ones that python already has? Should we add an error message to these classes if we want to keep them?

Interpreter

Evaluates and simplifies the function

• Should we rename this to evaluator? I think the naming is a little confusing, since the word "interpreter" has difference meanings

Eliminator

Eliminates the quantifier expression

• Add doc string to explain what is happening (move the doc string from normalizer into eliminator)

Normalization

Rename GeneralNormalizer and QuantifierFreeNormalizer

• GeneralNormalizer currently pushes the integral down into the leaves of a piecewise function. This does it lazily (and we also have another normalizer called LazyNormalizer. It also does not push down the add.

• QuantifierFreeNormalizer does it eagerly. It pushes down the add and the integral, and you have to renormalize after you normalize the leaf/ subsection of the tree

• Delete LazyNormalizer

Interval

[max(lower_bounds), min(upper_bound)] => Also need to normalize the interval => min and max is technically also a piecewise

• Refine Intervals
• Combine in MagicInterval into the normal Interval class

Docstring

Some docstrings are outdated

• Add docstrings for EquivalenceClass and OrderedZ3Expression in z3_expression

Profiler

• Look into python profiling tools

Typing

• Think of names to help with docstrings => min max aren't integrable fast

Linting

• Lint all the files for clean code

>>> _updated_type_dict_by_context({x:int, y:int, z:int, x*y: int->int->int, (x*y)*z: int->int->int}, True)
{x:int, y:int, z:int, x*y: int->int->int, x*y*z: int->int->int}

But it should return with the entry x*y*z: int->int->int->int, it's because we don't re-compute the type of the simplified keys.

If we do want to re-compute the type of the simplified keys, then type_dict is not a very useful design. By having type_dict, we assume the types of all subexpressions can be found in it. But that's not always true when we introduce simplification. For example, when we simplify (x+1)*(x+1) (its type_dict would be {x:int, x+1:int->int->int, (x+1)*(x+1): int->int->int}), we get x**2+2*x+1. Here x**2 and 2*x are subexpressions not in type_dict, and we need to compute their type from the variable types.

So we should just redesign type_dict to not contain types of built-in functions, but only types of variables (including uninterpreted functions). And always compute the function types based on pre-defined rules (e.g., "int + float returns float") and the variable types.

I was searching for a name for the class of operation of quantifier expression, which should not only be a Context/Expression, but also contain a property identity (as in identity element), so that when the constrain of a quantifier expression is unsatisfiable, it can be automatically evaluated to the identity of its operation.

When searching for the name, I found the usage of “quantifier” a bit hard to understand for anyone not familiar with Probabilistic Inference Modulo Theories (PIMT), which uses the word “quantifier” in an unusually generalized way. Since “quantifier” is mostly understood to mean either the universal quantifier (“forall”) or the existential quantifier (“exists”) (there are more edgy ones, not including summation or product). What PIMT calls “quantifier” is usually referred to as iterated binary operation, which is also a bit long and uncommon.

I found a mathematicians’ discussion on the topic of wording of iterated binary operation, where no good replacement was concluded on (they proposed “summation”, “magmatic operation”).

Further, our “quantifier” operation is supposed to be commutative and associative (as defined in the PIMT paper). So technically the set we operate on is a commutative monoid, or even an abelian group (given inversion).

To conclude, I think we have two names to decide:
a. The class name of “quantifier” operation
b. The class name of what’s currently “QuantifierExpression”

Some choices of “a” are:

1. QuantifierOperation. We need to explain to the library users that QuantifierOperation should be commutative and associative and contains an identity element.
2. CommutativeMonoidOperation/AbelianOperation. The operation class name is more precise; in terms of “CommutativeMonoidOperation” vs “AbelianOperation”, the former is too long, the latter is not that precise if we don’t include an inverse() method.
3. CommutativeAndAssociativeOperation (see the picture above). The fact the it has an identity element is not indicated. Also too long and not straightforward.

Some choices of “b” are:

1. QuantifierExpression. We should include some explanation of the “quantifier” definition here.
2. GeneralizedQuantifierExpression. More precise, as we are not altering the definition of quantifier but extending it.
3. IteratedBinaryExpression. But it’s actually more than that because we requires commutativity, associativity and identity element of the operation.
4. AbelianExpression. Too pedantic?

Typing System Around SymPy Expression

Issue:
Currently, we have a wrapping around SymPy Expression. We currently have a type dictionary for a whole expression. Inferring the type in SymPy is actually a bottleneck. This is where Xiang is finding a lot of the runtime at.

Our higher order inference typing is also already an issue (E.g.: Our typing for add is Callable[[float, float], float].) This is because SymPy and Z3 can both do (add, 1, 2, 3), which does NOT follow the structure of our callable. It may be easier to just discard this.

Solution:
SymPy has something called domain (like real number R, etc.). It might be more useful to use the domain. If it's a real number, just return float. => https://docs.sympy.org/latest/modules/polys/domainsintro.html

Xiang also hacked in another version of the typing for the operator. He changed the Callable[[float, float], float] to Callable[[], float].

Use SymPy Poly

Issue:
SymPy does not know anything about the thing it is trying to integrate. Thus, it is trying all of the heuristics it knows to integrate it.

Solution:
We are already only using polynomials, so we should use the built-in polynomial that SymPy has to offer. =>https://docs.sympy.org/latest/modules/polys/reference.html

Can also convert the SymPy Poly object into a binary

Can also simplify the object => not need to convert it into a binary