typeable / comparest Goto Github PK

blocked by biocad/openapi3#17

https://typeable.slack.com/archives/C01N72YV29L/p1615291524049800

Currently adding a type is considered a breaking change, and removing it is considered a non-breaking change.

Currently our "check issue" only remembers one trace, either the producer or the consumer. However we need a way to refer to both, because in some places we "flip variance", and the resulting tree of errors has the producers and the consumers jumbled up. Otherwise we need to pass the variance to the leaves, so that they know which source to include in the error.

Problems with the source spec are reported as semantic errors. This is not the best thing at least because the consumer can also have source spec errors.

In general: if a field was considered "additional" (was "open", allowed any value), adding a definition in the next API should not report that, for example, "type Null was removed".

https://github.com/typeable/openapi-diff/tree/master/test/golden/common/pathItem/operation/responses/change/headers/optional

We could have complete support for comparing regexes (in e.g. the pattern field in json schema), if we compiled the regexes to automata and did set-theoretic operations on the automata (product with intersection, product with union, negation), and used an emptiness test for an automaton. These operations can be carried out somewhat efficiently. After talking a little bit with ekmett about this he suggested that we use Binary decision diagrams and in particular ZDD, for determinization (NFA -> DFA) and NFA membership test.

Currently normalizePath cannot function, because after pattern matching on Snoc we don't know anything about b other than it being Steppable.

The current proposed solution is to instead define:
normalizeStep :: Step b c -> (Trace OpenApi b -> Trace OpenApi c)
as a method of Steppable, in turn making Steppable non-polykinded. The intention is that when defining the Step b c as a data family instance, we would know for which constructors normalizeStep returns a simple Snoc, and for which we instead restart from the root and return a const ....

An alternative solution is to define a series of things:

-- a type family:
type family Prev :: Type -> [Type]
-- a number of constraint implications (via superclass constraints):
APIStep a b => Elem a (Prev b)
(Subtree b, Elem a (Prev b)) => (Subtree a, APIStep a b)
(Subtree b, Steppable a b) => Elem a (Prev b)

This would allow to pattern match on Trace a c, obtaining Trace a b and Step b c, and given Subtree c conclude Subtree b, thus allowing recursion.

each test takes a triple of schema A, schema B, expectation

blocked by biocad/openapi3#14

Currently we sometimes pass traces around in an environment monad, using monadic idioms, and sometimes we use Traced, which is effectively a comonad, using comonadic idioms. Experience showed that the latter is the more fruitful approach.

https://typeable.slack.com/archives/C01N72YV29L/p1624459527017500

Until that is done, we always implicitly assume that oneOf clauses are disjoint. No unsupported warning is shown because that would imply a warning for every single instance of oneOf.

Running on relatively small (70 KB) inputs yields 1-2 seconds runtimes which seems unreasonably large.

Find out if there are easy ways of reducing runtime.

Complete all path-related tree compare. Except Schema which is quite big.

There are no identifiers for security schemes in openapi. Think how to identify them in reports

Simple CLI interface to call the tool on two schemas

Using optparse-generic seems like a reasonable first step.

In actual fact, they are unique by (name, position)

https://github.com/typeable/openapi-diff/blob/7df8b856239388b820762e859501262a3428a892/src/OpenAPI/Checker/Validate/Operation.hs#L116

Report the breaking changes for the following features of OpenAPI:

• add path (endpoint)
• remove path (endpoint)

The current implementation of path templates only supports the whole path fragment being a template. The OpenApi spec also allows parts of the path fragments to be templated.

Options like --only-errors and --silent are currently no-ops. Fix this.

Things like {a|b}c will not match ac.

https://github.com/typeable/openapi-diff/blob/b51d410a39553e3c948fecdec15616dff7b9b7b4/src/OpenAPI/Checker/Validate.hs#L42-L47

Callback have some non-trivial templating in the path keys that seems to not be implemented in openapi3

including json path parsing

If we have a conjunction of conditions in the producer, and a disjunction of conditions in the consumer, then we will test the conjunction against every item in the disjunction and see if anything succeeds. This is not always enough. We could try to heuristically split the producer's set along some e.g. enum component into a dijsunction, and then do the check for each of them (this process could be recursive).

See this test case: https://github.com/typeable/openapi-diff/tree/master/test/golden/common/property-removed

Currently if the producer schema indicates that no values of a particular type are allowed, we test the consumer formula for whether it is contradictory, i.e. whether it also allows no values.

The formula is in Disjuctive Normal Form, and every conjunct is checked for contradiction -- currently this always fails (but if there are no conjuncts the check succeeds).