Comments (1)
I'd like to see others weigh in on this too, but the tests you have proposed are not equivalent.
- The following case is never addressed:
(
builder.name("x", tt),
builder.name("set", SetT1(tt)),
builder.name("p", InvalidTypeMethods.notBool),
)
(nameType != elemType) ==> throwsTBuilderTypeException(instruction)
does not perform the same function as
(
builder.name("x", InvalidTypeMethods.differentFrom(tt)),
builder.name("set", SetT1(tt)),
builder.name("p", BoolT1),
),
(
builder.name("x", tt),
builder.name("set", SetT1(InvalidTypeMethods.differentFrom(tt))),
builder.name("p", BoolT1),
),
(
builder.name("x", tt),
builder.name("set", InvalidTypeMethods.notSet),
builder.name("p", BoolT1),
),
The more verbose collection above is a decomposition of the equivalence classes of builder failure. They are all instances of nameType != elemType
, and every instance of nameType != elemType
falls into one of the above buckets, however, unlike the probabilistic approach, they guarantee that all of these varied failure scenarios get tested with 100% probability. Also, typically each of these sub-cases corresponds to a different way in which e.g. the signature implementation could be wrong.
succeeds(instruction)
only checks whether a type failure was raised during execution (IIRC). It notably does nothing to check that the expression constructed has the correct syntax for the given operator. For instance, ifbuilder.forall
actually constructed aTlaOper(TlaBoolOper.exists, ...)
expression, which has the exact same type signature, the new tests would fail to detect this bug in any way.
Ultimately, by my assessment:
Less than half the length (and that's after the addition of clarifying comments and imports that wouldn't be included in the actual code)
Yes, because it skips structure checking, and because it collapses all failure scenarios into one, which I'm not sure you can even do for all operators.
Better coverage, since we test for arbitrary invalid types
Untrue, since the mkIllTyped
tests are just a decomposition of (nameType != elemType)
(for the example above), and Generators.singleTypeGen
generates arbitrary types.
More readable expression of the property being tested for
Clearer naming of the test cases, so it is clear what test case or property is failing
Subjective, but I'm not opposed to more verbose test names, or splintering away IllegalArgumentException
tests.
from apalache.
Related Issues (20)
- Unexpected exception HOT 2
- Document Apalache config format + options
- A set filter over PowSet[FinSet[Int]] is not implemented
- Wrong function sets counterexample? HOT 6
- Add warning for misplaced occurrence of `oneOf` when parsing from quint
- Raise error when an unexpanded type constant is found in quint IR output
- `parse` command adds superfluous `EXTENDS` statement HOT 2
- Use `parse` to format modules. HOT 2
- Unhandled Exception (java.util.NoSuchElementException: None.get) on empty file input HOT 4
- Crash with `funArrays` encoding HOT 2
- Typechecking crashes with `IllegalArgumentException: Unsupported expression` on unbounded quantification (e.g., `\A x: P`). HOT 8
- Operator overriding in annotations
- Include constant value assignments in ITF
- Allow specifying types in separate files
- Name clash leads to operator being incorrectly marked as recursive HOT 3
- Extend the server to be able to reply with formatted TLA+ generated from the IR HOT 1
- Pretty printing the apalache IR into TLA+ does not sanitize identifiers
- Assignments in quint `init` operators need to be unprimed HOT 2
- TLA+ pretty printer produces `[]` on empty tuples, which is invalid syntax
- Add support for Quint's `allListsUpTo`
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from apalache.