I asked on Discord about this and got some feedback; opening an issue to discuss if we should implement it.
Motivation
This is how tests currently work:
function testExample() public {
assertTrue(x); // okay
assertTrue(y); // fail
assertTrue(z); // okay
}
// [FAIL] testExample()
z
was evaluated even though y
did not meet the requirements.
But say you're testing an interaction with complex contracts with lots of external calls and traces:
function testExample() public {
c1.withdraw();
assertEq(myBalance(), 100e18); // fail
c2.deposit();
assertEq(c2.availableFunds(), 100e18); // fail
c2.borrow(); // revert
assertApproxEqRel(dBalance(), 2500e18, 0.98e18); // not reached
}
// [FAIL. Reason: Collateral cannot be something something.] testExample()
- This would produce many traces
- Trigger multiple errors and eventually cause an unrelated revert in an external contract
- Show it as the reason why the test failed, which is confusing
- Not stop deployment scripts if a condition is not met
- Take time to fail when testing against a forked network (until everything is executed)
Solution
If this sounds good, should it be done in Forge Std or Foundry?
Foundry
Allow halt_tests
, halt_on_failure
or similar, in foundry.toml
.
Forge Std
Add stdConfig
mapping - like a mini version of foundry.toml
but for Forge Std.
Then, users could access it their setUp
stdConfig["halt_tests"] = false;
.
Other things I'd considered:
halts
modifier, but got feedback that it's better to be a switch on the project level
This is how we would do it: dapphub/ds-test#40
Example
function testExample() public {
c1.withdraw();
assertEq(balance(), 100e18); // fail => halt
c2.deposit(); // not reached
assertEq(c2.availableFunds(), 100e18); // not reached
c2.borrow(); // not reached
assertApproxEqRel(dBalance(), 2500e18, 0.98e18); // not reached
}
[FAIL. Reason: Halted!] testExample()
Logs:
Error: a == b not satisfied [uint]
Expected: 100000000000000000000
Actual: 50000000