Comments (5)
So it is more an ignorelist
maybe? We can do both.
from go-ftw.
I think it's important that excludelist
would contains tests identifiers which could be ignored, if the test failed - not skipped the test. :)
from go-ftw.
After doing a first pass, I come with this. Having three additional maps for ignore, pass or fail.
An example is:
testmodify:
ignore:
'920400-1': 'This test result must be ignored'
... (this is a list)
forcepass:
'123456-1': 'We expect this test to pass once issue github.com/.... is resolved'
... (this is a list)
forcefail:
'200200-4': 'Force fail this test until we find a solution to bug #1234'
... (this is a list)
The id of the test should match with the proper test you want to modify its results, and the text should be handy for printing in the results.
This only modifies the results, but the test is still executed.
What do you think?
from go-ftw.
I don’t think we need that many kinds of lists - if case of ftwrunner I made only one list, but that list can modify the result as this:
if TEST == PASSED:
if TEST in WHITELIST:
result = PASSED_BUT_LISTED // this is important for user, because the test can be removed from the list
else:
result = PASSED // this is the expected
else:
if TEST in WHITELIST:
result = FAILED_BUT_LISTED // this is the expected
else:
result = FAILED // ops...
This only modifies the results, but the test is still executed.
Agree. But not sure this implies the number of lists.
from go-ftw.
This was implemented.
from go-ftw.
Related Issues (20)
- Ability to define custom request headers for logmarker requests HOT 2
- What are the log format requirements for ftw? HOT 14
- Dependency Dashboard
- Provide a proper way to define expected response data HOT 5
- httpbin does not reflect requests perfectly HOT 4
- Add a flag or configuration value to test based on rule attributes
- Allow precise configuration of raw request body HOT 1
- Make the include list configurable in ftw.yaml
- Make it possible to run the same test against multiple URIs
- Go install fails (directives to be interpreted differently than if it were the main module) HOT 6
- Throttle request speed option HOT 1
- Action Required: Fix Renovate Configuration
- Implement missing settings from schema HOT 1
- Remove magefile? HOT 3
- Regression test loading broken in 1.0.0 HOT 1
- New name for go-ftw HOT 10
- Add support for matrix / parameterization HOT 1
- Add support for self-update
- Header names are checked with case-sensitive match HOT 3
- Implement multi-value headers
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from go-ftw.