Comments (6)
I'll address the part of the question where you have provided a concrete example:
* def validator =
"""
function(list) {
var valid = true;
var userValidator = function(user) {
if (typeof user.userId != 'number') valid = false;
if (typeof user.id != 'number') valid = false;
if (typeof user.body != 'string') valid = false;
}
list.forEach(userValidator);
return valid;
}
"""
Given url 'https://jsonplaceholder.typicode.com/posts'
When method get
Then status 200
And assert response.length > 0
And assert validator(response)
from karate.
I appreciate the response, but this is not entirely what we intended to test.
Observe the following :
-
We did not test for a non empty list. .length is a property of string too, but that is implementation issue not a problem with the design - you can fix it. But requires some more code - which should be trivial... Array.isArray() .
-
of the schema - { "userId": int , "id": int , "title": string , "body" : string }
Here is why it gets incredibly hard. The naive current approach won't work - simply because schema means field1, field2,... must exists, and no other fields exist. That is a very crucial test.
That test is entirely missing - and is hard to implement.
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Object/getOwnPropertyNames will help, but then, you need to compare field names - and ordering is not guranted .... isa nightmare.
Being concrete - here is what we dont want to come up as an object - notice how current implementation passes it through:
{ "userId": int , "id": int , "title": string , "body" : string , "company" : "string" }
- Moving onto the tests:
if (typeof user.userId != 'number') valid = false;
if (typeof user.id != 'number') valid = false;
incredibly bad idea. if user.id was a floating point, that test would pass. Now one can argue that:
http://stackoverflow.com/questions/14636536/how-to-check-if-a-variable-is-an-integer-in-javascript
won't really have existed - if that check was in essence trivial. It is not. JavaScript was NOT build to do software testing - you need a different class of language to do so.
In fact I would argue, the framework must have these things thought about - and kept in place.
- The tests are NOT lazy. That is, after first failure, once the variable switches to false, it should have come out of the loop. In a fin org, we had to deal with fast failure - we had to deal with 100,000 of such json objects or more.
This is an implementation bottleneck - but I find it perplexing that no such higher order functions are actually present within the framework.
- Now that we have that code, if you want to change the schema of the JSON object to apply the same thing to another rest service, returning a different schema - we need to write all of these hard coded code back again, while in principle something can be written as :
// schema -> key value pair of field to type mapping
// obj -> they are always a key value pair of field to value mapping
is_valid_schema( obj, schema ){
// the fields set are equal
obj.keySet == schema.keySet &&
// no field exists such that the value of the field is different from the type in schema
!exists ( obj.keySet ) where { !(obj[$.o] isa schema[$.o]) }
}
// check valid schema
and now, it is good. Now, in this form, we do not need to change the code ever, even if the schema or anything changes. These are next to impossible to do in languages which were not built for Test Automation. Neither Java, not Python, nor JavaScript cuts it. Groovy is close, but not close enough.
Next is the data splicing. From the response I need to arbitrarily project/select and create objects to pass it to the next rest call. That will also be another problem.
Again, let me fix the head on aspects here. :-)
The approach you have is cool, and I appreciate that and I have already shared it with all my colleagues, and I appreciate the hard work you have put - both to create and to open source - I know how incredibly hard these are and even we could not open source. So, I whole heartedly congratulate you.
But my worry is the path you are taking is treaded too many times already - and how to validate predicates should be the main focus - which I do not see much here. With the changes I am suggesting if you try to write libraries in JS or Java, you would see why it won't work.
You need to have a better class of language. That is precisely why I said - "it won't work."
People would have to write code - and code and more code w/o a DSL.
Hope this strikes a cord.
Thanks for your time to answer and post.
from karate.
@nmondal I'll leave it to readers to decide whether your points and concerns are valid.
from karate.
After some consideration, opened #3 specifically to solve for iteration better. Thanks @nmondal for the feedback.
from karate.
Oops correction, #4 is for the iteration syntax.
from karate.
Thanks to @nmondal for the awesome feedback. With the latest commit and the introduction of the match each
syntax ( see #4 ), the solution has been simplified to:
Given url 'https://jsonplaceholder.typicode.com/posts'
When method get
Then status 200
And assert response.length > 0
And match each response == { userId: '#number', id: '#number', title: '#string', body: '#string' }
from karate.
Related Issues (20)
- Karate contains only shortcut fails in v1.4.1 HOT 4
- Schema validation with dynamic keys HOT 3
- How to call Json Object as a Parameter in Karate feature HOT 1
- Schema validation doesn't support optional objects HOT 1
- Scenarios should be referenced by their name rather than line number
- Karate Robot cannot handle the ‘>’ character, yield org.graalvm.polyglot.PolyglotException: Invalid key code HOT 4
- karate.abort() response status == 500 HOT 3
- Karate robot cannot handle panes with "/" in name. Very niche problem HOT 1
- Variable reference lost after calling a scenario in separate feature HOT 3
- Not able to karate test case, getting issue HOT 1
- Karate summary reports not showing all tested features HOT 1
- [karate 1.4.1] OOM when load testing with mock for several minutes HOT 4
- Add Karate configuration option to disable the "no features or scenarios found" assertion HOT 1
- Does karate gating support Java DSL ? HOT 2
- Mutual SSL documented the wrong way HOT 3
- GraalVM changes incompatible with Karate HOT 10
- How to switch from Java selenium webdriver to Karate driver HOT 1
- Missing header in spite of "configure headers" HOT 5
- afterScenario hook not running when scenario fails HOT 1
- File upload is not working when using Web driver when running test on Sauce Lab or Remote Grid HOT 7
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from karate.