Clean the CI and improve it - there is too much boilerplate at the moment. Should be done before releasing v0.4.0

Remove unwrap calls when possible and handle them as Unprocessable failures with an UnprocessableReason.

Check todo comments, look at assertion impls, json_body for example.

We currently manage logs via a structure and different methods called depending on the part under test.

Ideally each DSL should implement its own behaviour so we can personalize the output format. The recent work on jsonpath highlighted the need of a more complex log structure which is introducing a Hand::Compound variant (also used by between predicate).

The current log system needs to be revisited. The first step would be to investigate with small tests to decide the best way to generate logs and then write the feature.

Later we will also introduce the concept of drivers so we can connect to services like elasticsearch. To keep in mind.

For the moment we have built-in functions for asserting, it would be great to add a way to inject custom assertion functions to Assert.

Add SSL support since Hyper doesn't provide an implementation by default. See hyper-tls.

The API should be a bit more more natural without all the boilerplate for passing expected headers. We keep the current way, for flexibility purpose, but we add the possibility to provide &str tuples :

.headers(vec![
   ("Content-Type", "application/json; charset=utf-8"),
   ("Content-Length", "42"),
])

It will be less verbose compared to :

.headers(contains(vec![
   (
      CONTENT_TYPE,
      HeaderValue::from_static("application/json; charset=utf-8"),
   ),
   (
      CONTENT_LENGTH, HeaderValue::from_static("15")
   )
]))

A book where we provide code examples, architecture documentation, code snippets, roadmap, ... some characteristics :

• User-friendly (code snippets for the different cases, ...)
• Technical documentation : I plan to not only cover http but also ssl, dns, gRPC, ... a good technical documentation will help to navigate into the different sections
  • Document the architecture (dsl, assertions, builder, ...)
  • Document the different assertions (regexes, different options, ...)
• Document the roadmap : so users can easily follow and understand what has been prioritized
• Document the philosophy behind API testing / e2e / synthetic monitoring / UX monitoring : so people can learn about it at the same time and check if the library answer their needs
• Shameless plug of Owl Duty : that's how I plan to fund this work, the book should have a section about it to be transparent and to help the project at the same time

Once besok/jsonpath-rust#40 is fixed, we will be able to extend json path assertions and check if

• the path exists
  • also check for false positives/negatives in other assertions. Fail with Unprocessable?
• the path contains (use JsonPathFinder)
• the path does not contain

See for some kind of high order function restricting these checks to a Value::Number - or maybe returns a failure with Unprocessable result?

• the value at path is less than, is greater than or between

We can now add more documentation in the rust book about http assertions and predicates. Since the log system isn't really stable we can keep the doc for another time.

The current effort is for users first. But I can see two other sections in the book:

• Extending Grillon (lib, framework)
• Contributing to Grillon

Todo in the future.

We should be able top check the json schema from a file.

Continue to improve the API so we can test a single header and provide an expression as second argument:

.header("content-type", is("application/json; charset=utf-8"))

this HOF format was first found with #25 - I have an ongoing implementation where I'm able to provide an expression in an expression. Here we are at a level before, where header doesn't take one Expression argument but a header key followed by an Expression.

We are following the principle of composition, this one evolved since the first versions and the code could become a bit messy due to consecutive API updates. Some refacto could be necessary.

Add a more convenient way to configure Grillon and reuse configurations. By passing a struct GrillonConfig to the associated function new. We need to keep a flexible configuration, the builder should let the users to override the configuration with functions.

difference is unmaintained

The author of the difference crate is unresponsive.

Maintained alternatives:

• dissimilar

• similar

• treediff

• diffus

See advisory page for additional details.

For the moment we have a couple of assertions with the Value type that represents a json value.

As we are separating the assertion logic from the DSL, it makes sense to identify the different types of json Value, that we can call json contexts:

• json body
• json path
• pure json value

For example the Equality trait can be implemented for Values coming from those different contexts. For debug/logging purposes it is essential to know the Part under test, arising from the context.

The corresponding parts should be:

/// A json value
#[strum(serialize = "json value")]
#[serde(rename = "json value")]
JsonValue,
/// The json body of an http response.
#[strum(serialize = "json body")]
#[serde(rename = "json body")]
JsonBody,
/// The json value of an http response at the given path.
#[strum(serialize = "json path")]
#[serde(rename = "json path")]
JsonPath,

The lib API should stay simple and cover these different examples:

• json!(1).eq(&json!(1)): the resulting assertion's part should be a JsonValue
• JsonBody(json!(1)).eq(&json!(1)): the resulting assertion's part should be a JsonBody
• JsonPath(json!(1)).eq(&json!(1)): the resulting assertion's part should be a JsonPath

This way we can easily wrap the json value into its context, or just fallback to a default JsonValue when we want to assert some json data.

See #9 - extend checks with json schema.

For the moment we only support StatusCode which is a bit restrictive. Ideally we should support different types :

enum Status {
    Code(StatusCode),
    Message(String),
}

This will allow us to also convert int to StatusCode.

Now this could be a turning point for the library. Just like Tower, we could have a library that is

protocol agnostic, but is designed around a request / response pattern

Just like for #10 add a built-in function for multipart response body in Response returning an Option<Multipart> struct.

While users can use their own http client and handle the http response before asserting, it is not really convenient.

The library should come with a way to configure cookies for the requests before assertions. Originally commented on Reddit. Give an update once done.

This issue is related to #37

A common need when testing json bodies is to create jsonpath assertions. We currently support two assertions is and is_not that can be used with the json_body function. For example json_body(is(r#"{"a":"b"}"#)).

Now for testing a subpart of the json we could implement the following assertion : json_body(jsonpath(selector, expected)). Let's take this json data as an example (based on jsonpath_rust doc) :

 {
  "shop": {
   "orders": [
      {"id": 1, "active": true},
      {"id": 2 },
      {"id": 3 },
      {"id": 4, "active": true}
    ]
  }
}

The selector to find all the orders by id with the field active is : $.shop.orders[?(@.active)].id. The result is [1,4]. Based on this example, our assertion would be : json_body(jsonpath("$.shop.orders[?(@.active)].id", vec![1,4])).

To support this feature we can use a Rust library. We have several candidates :

• jsonpath_rust
• jsonpath
• jsonpath_lib

List of tasks

• Add assertion is
• Add assertion is_not
• #40
• Add assertion contains
• Add relevant doc in the book

Add an additional assertion for the response time.

Just like the body, we will need different predicates. Not all predicates are compatible with the functions. For example Predicate::Matches for the body assertion doesn't make sense to check the response time. We will need to apply different predicate scopes, for example a predicate module :

enum TimePredicate {
    LessThan(i32),
    MoreThan(i32),
    Between(i32, i32)
}

Connected to #17

Provide blanket implementations of the Response trait for reqwest, and replace Hyper client which is a legacy implementation now (following the release of v1).

Then update the API with BC to rename Response to Assert and struct Assert to something more meaningful since it's a wrapper over the http response to perform assertions. See how to manage base url for tests with the same instance.

Then subsequently add optional blanket implementations with feature flags for other clients.

I'm looking for any type of feedback on the project. This could be about the API, the code, the documentation, or just to know if you use it and in which context. As I'm mostly working alone on this project, I could use your feedback to better shape the lib!

Thanks!

For the moment grillon only supports json bodies, Extend the built-in functions with an html function in Response that returns an Option<String> (or a struct allowing to parse html content maybe).

The current version of the crate offers the main http methods, it was part of my MVP.

Now that the 0.1.0 version is out, it would be great to cover OPTIONS, CONNECT, HEAD, http methods.

TRACE will be implemented if there is a specific need, but for the moment I don't see any needs. And users can still call http_request(&self, method: Method, path: &str) -> Request on an instance of Grillon.

Offer a way to manage credentials. Ideally it should be generic enough to be used across protocols and for different credential types per protocol (ex http : oauth, basic auth, API keys).

For the moment we're only supporting a BodyExactMatcher which is not really convenient. Ideally we would like to match with different predicates :

• is
• is not
• contains
• does not contain
• matches (Regex)
• does not match (Regex)
• jsonpath (json path expression)

Example of API usage :

• body(Predicate::Is, json!({"id": 101})
• body(Predicate::Is, r#"{"id": 101}"#)
• body(Predicate::JsonPath, "$.store.book[0].title")
• body(Predicate::Matches, Regex::new(r"[\pN\p{Greek}\p{Cherokee}]+"))

is there a way to access the body json instead of doing an exact assert on it?

in my case I need to assert only on some parts and read others to go on with my test scenario

Add a new assertion json_schema. Naively we can think about three different data sources to provide in this assertion:

• A file path to load the schema json file from
• A json formatted string (and validated by serde_json behind the scene)
• A serde_json::Value (via json! macro or not)

The possibilities will depend on the Rust libraries. We have two of them:

• Valico
• jsonschema-rs

Both use the json schema test suite that's why I only linked those.

This issue will cover the assertion with raw json schema for the body part. Json path support and file support will be implemented in separate issues.

The jsonschema-rs lib offers an API easy to work with so we will go for this one.

• Implement schema assertion for json
  • unit tests for assertion impl
  • unit tests for Predicate serde
• Implement schema DSL
  • Value type
  • str type
  • String type
• Implement logs for both valid and invalid schema: see how to manage json pointers and errors
• Acceptance tests for the API of the schema assertion
• API doc
• Book updated
• README updated

Even in work in progress, a first version will encourage the documentation process itself. By adding the first stones we enable people to write doc and we enforce some good practices.

• Set CI
• Use Github pages
  Use owlduty custom domain : grillon.owlduty.com
• Provide a first documentation like a getting started