scalacheck case clases fluent experience about jqwik HOT 11 CLOSED

Just my first experiment, which I am mostly satisfied. My test class (mimicing fields from real application):

import lombok.Builder;
import lombok.Data;
import net.jqwik.api.Arbitraries;
import net.jqwik.api.Arbitrary;
import net.jqwik.api.Combinators;

import java.util.Random;

@Data
@Builder
public class UserForTests {
    private static final Random rnd = new Random();

    public String name;
    public String email;
    public String password;

    public static UserForTestsBuilder aUser() {
        Arbitrary<UserForTestsBuilder> r = Combinators.combine(
                Arbitraries.strings().all().ofMinLength(1).ofMaxLength(42), // name
                Arbitraries.strings().alpha().numeric().ofMinLength(1).ofMaxLength(42), //first part of email
                Arbitraries.strings().alpha().numeric().ofMinLength(1).ofMaxLength(42), //part of email after '@'
                Arbitraries.strings().alpha().ofMinLength(2).ofMaxLength(5), // domain of email (suffix after '.')
                Arbitraries.strings().all().ofMinLength(1).ofMaxLength(42) // password
        ).as(
                (name, email1, email2, emailDomain, pwd) -> new UserForTestsBuilder()
                        .name(name)
                        .email(email1 + '@' + email2 + '.' + emailDomain)
                        .password(pwd)
        );
        return r.generator(0).next(rnd).value();
    }
}

Than example test usage in integration (relatively slow) test:

        UserForTests a = UserForTests.aUser()
                .email("invalidEmail!")
                .build();
        SignUpAndLoginHelper.doSignUpRequest(this.webClient, a.name, a.email, a.password)
                .jsonPath("$.success").isBoolean()
                .jsonPath("$.success").isEqualTo(false);

This raises another question: Is there some more convenient way how to get single sample value? Using Arbitrary<UserForTestsBuilder>.generator(0).next(rnd).value() does:

• need genSize in generator method (through my UserForTestsBuilder does not use size anyhow,
• need Random instance (in next method),
• and need to call value(), to get actual value instead of Shrinkable

I would expect Arbitrary<UserForTestsBuilder>.getSample() to return single random UserForTestsBuilder instance. Does it cross competencies of Arbitrary interface? Do I miss something like some factory, which does take arbitrary and gives actual values?

PS: I know, that this is actually not property based testing. I will use PBT approach in unit tests, but now I am writing a bunch of integration tests and I am bored using static, manually crafted, values.

from jqwik.

I think what you're asking for is to use jqwik's generators as a replacement for test data builders (the pattern described in the famous GOOS book).

I've been thinking about that use case for quite a while. What I'm struggling with is coming up with a good (enough) idea to formulate and implement fixed value setting - like your email(...)- in a generic way. I assume that the implementation of UserForTestsBuilder is highly specific and that bugs me in a way. But maybe you have an idea?

There are two low hanging improvements that could be made to simplify your use case:

• Arbitraries.forType(...) could have a variant that descends recursively for types that cannot be provided by other generators. I have that somewhere in my backlog but no one has ever asked for it - yet.
• As you suggest there could be a convenience method Arbitrary.sample() for generating individual instances. The open question is from where to take the values for genSize and random.
  • genSize can have a strong influence on how generators distort pure randomness;
    for individual instances that might not play a siginificant role.
  • The actual random object however is important for the ability to recreate a test run
    with the exact same values when a property fails. There is an internal way to get hold
    of a single property trie's random object but I've been hesitant to make that coupling.
    You may be able to convince me, though, to implement sample() ;-)

from jqwik.

@luvarqpp
Just noticed that @Builder is provided by Lombok. So, would a recursive Arbitraries.forType() and a Arbitrary.sample() method fully satisfy your needs?

from jqwik.

Hello, thanks for reply. After a while of reasoning:

• I do not need (for now), recurse implementation of Arbitraties.forType() method, through it seems to be handy for many inputs that projects usually needs (invoice with items and other tree like structures)
• Arbitrary.sample(), I would like to have Arbitrary.samples(), because of usage in parametrized jupiter tests can be nice (see our comments here #41 (comment) ).
  • genSize can be requested as parameter for sample method if no other solution is good. I would like also your expertise about using some clean descendant function to generate genSize value for samples (i.e. first sample will get some higher genSize and all subsequent samples will have genSize lower till its minimum value). This can be further tuned if requested some number of samples. This way, samples can contain larger variance of generated values. Another question is, if genSize should start at some higher value, or it should start from minimum. Starting on minimal value could generate "easier" (~shrinked) values at the start. It does make fail the test earlier (if testing using for cycle :)
  • random value is harder, as random are hard in general (dilbert, xkcd and so on). I would expect to take take some thread local random generator. I expect that tests run by jupiter engine will be executed in parallel (according settings and way, how tests are written). In scala I would probably use some implicit (parameter in sample() method), but in java world, it would be tricky to have "good enough" solution. I expect that this sample[s]{0,1} method will be used to:
    • generate some samples just for slides and presentation -> (temporary generated Random instance would be enough)
    • generate some random values for actual tests (unit || integration || whatever) -> there are at least two needs:
      • determinism (be able to generate same random instances, i.e. using logged seed) => we need to have single instance of Random used here and it need to be used exclusively in+by our samples method (at least in same order and with same number of generated samples during single test)
      • generate different random numbers in subsequent runs (i.e. new Random(42) would not be ok)
    • generate some random instances, to actually test generator (arbitrary). I.e. if I am generating expected values. This can be turned out actually to be an unit test, which will check that all generated instances are valid and perhaps, that all edge cases have some examples there (labels, checkCoverage in second part of video of John Hughes on lambda days 2019, youtube) -> probably same needs for Random instance as previous item list

What do you think? Will use of newly created Random generator instance directly in Arbitraries.samples() method work? Is there any "store" in jupiter lifecycle, where we can store single Random instance for single test execution? Something like @BeforeEachMethod place to initialize and store during test, to be used for all Arbitraries.samples() calls, one-by-one.

from jqwik.

@luvarqpp

As far as I know there is no persisten store in Jupiter's lifecycle so you'd have to do that on your own - jqwik writes a file.

For coverage checking see #75

from jqwik.

I added a first implementation of Arbitrary.sample() and Arbitrary.sampleStream() in 1.2.2-SNAPSHOT.

Here's the newly added part in the user guide: https://jqwik.net/docs/snapshot/user-guide.html#using-arbitraries-directly

There's currently no way to supply your own Random instance. genSize can be overridden using Arbitrary.fixGenSize(genSize) if needed.

I am happy to take feedback.

from jqwik.

I am on it. I will try and put here my opinion after some time of using this thing. Just after first use, it seems fine.
PS: Snapshot repo is on url https://oss.sonatype.org/content/repositories/snapshots/ , as stated in documentation.

from jqwik.

I see no issues till now with Arbitrary.sample() method. I am happy with its functionality, through javadoc could have mention also consequences (if any from last discussion still apply) and intended usage (to provide sample values in integration tests, as a replacement of test data builders, as generator of prepared Builder, e.t.c.).

From my point, this issue is closeable. Thanks a lot for fast response and project improvement.

from jqwik.

I added a few points to the Javadoc: https://github.com/jlink/jqwik/blob/master/api/src/main/java/net/jqwik/api/Arbitrary.java#L356-L375

Does it cover your concerns?

from jqwik.

I added a few points to the Javadoc: https://github.com/jlink/jqwik/blob/master/api/src/main/java/net/jqwik/api/Arbitrary.java#L356-L375

Does it cover your concerns?

Absolutely.

Should I close this issue, or do you have some established workflow? Imho this is done.

from jqwik.

@luvarqpp Thanks for all the input.

from jqwik.