Giter Club home page Giter Club logo

fast-check's Introduction

fast-check logo

Property based testing framework for JavaScript/TypeScript

Build Status npm version monthly downloads

Coverage Status (unit tests) Package quality Snyk Package quality OpenSSF Scorecard OpenSSF Best Practices

PRs Welcome License Twitter

Getting started

Hands-on tutorial and definition of Property Based Testing: 🏁 see tutorial. Or directly try it online on our pre-configured CodeSandbox.

Property based testing frameworks check the truthfulness of properties. A property is a statement like: for all (x, y, ...) such that precondition(x, y, ...) holds predicate(x, y, ...) is true.

Install the module with: yarn add fast-check --dev or npm install fast-check --save-dev

Example of integration in mocha:

import fc from 'fast-check';

// Code under test
const contains = (text, pattern) => text.indexOf(pattern) >= 0;

// Properties
describe('properties', () => {
  // string text always contains itself
  it('should always contain itself', () => {
    fc.assert(fc.property(fc.string(), (text) => contains(text, text)));
  });
  // string a + b + c always contains b, whatever the values of a, b and c
  it('should always contain its substrings', () => {
    fc.assert(
      fc.property(fc.string(), fc.string(), fc.string(), (a, b, c) => {
        // Alternatively: no return statement and direct usage of expect or assert
        return contains(a + b + c, b);
      }),
    );
  });
});

In case of failure, the test raises a red flag. Its output should help you to diagnose what went wrong in your implementation. Example with a failing implementation of contain:

1) should always contain its substrings
    Error: Property failed after 1 tests (seed: 1527422598337, path: 0:0): ["","",""]
    Shrunk 1 time(s)
    Got error: Property failed by returning false

    Hint: Enable verbose mode in order to have the list of all failing values encountered during the run

Integration with other test frameworks: ava, jasmine, jest, mocha and tape.

More examples: simple examples, fuzzing and against various algorithms.

Useful documentations:

Why should I migrate to fast-check?

fast-check has initially been designed in an attempt to cope with limitations I encountered while using other property based testing frameworks designed for JavaScript:

  • Types: strong and up-to-date types - thanks to TypeScript
  • Extendable: easy map method to derive existing arbitraries while keeping shrink [more] - some frameworks ask the user to provide both a->b and b->a mappings in order to keep a shrinker
  • Extendable: kind of flatMap-operation called chain [more] - able to bind the output of an arbitrary as input of another one while keeping the shrink working
  • Extendable: precondition checks with fc.pre(...) [more] - filtering invalid entries can be done directly inside the check function if needed
  • Extendable: easily switch from fake data in tests to property based with fc.gen() [more] - generate random values within your predicates
  • Smart: ability to shrink on fc.oneof [more] - surprisingly some frameworks don't
  • Smart: biased by default - by default it generates both small and large values, making it easier to dig into counterexamples without having to tweak a size parameter manually
  • Debug: verbose mode [more][tutorial] - easier troubleshooting with verbose mode enabled
  • Debug: replay directly on the minimal counterexample [tutorial] - no need to replay the whole sequence, you get directly the counterexample
  • Debug: custom examples in addition of generated ones [more] - no need to duplicate the code to play the property on custom examples
  • Debug: logger per predicate run [more] - simplify your troubleshoot with fc.context and its logging feature
  • Unique: model based approach [more][article] - use the power of property based testing to test UI, APIs or state machines
  • Unique: detect race conditions in your code [more][tutorial] - shuffle the way your promises and async calls resolve using the power of property based testing to detect races
  • Unique: simplify user definable corner cases [more] - simplify bug resolution by asking fast-check if it can find an even simpler corner case

For more details, refer to the documentation in the links above.

Trusted

fast-check has been trusted for years by big projects like: jest, jasmine, fp-ts, io-ts, ramda, js-yaml, query-string...

Powerful

It also proved useful in finding bugs among major open source projects such as jest, query-string... and many others.

Compatibility

Here are the minimal requirements to use fast-check properly without any polyfills:

fast-check node ECMAScript version TypeScript (optional)
3.x β‰₯8(1) ES2017 β‰₯4.1(2)
2.x β‰₯8(1) ES2017 β‰₯3.2(3)
1.x β‰₯0.12(1) ES3 β‰₯3.0(3)
More details...
  1. Except for features that cannot be polyfilled - such as bigint-related ones - all the capabilities of fast-check should be usable given you use at least the minimal recommended version of node associated to your major of fast-check.
  2. Require either lib or target β‰₯ ES2020 or @types/node to be installed.
  3. Require either lib or target β‰₯ ES2015 or @types/node to be installed.

ReScript bindings

Bindings to use fast-check in ReScript are available in package rescript-fast-check. They are maintained by @TheSpyder as an external project.

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Nicolas DUBIEN
Nicolas DUBIEN

πŸ’» πŸ“– ⚠️ πŸš‡ 🎨 🚧
Aaron Elligsen
Aaron Elligsen

πŸ’» πŸ“– ⚠️
Will Heslam
Will Heslam

πŸ“–
kazchimo
kazchimo

πŸ’» πŸ“–
Brandon Chinn
Brandon Chinn

πŸ’» πŸ“–
Irakli Safareli
Irakli Safareli

πŸ“–
Andrew Herron
Andrew Herron

πŸ“– πŸ”Œ
Eric Crosson
Eric Crosson

πŸ“– πŸ’»
burrscurr
burrscurr

πŸ“–
JC (Jonathan Chen)
JC (Jonathan Chen)

πŸ“–
Larry Botha
Larry Botha

πŸ“– πŸ’» ⚠️
Roman Gusev
Roman Gusev

πŸ“–
Tim Wisniewski
Tim Wisniewski

πŸ“–
Brais PiΓ±eiro
Brais PiΓ±eiro

πŸ’» ⚠️
Renaud-Pierre Bordes
Renaud-Pierre Bordes

🎨
Jemma Nelson
Jemma Nelson

πŸ“–
John Haugeland
John Haugeland

πŸ“–
Trey Davis
Trey Davis

🎨
Leon Si
Leon Si

πŸ“–
Gorgi Kosev
Gorgi Kosev

πŸš‡
mayconsacht
mayconsacht

πŸ’»
Simon Friis Vindum
Simon Friis Vindum

πŸ’» ⚠️
Richard Gibson
Richard Gibson

πŸ“–
Alan Harper
Alan Harper

πŸ“–
Makien Osman
Makien Osman

πŸ’»
David Sommerich
David Sommerich

πŸ’» ⚠️
Diego Pedro
Diego Pedro

πŸ’» ⚠️
Borui Gu
Borui Gu

πŸ“–
Brian Donovan
Brian Donovan

πŸ“–
volrk
volrk

πŸ’» πŸ“– ⚠️
tinydylan
tinydylan

πŸ’» ⚠️
Caleb Jasik
Caleb Jasik

πŸ“–
Rulai Hu
Rulai Hu

πŸ“–
Afonso Jorge Ramos
Afonso Jorge Ramos

πŸ“–
Tom Jenkinson
Tom Jenkinson

πŸ“–
phormio
phormio

πŸ“–
Giovanni Gonzaga
Giovanni Gonzaga

πŸ’» ⚠️
Tomas Carnecky
Tomas Carnecky

πŸ’»
Kirill Romanov
Kirill Romanov

πŸ’» πŸ“– ⚠️
Giovanny GonzΓ‘lez
Giovanny GonzΓ‘lez

πŸ“–
Mark Kulube
Mark Kulube

πŸš‡
Peter Hamilton
Peter Hamilton

πŸ’»
Chinedu Ozodi
Chinedu Ozodi

πŸ“–
Gunar Gessner
Gunar Gessner

πŸ“–
Christian Batchelor
Christian Batchelor

⚠️
Tomer Aberbach
Tomer Aberbach

πŸ’» πŸ“– ⚠️
0xflotus
0xflotus

πŸ“–
Ryan Leonard
Ryan Leonard

πŸ’» πŸ“– ⚠️
Jason Dreyzehner
Jason Dreyzehner

πŸ’» ⚠️
Matin Zadeh Dolatabad
Matin Zadeh Dolatabad

πŸ’»
Juan JuliΓ‘n Merelo GuervΓ³s
Juan JuliΓ‘n Merelo GuervΓ³s

πŸ“–
Simen Bekkhus
Simen Bekkhus

πŸ“–
Tarjei Skjærset
Tarjei Skjærset

πŸ“–
Denis Gorbachev
Denis Gorbachev

πŸ“–
Trevor McCauley
Trevor McCauley

πŸ“–
Grant Kiely
Grant Kiely

πŸ“–
Attila Večerek
Attila Večerek

πŸ’» πŸ“– ⚠️
Zach Bjornson
Zach Bjornson

πŸ’» πŸ“–
Bennett Perkins
Bennett Perkins

πŸ“–
Alexandre Oger
Alexandre Oger

πŸ“–
ej shafran
ej shafran

πŸ“–
Niklas Gruhn
Niklas Gruhn

πŸ’»
Patrick Roza
Patrick Roza

πŸ’»

This project follows the all-contributors specification. Contributions of any kind welcome! Become one of them

Sponsors πŸ’Έ

Many individuals and companies offer their financial support to the project, a huge thanks to all of them too πŸ’“

all sponsors

You can also become one of them by contributing via GitHub Sponsors or OpenCollective.

fast-check's People

Contributors

0xflotus avatar 102 avatar brandon-leapyear avatar brds avatar dependabot[bot] avatar dubzzz avatar ericcrosson avatar gibson042 avatar github-actions[bot] avatar gruhn avatar hamiltop avatar hath995 avatar kazchimo avatar makeeno avatar matinzd avatar patroza avatar renovate[bot] avatar rulai-hu avatar safareli avatar senocular avatar simenb avatar thespyder avatar timwis avatar tjenkinson avatar tomeraberbach avatar treydavis avatar trollepierre avatar tskj avatar werehamster avatar willheslam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

fast-check's Issues

Issue in StringArbitrary :: unicodeString :: Should generate a string given maximal length

  1. StringArbitrary
    unicodeString
    Should generate a string given maximal length:
    Error: Failed after 61 tests and 1 shrinks. rngState: 82a63bce94017f8d52; Counterexample: 0; 0;
    at node_modules\jsverify\lib\jsverify.js:360:15
    at Object.map (node_modules\jsverify\lib\functor.js:35:12)
    at Object.checkThrow [as assert] (node_modules\jsverify\lib\jsverify.js:354:30)
    at Context.it (test\check\arbitrary\StringArbitrary.spec.ts:48:71)

Better shrinking tests

Better unit testing for shrink method. It should try all possible paths and test for:

  • no previously seen entry
  • strictly inferior
  • convergence whatever the path

Long run for ArrayArbitrary

The test:

ArrayArbitrary (seed: 1514073813156) :: Should shrink removing unecessary entries in the array

took too long to run.

It might prove a code issue that need to be fixed.
Analysis is needed.

Classify produced inputs

Add an option or method to be able to classify built randoms in order to evaluate the value of a generator

JSON Arbitrary

Based on the previously defined Object Arbitrary (with no undefined as it should be a valid JSON), it should generate JSON strings.

Arbitrary.then

Should ease the creation of new Arbitraries entirely based on a previous one.

For eg.: An Arbitrary building a range and one integer within this range can be built that way (or just [current way] by taking 3 integers and reordering them)

Invalid convergence of ArrayArbitrary

To be analysed:

  2) ArrayArbitrary (seed: 1514074620775)
       array
         Should shrink removing unecessary entries in the array:

      AssertionError [ERR_ASSERTION]: Should shrink to counterexample [5,5]
      + expected - actual

       [
         [
           5
      -    0
           5
         ]
       ]

Add option

For the moment, shrinking an enum keep playing on the initial generator. For option, we might be interested to converge to null state. We might extend it with undefined.

Arbitrary.noShrink

Add a noShrink method to Arbitrary base class in order to be able to idle a shrinker

Object Arbitrary

Think about the possibility of putting null or undefined in the generated dictionaries

Any/Anything Arbitrary

Just calling one of the others generators randomly:
Integer, String, Array of anything, Array of a precise type, Object, Double

Implement async/await compatibility

Need to find a non intrusive way to add support for async/await while keeping the ability to build non async/await tests (would be a nice option).

Reduce flakiness of generate all values tests

As soon as the use of pure-rand is effective, move tests checking that all values can be generated to e2e part. Then opt for a one seed test per run.

Basically the test should take a random seed and confirm it can generate all the possible values of the arbitrary under tests.

Dictionary Arbitrary

Given a key -> Arbitrary, it should be able to build a Dictionary by building its values from the given generatirs

Add Stream.getNthOrLast

This utility can be very useful for testing purposes. It should help unlocking better shrinking tests going through the whole tree and not just straightforward

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.