leanovate / gopter Goto Github PK
View Code? Open in Web Editor NEWGOlang Property TestER
License: MIT License
GOlang Property TestER
License: MIT License
Looking for clarification on how a complex state is supposed to be handled along with creation and destruction of a SUT.
For example, testing a circular buffer you could have a container/list mirror the contents of the circular buffer to verify behaviour in a state structure. However, I do not see how to correlate re-initializing the state when the SUT is destroyed and recreated. Is there something I am missing?
Is there any interest in supporting generics in gopter?
I'm willing to start taking a stab at it, but I want to make sure the work gets reviewed by someone.
Release 0.1.0 was made in April 2016. There have been changes since that would benefit users. With the increasing prevalence of dep
, the 0.1.0
release will be chosen by default. We should make a new release for dep
to use.
when you have a generator that looks like this:
genStruct({
a: genSlice(genStruct({
b: genString().SuchThat(...SomeCondition),
c: genString().SuchThat(...SomeOtherCondition)
}))
})
An index out of bounds exception is thrown here:
Line 67 in cdb2f0e
it happens because for some reason baseSieve
only has a single value (SomeCondition
) but ups
has two values (b
and c
) in it. So when you loop over ups
and grab the corresponding baseSieve
item for the up
(at i
) you get that panic.
This doesn't happen on the latest release but does happen on master.
minimal example:
type TestBook struct {
Title string
Content string
}
func genTestBook() gopter.Gen {
return gen.Struct(reflect.TypeOf(&TestBook{}), map[string]gopter.Gen{
"Title": gen.AlphaString().SuchThat(func(s string) bool {
fmt.Println("NonEmptyString1")
return s != ""
}),
"Content": gen.AlphaString().SuchThat(func(s string) bool {
fmt.Println("NonEmptyString2")
return s != ""
}),
})
}
type TestLibrary struct {
Books []TestBook
}
func genTestLibrary() gopter.Gen {
return gen.Struct(reflect.TypeOf(&TestLibrary{}), map[string]gopter.Gen{
"Books": gen.SliceOf(genTestBook()),
})
}
This should probably block #60
Hi @untoldwind,
We've recently started to use gopter (which is a great project thanks) and came across a bug when using multiple workers and a failure occurred it wouldn't return the full test result error, labels or prop args, making hard to understand what generated value caused the test failure. Which I raised a new PR for review: #36
However that exposed another bug unfortunately with math/rand. Essentially rand.New(rand.NewSource(..)) is not thread safe when calling for new rand Int - running across multiple workers and you can validate with this smallest example - it doesn't happen consistently but happens regularly
https://gist.github.com/jnormington/a832472edf9fa0db020ebd028b7849b9
Issues on golang;
I've looked at this and there are a few solutions;
There are a few problems with 1 & 2 though
We've updated our vendored version and works great.
So I wanted to get your thoughts on the issue and whether you had any ideas or was happy with this approach or if you had other concerns I haven't thought of. Below is an example of point 3
diff --git a/gen_parameters.go b/gen_parameters.go
index aca147c..a146b10 100644
--- a/gen_parameters.go
+++ b/gen_parameters.go
@@ -51,7 +51,7 @@ func (p *GenParameters) CloneWithSeed(seed int64) *GenParameters {
MinSize: p.MinSize,
MaxSize: p.MaxSize,
MaxShrinkCount: p.MaxShrinkCount,
- Rng: rand.New(rand.NewSource(seed)),
+ Rng: rand.New(NewLockedSource(seed)),
}
}
@@ -63,6 +63,6 @@ func DefaultGenParameters() *GenParameters {
MinSize: 0,
MaxSize: 100,
MaxShrinkCount: 1000,
- Rng: rand.New(rand.NewSource(seed)),
+ Rng: rand.New(NewLockedSource(seed)),
}
}
diff --git a/rand_source.go b/rand_source.go
new file mode 100644
index 0000000..34f5241
--- /dev/null
+++ b/rand_source.go
@@ -0,0 +1,77 @@
+// Copyright 2009 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+// Taken from golang lockedSource implementation https://github.com/golang/go/blob/master/src/math/rand/rand.go#L371-L410
+
+package gopter
+
+import (
+ "math/rand"
+ "sync"
+)
+
+type lockedSource struct {
+ lk sync.Mutex
+ src rand.Source64
+}
+
+// NewLockedSource takes a seed and returns a new
+// lockedSource for use with rand.New
+func NewLockedSource(seed int64) *lockedSource {
+ return &lockedSource{
+ src: rand.NewSource(seed).(rand.Source64),
+ }
+}
+
+func (r *lockedSource) Int63() (n int64) {
+ r.lk.Lock()
+ n = r.src.Int63()
+ r.lk.Unlock()
+ return
+}
+
+func (r *lockedSource) Uint64() (n uint64) {
+ r.lk.Lock()
+ n = r.src.Uint64()
+ r.lk.Unlock()
+ return
+}
+
+func (r *lockedSource) Seed(seed int64) {
+ r.lk.Lock()
+ r.src.Seed(seed)
+ r.lk.Unlock()
+}
+
+// seedPos implements Seed for a lockedSource without a race condition.
+func (r *lockedSource) seedPos(seed int64, readPos *int8) {
+ r.lk.Lock()
+ r.src.Seed(seed)
+ *readPos = 0
+ r.lk.Unlock()
+}
+
+// read implements Read for a lockedSource without a race condition.
+func (r *lockedSource) read(p []byte, readVal *int64, readPos *int8) (n int, err error) {
+ r.lk.Lock()
+ n, err = read(p, r.src.Int63, readVal, readPos)
+ r.lk.Unlock()
+ return
+}
+
+func read(p []byte, int63 func() int64, readVal *int64, readPos *int8) (n int, err error) {
+ pos := *readPos
+ val := *readVal
+ for n = 0; n < len(p); n++ {
+ if pos == 0 {
+ val = int63()
+ pos = 7
+ }
+ p[n] = byte(val)
+ val >>= 8
+ pos--
+ }
+ *readPos = pos
+ *readVal = val
+ return
+}
I have a use case where I want to generate strings with a length between n and m. Currently I am doing this:
makeStringGen := func(min, max int) gopter.Gen {
return gen.Sized(func(size int) gopter.Gen {
return gen.SliceOfN(size%max+1, gen.Rune()).
SuchThat(func(rs []rune) bool { return len(rs) >= min }).
Map(func(v []rune) string { return string(v) })
})
}
but I imagine that using % would probably mess with the distribution of sizes and using SuchThat is fine in my use case since the min is generally 0 or 1 but it seems suboptimal as a general solution. What's the best way of doing this sort of thing?
Property based tests are great tools to assert properties of an API. Though I haven't done it in Go, this package looks useful.
I'm more curious about the implementation and the algorithm. Do you know any references or papers I could read?
It seems to me this library has less powerful "generator inference" than the simple testing/quick
one, which has no problem with this case or with #27
Given this input program
package main
import (
"github.com/leanovate/gopter"
"github.com/leanovate/gopter/arbitrary"
)
type MyIDType int32
type Foo struct {
Name string
Id MyIDType
}
func main() {
parameters := gopter.DefaultTestParameters()
parameters.Rng.Seed(1234)
arbitraries := arbitrary.DefaultArbitraries()
properties := gopter.NewProperties(parameters)
properties.Property("Foo", arbitraries.ForAll(
func(foo *Foo) bool {
return true
}))
properties.Run(gopter.ConsoleReporter(false))
}
Output:
! Foo: Error on property evaluation after 0 passed tests: Check paniced:
reflect.Set: value of type int32 is not assignable to type main.MyIDType
goroutine 1 [running]:
runtime/debug.Stack(0xc42004b788, 0x4f0980, 0xc42007e4d0)
/usr/x86_64-pc-linux-gnu/lib/go/src/runtime/debug/stack.go:24 +0xa7
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.Save
Prop.func1.1(0xc42004bd28)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/prop.go:19 +0x6e
panic(0x4f0980, 0xc42007e4d0)
/usr/x86_64-pc-linux-gnu/lib/go/src/runtime/panic.go:505 +0x229
reflect.Value.assignTo(0x4f0340, 0xc42008bdf8, 0x85, 0x521cbc, 0xb,
0x4f0400, 0x0, 0x4f0340, 0x4f0340, 0xc42008bdf8)
/usr/x86_64-pc-linux-gnu/lib/go/src/reflect/value.go:2235 +0x427
reflect.Value.Set(0x4f0400, 0xc4200bc4d0, 0x185, 0x4f0340, 0xc42008bdf8,
0x85)
/usr/x86_64-pc-linux-gnu/lib/go/src/reflect/value.go:1373 +0xa4
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter/gen.
StructPtr.func1(0xc4200bc480, 0x1)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/gen/struct.go:64 +0x327
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter/prop
.ForAll.func1(0xc4200bc480, 0x529668)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/prop/forall.go:31 +0x140
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.Save
Prop.func1(0xc4200bc480, 0x0)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/prop.go:24 +0x6c
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.Prop
.Check.func1(0x0, 0xc42007e470, 0x5d4f60)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/prop.go:52 +0x15c
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.(*ru
nner).runWorkers(0xc420082720, 0x0)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/runner.go:45 +0x2d9
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.Prop
.Check(0xc42007e450, 0xc4200a05c0, 0x520b0c)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/prop.go:110 +0x1af
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.(*Pr
operties).Run(0xc4200822a0, 0x538c20, 0xc4200bc440, 0x3)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/properties.go:37 +0xc9
main.main()
/home/julian/go/src/github.com/hasufell/gopter-examples/gopterexamples.go:
28 +0x19d
Hi,
I really like gopter so far. Thanks for providing such a useful library. I've just run into something unexpected with stateful testing though.
What I want is after each operation to not only compare the Result of the SystemUnderTest
command with the model state (State
) as in Postcondition
, but also to ensure that the internal state of the SystemUnderTest
is logically equivalent to the model state. I tried including a pointer to the SystemUnderTest
in State
so I could just perform this check in Postcondition
, but it appears that by the time Postcondition runs, the pointer is pointing to a different value than what was initialized and returned from NewSystemUnderTest
.
The rationale for this feature is to be able to test mutable methods on types that don't return values. So far I think the only workaround is to provide some introspection methods that we add as commands so we can then run and compare the output to the model state.
I was wondering if you could provide this capability, via another callback like invariant/1
in eqc_statem, or if there is a specific reason you think it should not be included. And yes, effort is a fine justification :) Thanks!
It's important that generators produce the same value for the same GenParameters (same seed, etc.) The enumeration of Go maps is non-deterministic, in fact the Go designers have gone out of their way to make it random! This means that the values produced by the Gens in the map passed to gen.Struct are most likely to be different from one call to the next given the same PRNG.
I am looking for a way to generate values which depend on each other. In particular, I want to recreate the following simple example from ScalaCheck's User Guide (https://github.com/typelevel/scalacheck/blob/main/doc/UserGuide.md#generators), where two integer are generated, the first in the range of 10..20
, the second has a lower bound which is twice as large as the first value:
// ScalaCheck Example
val myGen = for {
n <- Gen.choose(10,20)
m <- Gen.choose(2*n, 500)
} yield (n,m)
My impression was that the gen.FlatMap()
should provide the required functionality (in Scala, <-
is a monadic assignment, implemented by FlatMap
), but I failed to find a way to succeed.
I defined a simple struct to generate two values which can be fed into the property:
type IntPair struct {
Fst int
Snd int
}
properties.Property("ScalaCheck example for a pair", prop.ForAll(
func(p IntPair) bool {
a := p.Fst
b := p.Snd
return a*2 <= b
},
genIntPairScala(),
))
The generator is a straight translation of the Scala code, first generating an integer and then generating a second via accessing the generated value of the first. Both generators are finally stored in the struct generator:
genIntPairScala := func() gopter.Gen {
n := gen.IntRange(10, 20).WithLabel("n (fst)")
m := n.FlatMap(func(v interface{}) gopter.Gen {
k := v.(int)
return gen.IntRange(2*k, 50)
}, reflect.TypeOf(int(0))).WithLabel("m (snd)")
var gen_map = map[string]gopter.Gen{"Fst": n, "Snd": m}
return gen.Struct(
reflect.TypeOf(IntPair{}),
gen_map,
)
}
However, it does not work:
=== RUN TestGopterGenerators
! ScalaCheck example for a pair: Falsified after 10 passed tests.
n (fst), m (snd): {Fst:17 Snd:32}
n (fst), m (snd)_ORIGINAL (1 shrinks): {Fst:19 Snd:32}
Elapsed time: 233.121ยตs
properties.go:57: failed with initial seed: 1617636578517672000
Remark: I set the upper bound to 50
instead of 500
. The property must still hold, but the generator has a smaller pool to pick suitable values: setting the upper bound to 500
often results in a passing property!
Gopter is a great tool for exploring a set of possible codepaths and discovering bugs. However, if I happen to find a bug in my code that comes from an unusual codepath, I would want to capture the sequence of operations that exposed that bug, and quickly turn them into a regression test to live in my suite independent of my normal Gopter tests. In other words, I want to be able to have my Gopter tests run, and then write regression tests in terms of Gopter.
E.g. Let's say my Gopter test for my simple counter fails with the sequence of events
initialState: {0}
sequential=[INCREMENT INCREMENT RESET DECREMENT]
I want some way to write that sequence of commands into a new test that will essentially run those exact commands within the Gopter framework, the same way I would expect them to run when running a more exploratory Gopter test.
Is there any way to leverage the current library to do this?
In the docs I can see that GenParameters.WithSize() can be used to control the maximum length of string arbitraries, but I can not figure out how to use it to limit the length of strings generated by gen.AlphaString()
, gen.AnyString
, gen.NumString()
, etc.
In the code they all call genString()
, which in turn calls SliceOf()
, which then makes use of genParams.Size
(from GenParameters), but genParams
gets passed in by some internal function and I do not know how to control this.
On the flip side I see that SliceOfN()
will allow an int parameter to specify the length, but my attempts to create a generator using it fails, as using the same map as genString()
wants a rune
and not []rune
.
First of all, I do appreciate all your work on this project. This library is fantastic. I was playing with gopter and found that the property doesn't seem to hold the original state of the sample for reporting, which gives confusing results.
First, I prepared a small target function named biggest
, which is expected to return the biggest value in the given slice of int.
func biggest(ns []int) (int, error) {
if len(ns) == 0 {
return 0, fmt.Errorf("slice must have at least one element")
}
return ns[0], nil
}
As the PBT of this function, I wrote the following test:
func TestBiggest(t *testing.T) {
properties := gopter.NewProperties(nil)
properties.Property("Non-zero length small int slice", prop.ForAll(
func(ns []int) bool {
result, _ := biggest(ns)
sort.Slice(ns, func(i, j int) bool {
return ns[i] > ns[j]
})
return result == ns[0]
},
gen.SliceOf(gen.IntRange(0, 20),
reflect.TypeOf(int(0))).
SuchThat(func(v interface{}) bool {
return len(v.([]int)) > 0
}),
))
properties.TestingRun(t)
}
The result of this test was:
! Non-zero length small int slice: Falsified after 0 passed tests.
ARG_0: [13 0]
ARG_0_ORIGINAL (1 shrinks): [14 13]
Elapsed time: 164.792ยตs
--- FAIL: TestBiggest (0.00s)
/home/ymotongpoo/personal/pbt/sample/properties.go:57: failed with initial seed: 1602330526004760703
FAIL
But these arguments, both the original and the shrunk, should pass the test. Though I haven't dug into the implementation of the default shrinker and reporting, I assumed that the shrinker doesn't hold the original state of the sample. I changed the test code a bit to keep the original state of sample and it returned the expected report.
func TestBiggest(t *testing.T) {
properties := gopter.NewProperties(nil)
properties.Property("Non-zero length small int slice", prop.ForAll(
func(ns []int) bool {
nns := make([]int, len(ns))
copy(nns, ns)
result, _ := biggest(nns)
sort.Slice(nns, func(i, j int) bool {
return nns[i] > nns[j]
})
return result == nns[0]
},
gen.SliceOf(gen.IntRange(0, 20),
reflect.TypeOf(int(0))).
SuchThat(func(v interface{}) bool {
return len(v.([]int)) > 0
}),
))
properties.TestingRun(t)
}
This test made the following report that makes sense to me:
! Non-zero length small int slice: Falsified after 0 passed tests.
ARG_0: [0 1]
ARG_0_ORIGINAL (6 shrinks): [3 19]
Elapsed time: 245.025ยตs
--- FAIL: TestBiggest (0.00s)
/home/ymotongpoo/personal/pbt/sample/properties.go:57: failed with initial seed: 1602333441977570032
FAIL
Sometimes we get a failure with the below code
Error on property evaluation after 1 passed
tests: Check paniced: reflect: Call using *schema.PathMatcherType_Prefix
as type *schema.PathMatcherType_Path
Could you please suggest if you see any issue ?
Code:
func genPathMatcherType() gopter.Gen {
return gen.StructPtr(reflect.TypeOf(&pb_schema.PathMatcherType{}), map[string]gopter.Gen{
"PathMatch": gen.OneGenOf(genPathMatcherType_Prefix(), genPathMatcherType_Path(), genPathMatcherType_Regex()),
})
}
func genPathMatcherType_Prefix() gopter.Gen {
return gen.StructPtr(reflect.TypeOf(&pb_schema.PathMatcherType_Prefix{}), map[string]gopter.Gen{
"Prefix": gen.AlphaString(),
})
}
func genPathMatcherType_Path() gopter.Gen {
return gen.StructPtr(reflect.TypeOf(&pb_schema.PathMatcherType_Path{}), map[string]gopter.Gen{
"Path": gen.AlphaString(),
})
}
func genPathMatcherType_Regex() gopter.Gen {
return gen.StructPtr(reflect.TypeOf(&pb_schema.PathMatcherType_Regex{}), map[string]gopter.Gen{
"Regex": gen.AlphaString(),
})
}
Hi folks,
I've got a test whose runtime is real slow and I wonder if I'm doing something very wrong. Or, if I'm not doing something wrong if there are any options I've got to improve the test runtime. Here's a representative example of my actual tests:
package main
import (
"github.com/leanovate/gopter"
"github.com/leanovate/gopter/arbitrary"
"github.com/leanovate/gopter/gen"
"reflect"
"testing"
)
type TestBook struct {
Title string
Content string
}
func genTestBook() gopter.Gen {
return gen.Struct(reflect.TypeOf(&TestBook{}), map[string]gopter.Gen{
"Title": gen.AlphaString(),
"Content": gen.AlphaString(),
})
}
type TestLibrary struct {
Name string
Librarians uint8
Books []TestBook
}
func genTestLibrary() gopter.Gen {
return gen.Struct(reflect.TypeOf(&TestLibrary{}), map[string]gopter.Gen{
"Name": gen.AlphaString(),
"Librarians": gen.UInt8Range(1, 255),
"Books": gen.SliceOf(genTestBook()),
})
}
type CityName = string
type TestCities struct {
Libraries map[CityName][]TestLibrary
}
func genTestCities() gopter.Gen {
return gen.StructPtr(reflect.TypeOf(&TestCities{}), map[string]gopter.Gen{
"Libraries": gen.MapOf(gen.AlphaString(), gen.SliceOf(genTestLibrary())),
})
}
func TestLibraries(t *testing.T) {
parameters := gopter.DefaultTestParameters()
parameters.MinSuccessfulTests = 10
parameters.MaxSize = 4
arbitraries := arbitrary.DefaultArbitraries()
arbitraries.RegisterGen(genTestCities())
properties := gopter.NewProperties(parameters)
properties.Property("no unsupervised libraries", arbitraries.ForAll(
func(tc *TestCities) bool {
for _, libraries := range tc.Libraries {
for _, library := range libraries {
if library.Librarians == 0 {
return false
}
}
}
return true
},
))
properties.TestingRun(t)
}
My test builds a tree structure -- TestCities
-> TestLibrary
-> TestBook
-- and then runs a property to assert that no libraries are unsupervised. By construction this property will always pass: genTestLibrary
does not allow for TestLibrary.Libarians
to be 0. As expected, the property does pass but the runtime is around 30 seconds on my host. An example:
> go test
+ no unsupervised libraries: OK, passed 10 tests.
Elapsed time: 356.71ยตs
PASS
ok _/Users/btroutwine/go/propexample 26.830s
Am I doing anything obviously wrong here?
I've been playing around with gopter
for a little while now, trying to understand how to write my own generator for my use case, which is not as straight forward as those in the repo. My use case is the following; I want to test a function (ReadQF
) that should return a value and true when enough, i.e. a quorum of replies have been received and passed in to the ReadQF
function. It should return false otherwise.
I've hacked together something that seems to work in the following:
https://github.com/relab/byzq/blob/master/authdataspec_property_test.go#L37
https://github.com/relab/byzq/blob/master/authdataspec_property_test.go#L63
However, I suspect it isn't quite in the spirit of property-based testing, and I'm struggling to break it up into multiple generators, since the input parameter n
to the NewAuthDataQ
constructor that creates a qspec
object and computes the parameter q
, which is used to decide the minimal/maximal length of the replies array. And furthermore, I need access to the qspec
object in the end to decide if a quorum has been received.
I would really appreciate to get some feedback on my two tests linked above, especially, if you can provide some advice on how to decouple things.
(Below is an initial attempt at writing a generator, but I don't know how to get both the quorumSize
and qspec
parameters out of the generator for consumption in the condition
function passed to prop.ForAll()
.)
func genQuorums(min, max int, hasQuorum bool) gopter.Gen {
return func(genParams *gopter.GenParameters) *gopter.GenResult {
rangeSize := uint64(max - min + 1)
n := int(uint64(min) + (genParams.NextUint64() % rangeSize))
qspec, err := NewAuthDataQ(n, priv, &priv.PublicKey)
if err != nil {
panic(err)
}
// initialize as non-quorum
minQuorum, maxQuorum := math.MinInt32, qspec.q
if hasQuorum {
minQuorum, maxQuorum = qspec.q+1, qspec.n
}
rangeSize = uint64(maxQuorum - minQuorum + 1)
quorumSize := int(uint64(minQuorum) + (genParams.NextUint64() % rangeSize))
genResult := gopter.NewGenResult(qspec, gopter.NoShrinker)
return genResult
}
}
A test I wrote using PtrOf sometimes results in a panic like this:
Parent ids not equal
! fields round-trip: Error on property evaluation after 0 passed tests:
Check paniced: reflect: Call using zero Value argument goroutine 14
[running]:
runtime/debug.Stack(0x0, 0x0, 0x0)
/usr/lib/go/src/runtime/debug/stack.go:24 +0x80
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter.SaveProp.
func1.1(0xc8202a9ba8)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/prop.go:19 +0x53
panic(0x79ffc0, 0xc8203daaf0)
/usr/lib/go/src/runtime/panic.go:426 +0x4e9
reflect.Value.call(0x8311a0, 0xc8203da2d0, 0x13, 0x8e40c8, 0x4,
0xc820395f90, 0x3, 0x3, 0x0, 0x0, ...)
/usr/lib/go/src/reflect/value.go:367 +0x48b
reflect.Value.Call(0x8311a0, 0xc8203da2d0, 0x13, 0xc820395f90, 0x3, 0x3,
0x0, 0x0, 0x0)
/usr/lib/go/src/reflect/value.go:303 +0xb1
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter/prop.chec
kConditionFunc.func2(0xc8203b2900, 0x3, 0x3, 0x3)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/prop/check_condition_func.go:45 +0x1db
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter/prop.ForA
ll.func1.1(0x0, 0x0, 0x86a401)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/prop/forall.go:51 +0x112
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter/prop.firs
tFailure(0xc8203cea80, 0xc8202a9b40, 0xc8203cea80, 0x0, 0x0)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/prop/forall.go:104 +0x5f
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter/prop.shri
nkValue(0x3e8, 0xc820395e00, 0x778480, 0xc8203da7e0, 0xc820395f40,
0xc8202a9b40, 0xc820395f40, 0x0, 0x0)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/prop/forall.go:88 +0xb5
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter/prop.ForA
ll.func1(0xc8203edc00, 0x9b53c8)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/prop/forall.go:52 +0x53d
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter.SaveProp.
func1(0xc8203edc00, 0x0)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/prop.go:24 +0x68
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter.Prop.Chec
k.func1(0x0, 0xc8203da360, 0xd44b20)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/prop.go:51 +0x19e
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter.(*runner)
.runWorkers(0xc8203b25a0, 0x0)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/runner.go:45 +0x102
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter.Prop.Chec
k(0xc8203da340, 0xc8201f4140, 0x938f20)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/prop.go:106 +0x1c2
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter.(*Propert
ies).Run(0xc8203b2480, 0x7ff522cf6968, 0xc8203ed860, 0x45085e)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/properties.go:35 +0xe7
github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gopter.(*Propert
ies).TestingRun(0xc8203b2480, 0xc8200845a0)
/golang/src/github.com/ClusterHQ/dataplane/vendor/github.com/leanovate/gop
ter/properties.go:46 +0x119
github.com/ClusterHQ/dataplane/dataset.TestAddSnapshot(0xc8200845a0)
/golang/src/github.com/ClusterHQ/dataplane/dataset/dataset_test.go:409
+0x295
testing.tRunner(0xc8200845a0, 0xd2da98)
I think I minimized the test case that provokes this behavior. You can see it at master...ClusterHQ:call-using-zero-value-argument
I wasn't quite sure where to put this test or even if it's a real problem I've found or just a mis-use of PtrOf. Finishing up for today, will probably look more tomorrow, thought I'd see if you had any thoughts on the issue.
The following code panics sometimes:
package main
import (
"github.com/leanovate/gopter"
"github.com/leanovate/gopter/gen"
"github.com/leanovate/gopter/prop"
)
type A struct {
Value *float64
}
func genA() gopter.Gen {
return gopter.DeriveGen(
func(f *float64) A {
return A{Value: f}
},
func(a A) *float64 {
return a.Value
},
gen.PtrOf(gen.Float64()),
)
}
type B struct {
A1 A
}
func genB() gopter.Gen {
return gopter.DeriveGen(
func(a1 A) B {
return B{a1}
}, func(b B) A {
return b.A1
},
genA(),
)
}
func main() {
properties := gopter.NewProperties(nil)
properties.Property("should not panic", prop.ForAll(
func(b B) bool {
return false
},
genB(),
))
properties.Run(gopter.ConsoleReporter(false))
}
with the output being:
! should not panic: Error on property evaluation after 0 passed tests:
Check paniced: reflect: call of reflect.Value.Interface on zero Value
goroutine 1 [running]:
runtime/debug.Stack(0xc4200556b0, 0x10f6840, 0xc42000aa80)
/usr/local/Cellar/go/1.10.3/libexec/src/runtime/debug/stack.go:24 +0xa7
github.com/leanovate/gopter.SaveProp.func1.1(0xc420055d30)
/Users/joel/code/go/src/github.com/leanovate/gopter/prop.go:19 +0x6e
panic(0x10f6840, 0xc42000aa80)
/usr/local/Cellar/go/1.10.3/libexec/src/runtime/panic.go:502 +0x229
reflect.valueInterface(0x0, 0x0, 0x0, 0x1, 0x10e49e0, 0x0)
/usr/local/Cellar/go/1.10.3/libexec/src/reflect/value.go:953 +0x1a3
reflect.Value.Interface(0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/Cellar/go/1.10.3/libexec/src/reflect/value.go:948 +0x44
github.com/leanovate/gopter/gen.PtrShrinker.func1(0x10e49e0, 0x0, 0x1)
/Users/joel/code/go/src/github.com/leanovate/gopter/gen/ptr_shrink.go:28
+0xc1
github.com/leanovate/gopter.CombineShrinker.func1(0x10e8c80, 0xc42000aa60,
0x10e8c80)
/Users/joel/code/go/src/github.com/leanovate/gopter/shrink.go:168 +0x139
github.com/leanovate/gopter.(*derivedGen).Shrinker(0xc420094500,
0x10fc000, 0x0, 0xc42000c0e0)
/Users/joel/code/go/src/github.com/leanovate/gopter/derived_gen.go:81
+0xe3
github.com/leanovate/gopter.(*derivedGen).Shrinker-fm(0x10fc000, 0x0, 0x1)
/Users/joel/code/go/src/github.com/leanovate/gopter/derived_gen.go:31
+0x3e
github.com/leanovate/gopter.CombineShrinker.func1(0x10e8c80, 0xc42000aa00,
0x10e8c80)
/Users/joel/code/go/src/github.com/leanovate/gopter/shrink.go:168 +0x139
github.com/leanovate/gopter.(*derivedGen).Shrinker(0xc420094780,
0x10fc080, 0x0, 0xc420055b90)
/Users/joel/code/go/src/github.com/leanovate/gopter/derived_gen.go:81
+0xe3
github.com/leanovate/gopter.(*derivedGen).Shrinker-fm(0x10fc080, 0x0,
0x10fc080)
/Users/joel/code/go/src/github.com/leanovate/gopter/derived_gen.go:31
+0x3e
github.com/leanovate/gopter/prop.shrinkValue(0x3e8, 0xc420094a00,
0x10fc080, 0x0, 0xc420094a50, 0xc420055cb8, 0x10c4dbc, 0x10fc980,
0xc42000e480)
/Users/joel/code/go/src/github.com/leanovate/gopter/prop/forall.go:99
+0x4e
github.com/leanovate/gopter/prop.ForAll.func1(0xc42000a620, 0x1122320)
/Users/joel/code/go/src/github.com/leanovate/gopter/prop/forall.go:47
+0x532
github.com/leanovate/gopter.SaveProp.func1(0xc42000a620, 0x0)
/Users/joel/code/go/src/github.com/leanovate/gopter/prop.go:24 +0x6c
github.com/leanovate/gopter.Prop.Check.func1(0x0, 0xc42000e4c0, 0x11c69a0)
/Users/joel/code/go/src/github.com/leanovate/gopter/prop.go:52 +0x15c
github.com/leanovate/gopter.(*runner).runWorkers(0xc420088570, 0x0)
/Users/joel/code/go/src/github.com/leanovate/gopter/runner.go:49 +0x2d9
github.com/leanovate/gopter.Prop.Check(0xc42000e4a0, 0xc42001e6c0,
0x111b6c4)
/Users/joel/code/go/src/github.com/leanovate/gopter/prop.go:110 +0x1af
github.com/leanovate/gopter.(*Properties).Run(0xc4200882d0, 0x1131560,
0xc42000a5e0, 0x10)
/Users/joel/code/go/src/github.com/leanovate/gopter/properties.go:37 +0xc9
main.main()
/Users/joel/code/gopter_sample.go:50 +0x1b0
It seems to only happen if the shrinker is used. If I change, for example, the return false
in the property to return true
it does never happen.
Somehow it requires these two nested structs. Using just struct A does not result in a panic... Not sure why or if I was just unlucky when reproducing...
It looks like the problem is in the Ptr shrink implementation where the call to Interface
panics on a nil value.
I'm happy to help to fix this but I need a bit of guidance :)
Currently, gen.Struct (and gen.StructPtr) create results by taking elements from field generators, and combining them into a struct, which is emitting by the struct generator. However, this struct is emitted using NoShrinker, so it cannot be shrunk. In order to get shrinkable structs, I have been manually writing Gens using DeriveGen, but it would be much nicer to be able to use the helper functions like gen.Struct, and if the default arbitrary generators gave shrinkable results as well.
I've been skimming through the docs, and coming from ScalaCheck i was looking for generating parallel command sequences. Is it supported?
gopter/arbitrary/arbitraries.go
Line 26 in f24d09d
Really sorry about this, turns out I've messed up on the types of the Shrinker. Shrunk values are still time.Time
rather than *time.Time
. I'll make some time to fix this up over the next few days.
The adjusted example from the doc:
package main
import (
"errors"
"math/cmplx"
"github.com/leanovate/gopter"
"github.com/leanovate/gopter/arbitrary"
"github.com/leanovate/gopter/gen"
)
type QudraticEquation struct {
A, B, C complex128
}
func (q QudraticEquation) Eval(x complex128) complex128 {
return q.A*x*x + q.B*x + q.C
}
func (q QudraticEquation) Solve() (complex128, complex128, error) {
if q.A == 0 {
return 0, 0, errors.New("No solution")
}
v := q.B*q.B - 4*q.A*q.C
v = cmplx.Sqrt(v)
return (-q.B + v) / 2 / q.A, (-q.B - v) / 2 / q.A, nil
}
func main() {
parameters := gopter.DefaultTestParameters()
parameters.Rng.Seed(1234) // Just for this example to generate reproducable results
arbitraries := arbitrary.DefaultArbitraries()
arbitraries.RegisterGen(gen.Complex128Box(-1e8-1e8i, 1e8+1e8i)) // Only use complex values within a range
properties := gopter.NewProperties(parameters)
properties.Property("Quadratic equations can be solved", arbitraries.ForAll(
func(quadratic QudraticEquation) bool {
x1, x2, err := quadratic.Solve()
if err != nil {
return true
}
return cmplx.Abs(quadratic.Eval(x1)) < 1e-5 && cmplx.Abs(quadratic.Eval(x2)) < 1e-5
}))
properties.Property("Quadratic equations can be solved alternative", arbitraries.ForAll(
func(a, b, c complex128) bool {
quadratic := &QudraticEquation{
A: a,
B: b,
C: c,
}
x1, x2, err := quadratic.Solve()
if err != nil {
return true
}
return cmplx.Abs(quadratic.Eval(x1)) < 1e-5 && cmplx.Abs(quadratic.Eval(x2)) < 1e-5
}))
// When using testing.T you might just use: properties.TestingRun(t)
properties.Run(gopter.ConsoleReporter(false))
}
Output:
! Quadratic equations can be solved: Error on property evaluation after 0
passed tests: Check paniced: runtime error: invalid memory address or nil
pointer dereference goroutine 1 [running]:
runtime/debug.Stack(0xc42004ba90, 0x4ffa40, 0x5d0ca0)
/usr/x86_64-pc-linux-gnu/lib/go/src/runtime/debug/stack.go:24 +0xa7
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.Save
Prop.func1.1(0xc42004bd20)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/prop.go:19 +0x6e
panic(0x4ffa40, 0x5d0ca0)
/usr/x86_64-pc-linux-gnu/lib/go/src/runtime/panic.go:505 +0x229
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter/prop
.ForAll.func1(0xc420098460, 0x52aaf8)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/prop/forall.go:31 +0x13b
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.Save
Prop.func1(0xc420098460, 0x0)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/prop.go:24 +0x6c
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.Prop
.Check.func1(0x0, 0xc42007e3d0, 0x5d7f60)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/prop.go:52 +0x15c
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.(*ru
nner).runWorkers(0xc420082510, 0x0)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/runner.go:45 +0x2d9
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.Prop
.Check(0xc42007e3a0, 0xc4200a05c0, 0x52741e)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/prop.go:110 +0x1af
github.com/hasufell/gopter-examples/vendor/github.com/leanovate/gopter.(*Pr
operties).Run(0xc420082480, 0x53a700, 0xc420098420, 0x2d)
/home/julian/go/src/github.com/hasufell/gopter-examples/vendor/github.com/
leanovate/gopter/properties.go:37 +0xc9
main.main()
/home/julian/go/src/github.com/hasufell/gopter-examples/gopterexamples.go:
64 +0x2af
+ Quadratic equations can be solved alternative: OK, passed 100 tests.
When transitioning a model using Command.NextState -> commands.State
, is it dangerous to share pointers or reference types between model states?
For example,
type State struct {
index map[string]string
}
func (writeFooBarCommand) NextState(s commands.State) commands.State {
current := s.(State)
next := current
next.index["foo"] = "bar"
return next
}
I believe that the existing model state and the new model state will share a reference to the underlying map, meaning manipulations via the old state instance or the new state instance will reflect in both objects. I currently make it a habit to deep copy my model states when transitioning, but my question is whether or not the deep copy is necessary.
Hello! Thanks for this great library and apologies in advance if this is me just missunderstanding it (go newbie).
I've got a stateful system, a bitarray and the state is a []bool
. I'm trying to assert that the bitarray is equivalent to the state ([]bool).
I'm using 2 gen.IntRange:
Despite using IntRange to stay within the bounds of he SUT and state, I eventually see illegal values (I suspect during the shrink phase, as they only appear after the post condition fails), which causes the test to panic.
Is this intended behaviour for the gen.IntRange (to shrink outside the range)?
Another interesting detail: if I remove the IntRange from the initial state and only keep it in the command, the Index values for the command stay within range and I never see the panic.
See an example here: https://play.golang.org/p/xplrfKS_Nvm
Thanks!
There are generators for slices, but none for maps. What I have been doing to work around this is generating helper structs containing slices and then computing the maps after generation. This seems to work fine, but is a bit ugly.
Is there a fundamental reason why generating maps is hard, or is it just a matter of work to build it?
passing an array type to GenForType
returns nil, and this only panics once the properties are actually ran so it isn't obvious what the issue is, it produces the following stack trace:
! roundtrip: Error on property evaluation after 0 passed tests: Check
paniced: runtime error: invalid memory address or nil pointer dereference
goroutine 19 [running]:
runtime/debug.Stack()
/usr/local/go/src/runtime/debug/stack.go:24 +0x5e
github.com/leanovate/gopter/prop.ForAll.SaveProp.func3.1()
/home/wessie/go/pkg/mod/github.com/leanovate/[email protected]/prop.go:20 +0x6b
panic({0x6c7ec0?, 0x96c6a0?})
/usr/local/go/src/runtime/panic.go:914 +0x21f
github.com/leanovate/gopter/prop.ForAll.func1(0xc0000be2e0)
/home/wessie/go/pkg/mod/github.com/leanovate/[email protected]/prop/forall.go:33 +0xc7
github.com/leanovate/gopter/prop.ForAll.SaveProp.func3(0x7fc1b4964b98?)
/home/wessie/go/pkg/mod/github.com/leanovate/[email protected]/prop.go:25 +0x5b
github.com/leanovate/gopter.Prop.Check.func1(0x0, 0xc000090580)
/home/wessie/go/pkg/mod/github.com/leanovate/[email protected]/prop.go:53 +0x132
github.com/leanovate/gopter.(*runner).runWorkers(0xc000094f90)
/home/wessie/go/pkg/mod/github.com/leanovate/[email protected]/runner.go:49 +0x1cd
github.com/leanovate/gopter.Prop.Check(0xc000090560, 0xc0000f0b80)
/home/wessie/go/pkg/mod/github.com/leanovate/[email protected]/prop.go:112 +0x1be
github.com/leanovate/gopter.(*Properties).Run(0xc0000d9f38, {0x78f100, 0xc0000be2a0})
/home/wessie/go/pkg/mod/github.com/leanovate/[email protected]/properties.go:37 +0xa9
github.com/leanovate/gopter.(*Properties).TestingRun(0xc0000d9f38, 0xc000084ea0, {0x0, 0x0, 0x6b14a0?})
/home/wessie/go/pkg/mod/github.com/leanovate/[email protected]/properties.go:56 +0xfc
GenForType
should probably panic if there is an unsupported type
I've got the following situation:
// regular generator using gen.* types with gopter.CombineGens/gopter.Map
func genX() gopter.Gen { ... }
// more complicated generator, which needs the output of `genX`, and the ability
// to use the rng from `*gopter.GenParams`.
func genXY() gopter.Gen {}
I can't use any of Map/FlatMap/MapResult
in genXY()
, as I still need access to the *gopter.GenParams
. I ended up doing something hacky like:
func genXY() gopter.Gen {
return func(params *gopter.GenParams) *gopter.GenResult {
x := genX()(params)
// using the result of x, especially the shrinkers to create a new result
// along with the rng from `params`
}
}
This works but causes data races in the rng if I'm using multiple workers. Can we expose a new API (e.g. MapExtended(params *gopter.GenParams, result gopter.GenResult)
) to allow for such composition as a first class in the API?
NB: in an un-related note, thank you so much for this library! My team & I have caught so many issues using it. Massive massive props.
I have a scenario where I'd like to generate commands for a system under test based on the current state of the system. For example, let's say my system under test is an on-disk set of strings and I've modeled it with an in-memory set. It supports two operations: Add and Has:
Add() string
adds a random string to the set and returns this string
Has(s string) bool
returns true if the given string is in the set
To test this system I might have a generator for Add
and Has
commands and my test would apply the commands to each system and check that the results match. When I generate my Has
commands it would be useful if the current in-memory state could inform my generator so that I ensure that the string I'm passing to Has
actually was added to the set. Would the state
parameter of the GenCommandFunc
function allow me to do something like this?
Here's the general idea I had for how I might structure the generators:
type MySet struct {
s map[string]bool
}
func (s *MySet) Add() string {
r := randomString()
s.s[r] = true
return r
}
func (s *MySet) Has(s string) bool {
_, ok := s.s[s]
return ok
}
func AddCommand() gopter.Gen {
return gen.Const(&addCommand{})
}
func HasCommand(state *MySet) gopter.Gen {
return gen.Int().Map(func(i int) commands.Command {
if len(state.s) == 0 {
return hasCommand("")
}
i = i % len(state.s)
for _, str := range {
if i == 0 {
return hasCommand(str)
}
i--
}
})
}
In a scenario like this it's difficult because passing any random string to Has()
would probably result in it returning false. It's very unlikely that a randomly generated string would match one returned by the system under test. What is the intended use of that parameter in the GenCommandFunc
function? Is there a better way to construct generators for commands where the parameters of one command might rely on the result of another?
I do a couple of things to work around the default interfaces exposed by gopter
(1) I like using time.Now()
as a seed for my tests, but also having this reported upon failure so I can reproduce the failure. My tests end up looking like:
func TestSomethingOrTheOther(t *testing.T) {
parameters := gopter.DefaultTestParameters()
seed := time.Now().UnixNano()
parameters.Rng = rand.New(rand.NewSource(seed))
properties := gopter.NewProperties(parameters)
...
properties.Property("...", prop.ForAll(...)
...
reporter := gopter.NewFormatedReporter(true, 160, os.Stdout)
if !properties.Run(reporter) {
t.Errorf("failed with initial seed: %d", seed)
}
}
(2) Overriding width for the formatted reporter
The default test reporter truncates the output to 79 columns (iirc); I've always overriden this to a larger value so that stacktraces do not get chopped into different pieces.
Would you be open to changing the API (exposing new ctors) to make these the defaults? I'm happy to contribute the changes if so.
Hi,
I got an error when trying to generating a slice of pointers using the code below:
package main
import (
"github.com/leanovate/gopter"
"github.com/leanovate/gopter/gen"
"github.com/leanovate/gopter/prop"
"reflect"
)
type Empty struct {
ID uint
}
func main() {
properties := gopter.NewProperties(nil)
properties.Property("just test pointer", prop.ForAll(
func(nodes []*Empty) bool {
return true
},
gen.SliceOf(gen.Struct(reflect.TypeOf(&Empty{}), map[string]gopter.Gen{
"ID": gen.UInt(),
})),
))
properties.Run(gopter.ConsoleReporter(false))
}
The error was:
! just test pointer: Error on property evaluation after 0 passed tests:
Check paniced: reflect: Call using []main.Empty as type []*main.Empty
ARG_0: []
I think reflect.TypeOf(&Empty{})
returns *main.Empty
instead of main.Empty
, but according to the error messages, the generated type is []main.Empty
; so I just wonder whether there's something wrong with my code here.
Hello - thanks for a great testing library!
I'm trying to use gopter to test a simple stateful object. The object is initialized with no existing state, and then methods are called. After each method is called, I'd like to execute a check on the internal state of the object. Note this means I don't want to maintain a separate State object, I'd basically like my SystemUnderTest to also act as the State object and I don't want to have to do any kind of updates in NextState. Ideally the Run method will be called, executing one of the object's methods, and then PostCondition will check some property of the object.
I believe I managed to get this working, but a few things that aren't clear:
I've included a minimum viable example below.
Thanks!
package keeper
import (
"fmt"
"testing"
"github.com/leanovate/gopter"
"github.com/leanovate/gopter/commands"
"github.com/leanovate/gopter/gen"
)
type sys struct {
height int
heights map[int]uint8
}
// store i under the latest height value and incremenet the height value
func (s *sys) New(i uint8) {
h := s.height
s.height += 1
// bug when height is 5
if s.height == 5 {
return
}
s.heights[h] = i
}
type newCommand uint8
func (value newCommand) Run(q commands.SystemUnderTest) commands.Result {
s := q.(*sys)
s.New(uint8(value))
return s
}
// expect there to be an entry in the `sys.heights` map for each height up to the `sys.height`
func (newCommand) PostCondition(state commands.State, result commands.Result) *gopter.PropResult {
// XXX: if I try to use the state instead of the result here, it doesn't work
// (state doesn't seem to see the effects from Run)
s := result.(*sys)
for i := 0; i < s.height; i++ {
_, ok := s.heights[i]
if !ok {
return &gopter.PropResult{Status: gopter.PropFalse}
}
}
return &gopter.PropResult{Status: gopter.PropTrue}
}
func (value newCommand) NextState(state commands.State) commands.State {
s := state.(*sys)
return s
}
func (newCommand) PreCondition(state commands.State) bool {
return true
}
func (value newCommand) String() string {
return fmt.Sprintf("New(%d)", value)
}
var genNewCommand = gen.UInt8().Map(func(value uint8) commands.Command {
return newCommand(value)
})
func cbCommands(t *testing.T) *commands.ProtoCommands {
return &commands.ProtoCommands{
NewSystemUnderTestFunc: func(initState commands.State) commands.SystemUnderTest {
return initState
},
InitialStateGen: func(p *gopter.GenParameters) *gopter.GenResult {
// XXX: this function seems to execute many more times than Run
result := &sys{
height: 0,
heights: make(map[int]uint8),
}
return gopter.NewGenResult(result, gopter.NoShrinker)
},
GenCommandFunc: func(state commands.State) gopter.Gen {
return gen.OneGenOf(genNewCommand)
},
}
}
func TestSys(t *testing.T) {
parameters := gopter.DefaultTestParametersWithSeed(1235)
properties := gopter.NewProperties(parameters)
properties.Property("circular buffer", commands.Prop(cbCommands(t)))
properties.TestingRun(t)
}
Do you want to add the badge from goreportcard link to the README.md to get listed in awesome-go list. For more info refer to this PR.
I find that code like the following:
package panics
import (
"testing"
"github.com/leanovate/gopter"
"github.com/leanovate/gopter/gen"
"github.com/leanovate/gopter/prop"
)
func TestPanic(t *testing.T) {
properties := gopter.NewProperties(nil)
properties.Property("Will panic", prop.ForAll(
func(i int) bool {
if i%2 == 0 {
panic("hi")
}
return true
},
gen.Int().WithLabel("number")))
properties.TestingRun(t)
}
does catch the panic, but while it prints the random seed, it does not print the parameters that resulted in the panic:
$ go test -run TestPanic . ! Will panic: Error on property evaluation after 0 passed tests: Check paniced: hi goroutine 20 [running]: runtime/debug.Stack(0xc0000b36e8, 0x130fa80, 0x13ded80) /usr/local/Cellar/go/1.11.5/libexec/src/runtime/debug/stack.go:24 +0xa7 github.com/leanovate/gopter.SaveProp.func1.1(0xc0000b3cf0) /Users/asf/go/src/github.com/leanovate/gopter/prop.go:19 +0x6e panic(0x130fa80, 0x13ded80) /usr/local/Cellar/go/1.11.5/libexec/src/runtime/panic.go:513 +0x1b9 github.com/antifuchs/o_test.TestPanic.func1(0x0) /Users/asf/go/src/github.com/antifuchs/o/ranges_properties_test.go:131 +0x39 reflect.Value.call(0x130d6c0, 0x13a01f0, 0x13, 0x138a736, 0x4, 0x1643500, 0x0, 0x0, 0x0, 0x0, ...) /usr/local/Cellar/go/1.11.5/libexec/src/reflect/value.go:447 +0x454 reflect.Value.Call(0x130d6c0, 0x13a01f0, 0x13, 0x1643500, 0x0, 0x0, 0x13898c0, 0x1, 0x1643500) /usr/local/Cellar/go/1.11.5/libexec/src/reflect/value.go:308 +0xa4 github.com/leanovate/gopter/prop.checkConditionFunc.func2(0x1643500, 0x0, 0x0, 0x1643500) /Users/asf/go/src/github.com/leanovate/gopter/prop/check_condition_func.go :37 +0x5e github.com/leanovate/gopter/prop.ForAll.func1(0xc00009a560, 0x13a0258) /Users/asf/go/src/github.com/leanovate/gopter/prop/forall.go:40 +0x266 github.com/leanovate/gopter.SaveProp.func1(0xc00009a560, 0x0) /Users/asf/go/src/github.com/leanovate/gopter/prop.go:24 +0x6c github.com/leanovate/gopter.Prop.Check.func1(0x0, 0xc000084fd0, 0x1625ca0) /Users/asf/go/src/github.com/leanovate/gopter/prop.go:52 +0x155 github.com/leanovate/gopter.(*runner).runWorkers(0xc000089170, 0x0) /Users/asf/go/src/github.com/leanovate/gopter/runner.go:49 +0x2d1 github.com/leanovate/gopter.Prop.Check(0xc000084fb0, 0xc000090840, 0x138be08) /Users/asf/go/src/github.com/leanovate/gopter/prop.go:110 +0x194 github.com/leanovate/gopter.(*Properties).Run(0xc000089110, 0x13e0d60, 0xc00009a520, 0xc00008b1e0) /Users/asf/go/src/github.com/leanovate/gopter/properties.go:37 +0xbc github.com/leanovate/gopter.(*Properties).TestingRun(0xc000089110, 0xc000116100, 0x0, 0x0, 0x0) /Users/asf/go/src/github.com/leanovate/gopter/properties.go:56 +0x137 github.com/antifuchs/o_test.TestPanic(0xc000116100) /Users/asf/go/src/github.com/antifuchs/o/ranges_properties_test.go:133 +0x126 testing.tRunner(0xc000116100, 0x13a01f8) /usr/local/Cellar/go/1.11.5/libexec/src/testing/testing.go:827 +0xbf created by testing.(*T).Run /usr/local/Cellar/go/1.11.5/libexec/src/testing/testing.go:878 +0x35cElapsed time: 275.319ยตs
--- FAIL: TestPanic (0.00s)
properties.go:57: failed with initial seed: 1552922262110846000
FAIL
FAIL github.com/antifuchs/o 0.015s
What I would expect to see is to see the "Labels of the failing property" printed.
To work around this, I hard-code the random seed in the test options and return false
at the start of my function if I am interested in the values, but this seems like a thing gopter could & should do automatically, especially as it already captures the panic.
Trying to get a handle on how WithLabel
could help decoding inputs in the event of test failure. For the simple example shown on https://godoc.org/github.com/leanovate/gopter with two simple inputs, it seems fine. But here is what I have after I have put WithLabel
on most everything in my test generators:
Project_Name, Project_Id, Rules: {{๏ถโบเบก๐เฌ b- {} [] 0} [{5p7 -
๐๏ทผโด๐ญแผฒ 1 [0xc0002e1a40 0xc0002e4c30 0xc0002e5e00 0xc000558ff0
0xc00055c1e0] {} [] 515} {xm x แ๐ญ๐เ พแณ 1 [0xc0005aba90
0xc0005aec80 0xc0005afe50 0xc0005b7040 0xc0005fe8c0] {} [] 0} {h5 w
เฑ๐ฌน๐๐กำ 0 [0xc000575130 0xc00053a690 0xc00053b900 0xc000554dc0
0xc000538410] {} [] 0} {-y -y เงเค๐ฐ๐ฌเฟ 0 [0xc000525450
0xc000520f50 0xc00051ea00 0xc00051c460 0xc00051df40] {} [] 0} {2-9v a
เทแฉฅแฌฅเถโตฃ 1 [0xc00050f8b0 0xc00050aff0 0xc000506730 0xc000507f40
0xc000505720] {} [] 0}]}
First, it seems to have shown the labels only for the topmost items.
Second, it put all the labels first, not intermixed with their values.
Here is how those labels match the inputs:
Just to give some feel for how my generators are organized, here it is in pseudo-code:
createProjectAndRuleGen =>
CombineGens(projectReqGen, gen.SliceOf(ruleReqGen)
projectReqGen => (id, name)
ruleReqGen => CombineGens(gen.Bool(), ruleReqGenEvent, ruleReqGenNode)
ruleReqGenEvent => (id, name, conditions1, and some other properties...)
ruleReqGenNode => (id, name, conditions2, and some other properties...)
conditions1 => gen.SliceOf(conditionsGenEvent)
conditions2 => gen.SliceOf(conditionsGenNode)
conditionsGenEvent => (type, value)
conditionsGenNode => (type, value)
Looking for tips on how to make the input blob be less indecipherable.
Hi,
I'm seeing a performance issue when using generators in somewhat high numbers.
DefaultGenParameters()
function is calling rand.New(rand.NewSource(seed))
which internally uses locks among other things in Go. However if one will call DefaultGenParameters()
once and use this instance instead of repeatedly calling the function it can improve performance dramatically.
In all the examples shown the usage of DefaultGenParameters()
function is only for the side effect and not the result.
https://github.com/leanovate/gopter/blob/master/derived_gen.go#L109
https://github.com/leanovate/gopter/blob/master/gen.go#L51
https://github.com/leanovate/gopter/blob/master/gen.go#L105
I can create a pull request if needed.
I'm just getting started with gopter (also being fairly new to property based testing in general) and I was trying to write a custom generator for a struct type of mine with private fields/initalizers.
A simplified example:
type Struct struct {
name string
version string
}
func NewStruct(n, v string) *Struct {
return &Struct{n, v}
}
func (s Struct) String() string {
return s.name //+ ":" + s.version
}
func GenStruct() gopter.Gen {
return gopter.CombineGens(
gen.Identifier(),
gen.NumString().SuchThat(nonZeroLen),
).Map(func(values []interface{}) Struct {
name := values[0].(string)
ver := values[1].(string)
return NewStruct(name, ver)
})
}
This works but as documented Map
looses the shrinker from CombineGens
. (Okay, slightly surprising but documented.)
My attempt at writing a version that adds a Shrinker ran into some problems - it seemed to loop and not shrink as well (or almost at all) as the CombineGen case. I was trying this:
func GenStructShrink() gopter.Gen {
combines := gopter.CombineGens(
gen.Identifier().WithLabel("name"),
gen.NumString().SuchThat(nonZeroLen).WithLabel("ver"),
)
return func(genParams *gopter.GenParameters) *gopter.GenResult {
input := combines(genParams)
values, ok := input.Retrieve()
var s *Struct
if !ok {
return gopter.NewEmptyResult(reflect.TypeOf(s))
}
mapper := func(strings []interface{}) *Struct {
name := strings[0].(string)
ver := strings[1].(string)
return NewStruct(name, ver)
}
s = mapper(values.([]interface{}))
myShrinker := func(value interface{}) gopter.Shrink {
fmt.Printf("Creating chrinker for %#v\n", value)
return input.Shrinker(values).Map(mapper)
}
return gopter.NewGenResult(s, myShrinker)
}
}
I would have expected it to create the shrinker once and call it again, but it seems to create a new version of the shrinker many times which means it resets the shrink (because I'm ignoring the input value
to try and get back at the underlying generators.
I guess this is a long way of saying: what is the exact interface of the Shrinkers? What is the Shrinker called with? When is it called? How often?
The following test panics:
package main_test
import (
"reflect"
"testing"
"github.com/leanovate/gopter"
"github.com/leanovate/gopter/gen"
"github.com/leanovate/gopter/prop"
)
type Box struct {
Content interface{}
}
func BoxOf(inner gopter.Gen) gopter.Gen {
return gen.Struct(reflect.TypeOf(Box{}), map[string]gopter.Gen{
"Content": inner,
})
}
func TestPanics(t *testing.T) {
properties := gopter.NewProperties(nil)
properties.Property("Should not panic",
prop.ForAll(
func(e Box) bool {
return reflect.DeepEqual(e, Box{0})
},
BoxOf(gen.Int()),
))
properties.TestingRun(t)
}
What seems to be happening is that I can't use a generator of int
to fill a variable of type interface{}
in Box. What I expected to happen is for this to be fine. After all, there's nothing wrong with var i interface{} = 3
so why is there a problem with BoxOf(gen.Int())
?
I can work around by using an explicit Map() to change the type of the Int() generator to interface{}:
func(e Box) bool {
return reflect.DeepEqual(e, Box{0})
},
- BoxOf(gen.Int()),
+ BoxOf(
+ gen.Int().Map(func(in *gopter.GenResult) *gopter.GenResult {
+ in.ResultType = reflect.TypeOf((*interface{})(nil)).Elem()
+ return in
+ }),
+ ),
))
properties.TestingRun(t)
but this feels pretty heavyweight. Is it possible to get structs to accept concrete generators for interface types?
Thanks for this package, it's great!
I'm working on something and running into a bit of trouble with a recursive data type. How might you use this package for recursive data? For instance in the ScalaCheck documentation, there's an example of binary integer trees. How could you emulate that example with this package?
I'd assume you start like:
type tree interface {
isTree()
}
type node struct {
left tree
right tree
v int
}
func (*node) isTree() {}
type leaf struct{}
func (*leaf) isTree() {}
var genLeaf gopter.Gen = gen.Const(&leaf{})
var genNode gopter.Gen = gen.Struct(reflect.TypeOf(&node{}), map[string]gopter.Gen{
"left": genTree,
"right": genTree,
"v": gen.Int(),
})
var genTree gopter.Gen = gen.OneGenOf(genLeaf, genNode)
But that ends up with an initialization loop. Converting genNode
and genTree
to functions allows it to be initialized:
func genNode() gopter.Gen {
return gen.Struct(reflect.TypeOf(&node{}), map[string]gopter.Gen{
"left": genTree(),
"right": genTree(),
"v": gen.Int(),
})
}
func genTree() gopter.Gen { return gen.OneGenOf(genLeaf, genNode()) }
But, using either one ends up with a stack overflow.
Any thoughts on how to implement that binary int tree example?
Hey!
I am trying to generate unique ids however I am having some troubles with it. I have tried adding a list to which ids have already been generated and then discard any that matches. questionIdGenOptions.alreadyGeneratedQuestions
is an array of strings. Utils.Contains
is a function which returns true if the given array contains the corresponding string otherwise false.
func QuestionIdGen(questionIdGenOptions *QuestionIdGenOptions) gopter.Gen{
if questionIdGenOptions != nil && questionIdGenOptions.alreadyGeneratedQuestions == nil {
var newAlreadyGeneratedQuestions = []string{}
questionIdGenOptions.alreadyGeneratedQuestions = &newAlreadyGeneratedQuestions
}
pgen := gen.PtrOf(gen.AnyString())
if questionIdGenOptions != nil {
pgen = pgen.SuchThat(
func(generatedId *string) bool {
//Some characters look the same however their bytes are not i.e. it is therefore unique.
if questionIdGenOptions.GenUniqueId != nil && *questionIdGenOptions.GenUniqueId == true && (generatedId == nil || Utils.Contains(*questionIdGenOptions.alreadyGeneratedQuestions, *generatedId)) {
return false
}
newAlreadyGeneratedQuestions := append(*questionIdGenOptions.alreadyGeneratedQuestions, *generatedId)
questionIdGenOptions.alreadyGeneratedQuestions = &newAlreadyGeneratedQuestions
return true
})
}
return pgen
}
However atm. with this if statement, the wrapper generator which uses this function fails to generate anything... And I am unsure why almost like something internal error is happening. If I remove it everything works fine except the ids are not unique. Is there another way of doing it or?
I'm running some tests but for some reason gopter.Run doesn't stop. The tests make calls against a db that isn't very fast and after around 2000 tests I just kill the gopter.Run process because it takes too long. Is there any way to limit the amount of tests that get run before gopter says that the tests have passed?
Also I don't know why but other tests don't need nearly as many attempts.
I'm thinking of writing a gen.StructStrict
, which panics if gens
contains keys which are not fields on rt
, so for instance the "NotThere"
key in struct_test.go::TestStruct
would cause a failure. Would you be likely to accept this as a PR?
Looks like there have been a bunch of changes since the last release (1 year ago) including gen.Sized which looks quite useful. Could we cut a new release?
I have this massive generator which creates an arbitrary complex struct however each time I run the test I get a different result. So I started implementing tests for the generators to see if I could reproduce results from the same generator using the same seed in in two instances of properties. This is the layout of the test and its support functions:
func SetupReproducibleProperties(seed *int64) (*gopter.Properties, *gopter.Properties) {
var parameters1 *gopter.TestParameters
var parameters2 *gopter.TestParameters
if seed != nil {
parameters1 = gopter.DefaultTestParametersWithSeed(*seed)
parameters2 = gopter.DefaultTestParametersWithSeed(*seed)
}else{
parameters1 = gopter.DefaultTestParameters()
parameters2 = gopter.DefaultTestParametersWithSeed(parameters1.Seed)
}
properties1 := gopter.NewProperties(parameters1)
properties2 := gopter.NewProperties(parameters2)
return properties1, properties2
}
func GetHashStringFromInterface(i interface{}) string {
newJsonBytes, _ := json.Marshal(i)
newHash := md5.Sum(newJsonBytes)
return string(newHash[:])
}
func Test(t *testing.T) {
var seedToUse *int64
//seedToUse = Utils.Int64ToPtr(1604056858868077464)
properties1, properties2 := SetupReproducibleProperties(seedToUse)
generatorToUse := SomeGenerator
var oldGens []GeneratedStruct
properties1.Property("Generate first iteration", prop.ForAll(
func(value GeneratedStruct) bool {
oldGens = append(oldGens, value)
return true
},
generatorToUse,
))
counter := Utils.IntToPtr(0)
properties2.Property("Comparing hashes should be a match", prop.ForAll(
func(value GeneratedStruct) bool {
currentHash := Utils.GetHashStringFromInterface(value)
storedValue := oldGens[*counter]
storedHash := Utils.GetHashStringFromInterface(storedValue)
counter = Utils.IntToPtr(*counter + 1)
if storedHash == currentHash {
return true
}
return false
},
generatorToUse,
))
properties1.TestingRun(t)
properties2.TestingRun(t)
}
Running this test always succeeds. Same goes if you replace GeneratedStruct
with string and SomeGenerator
with gen.AnyString()
. However when I close the test down and start it up again I start seeing different results. If I use the AnyString example it generates the same value when I run the test and restart it. If I do it with my custom generated struct it does not generate the same result.
type SelectOption struct {
// gopg specific fields to make database management easier
tableName struct{} `pg:"selectoptions"`
// Fields specified in AsyncAPI configuration
Value string `json:"value" `
Selected *bool `json:"selected,omitempty" `
Name string `json:"name" `
SubsetQuestions *QuestionsKeyObject `json:"subsetQuestions,omitempty" `
}
The QuestionsKeyObject and Mapper are never generated so ignore them
type Question struct {
// gopg specific fields to make database management easier
tableName struct{} `pg:"questions"`
// Fields specified in AsyncAPI configuration
Id *string `json:"id,omitempty" `
Type QuestionTypesEnum `json:"type" `
Hint *string `json:"hint,omitempty" `
Placeholder *string `json:"placeholder,omitempty" `
DefaultValue *string `json:"defaultValue,omitempty" `
Options []SelectOption `json:"options,omitempty" `
CheckedSubsetQuestions *QuestionsKeyObject `json:"checkedSubsetQuestions,omitempty" `
UncheckedSubsetQuestions *QuestionsKeyObject `json:"uncheckedSubsetQuestions,omitempty" `
Questions *QuestionsKeyObject `json:"questions,omitempty" `
Mappers []Mapper `json:"mappers,omitempty" `
}
type QuestionTypesEnum int
type questionTypesStruct struct {
Text QuestionTypesEnum `json:"text"`
Select QuestionTypesEnum `json:"select"`
Container QuestionTypesEnum `json:"container"`
Textarea QuestionTypesEnum `json:"textarea"`
Checkbox QuestionTypesEnum `json:"checkbox"`
_values [5]QuestionTypesEnum
_names [5]string
}
var QuestionTypes = &questionTypesStruct{
Text: 0,
Select: 1,
Container: 2,
Textarea: 3,
Checkbox: 4,
_values: [5]QuestionTypesEnum{
0,
1,
2,
3,
4},
_names: [5]string{
"text",
"select",
"container",
"textarea",
"checkbox",
},
}
func TestQuestionGenShouldReproduceResults(t *testing.T) {
var seedToUse *int64
//seedToUse = Utils.Int64ToPtr(1604056858868077464)
properties1, properties2 := SetupReproducibleProperties(seedToUse)
generatorToUse := QuestionGenWithOptions()
var oldGens []schema.Question
properties1.Property("Generate first iteration", prop.ForAll(
func(question schema.Question) bool {
oldGens = append(oldGens, question)
return true
},
generatorToUse,
))
counter := Utils.IntToPtr(0)
properties2.Property("Comparing hashes should be a match", prop.ForAll(
func(question schema.Question) bool {
questionHash := Utils.GetHashStringFromInterface(question)
storedQuestion := oldGens[*counter]
storedHash := Utils.GetHashStringFromInterface(storedQuestion)
counter = Utils.IntToPtr(*counter+1)
if storedHash == questionHash {
return true
}
return false
},
generatorToUse,
))
properties1.TestingRun(t)
properties2.TestingRun(t)
}
func QuestionGenWithOptions() gopter.Gen {
questionGenOptionsGen := gen.PtrOf(QuestionGenOptionsGen()).WithLabel("QuestionGenOptionsGenPtrOf")
questionGenOptionsGen = questionGenOptionsGen.FlatMap(func(i interface{}) gopter.Gen {
var questionGenOptions *QuestionGenOptions
if i != nil {
questionGenOptions = i.(*QuestionGenOptions)
}
return SpecificQuestionGenWithOptions(schema.Question{}, questionGenOptions)
}, reflect.TypeOf(schema.Question{})).WithLabel("QuestionGenOptionsGenFlatMap")
return questionGenOptionsGen
}
func QuestionIdGenOptionsGen() gopter.Gen {
presetGens := map[string]gopter.Gen{
"DifferentThen": gen.PtrOf(gen.SliceOfN(10, gen.PtrOf(gen.AnyString()))).WithLabel("QuestionIdGenOptionsGenDifferentThen"),
"NotNil": gen.PtrOf(gen.Bool()).WithLabel("QuestionIdGenOptionsGenNotNil"),
"UseNil": gen.PtrOf(gen.Bool()).WithLabel("QuestionIdGenOptionsGenUseNil"),
"GenUniqueId": gen.PtrOf(gen.Bool()).WithLabel("QuestionIdGenOptionsGenGenUniqueId"),
}
return gen.Struct(reflect.TypeOf(QuestionIdGenOptions{}), presetGens).WithLabel("QuestionIdGenOptionsGen")
}
func QuestionGenOptionsGen() gopter.Gen {
presetGens := map[string]gopter.Gen{
"IdGenOptions": gen.PtrOf(QuestionIdGenOptionsGen()).WithLabel("QuestionGenOptionsGenIdGenOptions"),
"ExcludeId": gen.PtrOf(gen.Bool()).WithLabel("QuestionGenOptionsGenExcludeId"),
}
return gen.Struct(reflect.TypeOf(QuestionGenOptions{}), presetGens).WithLabel("QuestionGenOptionsGen")
}
type QuestionGenOptions struct {
//Use a custom generator for ids
IdGen *gopter.Gen
//Affect the current id generator by providing options
IdGenOptions *QuestionIdGenOptions
//Dont generate id, it will then always get the value nil
ExcludeId *bool
}
type QuestionIdGenOptions struct {
//Used to not generate the same id
DifferentThen *[]*string
//Used to ensure that nil are not generated, the generator returned are no longer a pointer
NotNil *bool
UseNil *bool
//Generate unique ids
GenUniqueId *bool
alreadyGeneratedQuestions *[]string
}
func QuestionIdGen(questionIdGenOptions *QuestionIdGenOptions) gopter.Gen {
if questionIdGenOptions != nil && questionIdGenOptions.alreadyGeneratedQuestions == nil {
var newAlreadyGeneratedQuestions = []string{}
questionIdGenOptions.alreadyGeneratedQuestions = &newAlreadyGeneratedQuestions
}
var pgen gopter.Gen
if questionIdGenOptions != nil && questionIdGenOptions.UseNil != nil && *questionIdGenOptions.UseNil == true {
var data *string = nil
pgen = gen.Const(data).WithLabel("QuestionIdGenNil")
}else{
pgen = gen.PtrOf(gen.AnyString()).WithLabel("QuestionIdGenPtrOf")
}
if questionIdGenOptions != nil {
pgen = pgen.SuchThat(
func(generatedId *string) bool {
//If they want nil dont make it different then
if (questionIdGenOptions.UseNil == nil || questionIdGenOptions.UseNil != nil && *questionIdGenOptions.UseNil == false) && questionIdGenOptions.DifferentThen != nil && generatedId != nil && Utils.ContainsPointers(*questionIdGenOptions.DifferentThen, generatedId) {
return false
}
if questionIdGenOptions.NotNil != nil && *questionIdGenOptions.NotNil == true && generatedId == nil {
return false
}
//Even if the characters are the same the bytes are not i.e. it is therefore unique.
//if questionIdGenOptions.GenUniqueId != nil && *questionIdGenOptions.GenUniqueId == true && (generatedId == nil || Utils.Contains(*questionIdGenOptions.alreadyGeneratedQuestions, *generatedId)) {
// return false
//}
if generatedId != nil {
newAlreadyGeneratedQuestions := append(*questionIdGenOptions.alreadyGeneratedQuestions, *generatedId)
questionIdGenOptions.alreadyGeneratedQuestions = &newAlreadyGeneratedQuestions
}
return true
}).WithLabel("QuestionIdGenSuchThat")
}
return pgen.WithLabel("QuestionIdGen")
}
func SpecificQuestionGenWithOptions(presetQuestions schema.Question, options *QuestionGenOptions) gopter.Gen {
presetGens := map[string]gopter.Gen{
"Type": gen.OneConstOf(
schema.QuestionTypes.Checkbox,
schema.QuestionTypes.Select,
schema.QuestionTypes.Container,
schema.QuestionTypes.Text,
schema.QuestionTypes.Textarea).WithLabel("QuestionTypeGen"),
}
//Based on the already existing presetQuestions decide if we should keep the values
if presetQuestions.Id != nil {
presetGens["Id"] = gen.Const(presetQuestions.Id).WithLabel("QuestionIdGen1")
} else {
if options != nil && options.IdGen != nil {
presetGens["Id"] = (*options.IdGen).WithLabel("QuestionIdGen2")
} else if options != nil {
presetGens["Id"] = QuestionIdGen(options.IdGenOptions).WithLabel("QuestionIdGen3")
}else{
presetGens["Id"] = QuestionIdGen(nil).WithLabel("QuestionIdGen4")
}
}
if presetQuestions.Hint != nil {
presetGens["Hint"] = gen.Const(presetQuestions.Hint).WithLabel("QuestionHintGen1")
} else {
presetGens["Hint"] = gen.PtrOf(gen.AnyString()).WithLabel("QuestionHintGen2")
}
if presetQuestions.Placeholder != nil {
presetGens["Placeholder"] = gen.Const(presetQuestions.Placeholder).WithLabel("QuestionHintGen1")
} else {
presetGens["Placeholder"] = gen.PtrOf(gen.AnyString()).WithLabel("QuestionHintGen1")
}
if presetQuestions.DefaultValue != nil {
presetGens["DefaultValue"] = gen.Const(presetQuestions.DefaultValue).WithLabel("QuestionDefaultValueGen1")
} else {
presetGens["DefaultValue"] = gen.PtrOf(gen.AnyString()).WithLabel("QuestionDefaultValueGen1")
}
if presetQuestions.Questions != nil {
presetGens["Questions"] = gen.Const(presetQuestions.Questions).WithLabel("QuestionQuestionsGen")
}
if presetQuestions.CheckedSubsetQuestions != nil {
presetGens["CheckedSubsetQuestions"] = gen.Const(presetQuestions.CheckedSubsetQuestions).WithLabel("QuestionCheckedSubsetQuestionsGen")
}
if presetQuestions.Mappers != nil {
presetGens["Mappers"] = gen.Const(presetQuestions.Mappers).WithLabel("QuestionMappersGen")
}
if presetQuestions.UncheckedSubsetQuestions != nil {
presetGens["UncheckedSubsetQuestions"] = gen.Const(presetQuestions.UncheckedSubsetQuestions).WithLabel("QuestionUncheckedSubsetQuestionsGen")
}
standardQuestion := gen.Struct(reflect.TypeOf(schema.Question{}), presetGens).WithLabel("QuestionStandardGen")
if presetQuestions.Options == nil {
standardQuestion = standardQuestion.FlatMap(
func(generatedValue interface{}) gopter.Gen {
generatedQuestion := generatedValue.(schema.Question)
presetGens := map[string]gopter.Gen{
"Type": gen.Const(generatedQuestion.Type),
"Hint": gen.Const(generatedQuestion.Hint),
"Id": gen.Const(generatedQuestion.Id),
"Placeholder": gen.Const(generatedQuestion.Placeholder),
"DefaultValue": gen.Const(generatedQuestion.DefaultValue),
"Questions": gen.Const(generatedQuestion.Questions),
"CheckedSubsetQuestions": gen.Const(generatedQuestion.CheckedSubsetQuestions),
"Mappers": gen.Const(generatedQuestion.Mappers),
"Options": gen.Const(generatedQuestion.Options),
"UncheckedSubsetQuestions": gen.Const(generatedQuestion.UncheckedSubsetQuestions),
}
if presetQuestions.Options != nil {
presetGens["Options"] = gen.Const(presetQuestions.Options).WithLabel("QuestionOptionsGen")
}else{
switch generatedQuestion.Type {
case schema.QuestionTypes.Select:
presetGens["Options"] = generateOptions()
break
}
}
return gen.Struct(reflect.TypeOf(schema.Question{}), presetGens)
},
reflect.TypeOf(schema.Question{}),
).WithLabel("QuestionOptionsGenFlatMap")
}
return standardQuestion
}
func generateOptions() gopter.Gen {
optionsGen := func(genParams *gopter.GenParameters) *gopter.GenResult {
genParams.MinSize = 1
genParams.MaxSize = 10
presetGens := map[string]gopter.Gen{
"Value": gen.AnyString().WithLabel("OptionsSliceValueGen"),
"Name": gen.AnyString().WithLabel("OptionsSliceNameGen"),
}
return gen.SliceOf(gen.Struct(reflect.TypeOf(schema.SelectOption{}), presetGens).WithLabel("OptionGen")).WithLabel("OptionsSliceGen")(genParams)
}
return optionsGen
}
Do you have any idea what might do this? I'll investigate further and keep adding comments, but thought it was time to share the bug or if it is my wrong doing. Keep in mind Go is new for me so if anything looks weird I apologise ๐
For example:
$ go test ./...
--- FAIL: TestRunnerParallelWorkers (3.02s)
runner_test.go:174: [0] expected result time to be positive number but got 0s
runner_test.go:174: [1] expected result time to be positive number but got 0s
runner_test.go:174: [2] expected result time to be positive number but got 0s
! always fail: Falsified after 0 passed tests.
ARG_0: 0
ARG_0_ORIGINAL (1 shrinks): 299854320
Elapsed time: 0s
! always fail: Falsified after 0 passed tests.
ARG_0: 0
ARG_0_ORIGINAL (1 shrinks): 299854320
Elapsed time: 0s
FAIL
FAIL github.com/leanovate/gopter 3.320s
ok github.com/leanovate/gopter/arbitrary 0.683s
ok github.com/leanovate/gopter/commands 0.739s
--- FAIL: TestOneConstOf (0.00s)
one_of_test.go:51: Not all consts where generated: map[string]bool{"four":true, "two":true}
--- FAIL: TestOneGenOf (0.00s)
one_of_test.go:51: Not all consts where generated: map[string]bool{"four":true, "three":true}
+ PtrOf: OK, passed 100 tests.
Elapsed time: 2.0028ms
--- FAIL: TestWeighted (0.01s)
weighted_test.go:33: Result 250 for A falls outside acceptable range 50, 150
weighted_test.go:33: Result 356 for B falls outside acceptable range 150, 250
weighted_test.go:33: Result 394 for C falls outside acceptable range 650, 750
FAIL
FAIL github.com/leanovate/gopter/gen 1.632s
ok github.com/leanovate/gopter/prop 0.197s
FAIL
Sometimes the only way to know what the next state should be is if you let the system tell you. So it would be nice to be able to have the Result of Run in the NextState callback. Right now I am modifying the state in the post condition.
For example in a CRUD application you can know all of the parameters except for the unique id that gets generated by the system.
Concrete use case:
command: create user
command: get user
Although this would be reasonably involved, it would be really nice if it were possible to generate strings from an instance of the RangeTable
struct from unicode
.
This seems like it would be reasonably difficult to do given the variable strides and variable width of the ranges in order to generate a uniform distribution. But seems like quite a useful feature for testing properties under certain conditions.
I will take a look to see if I can see an obvious way to implement it.
Test framework panics on false precondition. Example change to expose the panic can be seen in jamesd@d411fc1
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.