Giter Club home page Giter Club logo

goconvey's Introduction

SMARTY DISCLAIMER: Subject to the terms of the associated license agreement, this software is freely available for your use. This software is FREE, AS IN PUPPIES, and is a gift. Enjoy your new responsibility. This means that while we may consider enhancement requests, we may or may not choose to entertain requests at our sole and absolute discretion.

GoConvey is awesome Go testing

Build Status GoDoc

Welcome to GoConvey, a yummy Go testing tool for gophers. Works with go test. Use it in the terminal or browser according to your viewing pleasure.

GoConvey supports the current versions of Go (see the official Go release policy). Currently this means Go 1.16 and Go 1.17 are supported.

Features:

  • Directly integrates with go test
  • Fully-automatic web UI (works with native Go tests, too)
  • Huge suite of regression tests
  • Shows test coverage
  • Readable, colorized console output (understandable by any manager, IT or not)
  • Test code generator
  • Desktop notifications (optional)
  • Immediately open problem lines in Sublime Text (some assembly required)

You can ask questions about how to use GoConvey on StackOverflow. Use the tags go and goconvey.

Menu:

Installation

$ go install github.com/smartystreets/goconvey

Make a test, for example:

package package_name

import (
    "testing"
    . "github.com/smartystreets/goconvey/convey"
)

func TestSpec(t *testing.T) {

	// Only pass t into top-level Convey calls
	Convey("Given some integer with a starting value", t, func() {
		x := 1

		Convey("When the integer is incremented", func() {
			x++

			Convey("The value should be greater by one", func() {
				So(x, ShouldEqual, 2)
			})
		})
	})
}

Start up the GoConvey web server at your project's path:

$ $GOPATH/bin/goconvey

Then watch the test results display in your browser at:

http://localhost:8080

If the browser doesn't open automatically, please click http://localhost:8080 to open manually.

There you have it. As long as GoConvey is running, test results will automatically update in your browser window.

The design is responsive, so you can squish the browser real tight if you need to put it beside your code.

The web UI supports traditional Go tests, so use it even if you're not using GoConvey tests.

Just do what you do best:

$ go test

Or if you want the output to include the story:

$ go test -v

Check out the

  • GoConvey wiki,
  • GoDoc
  • and the *_test.go files scattered throughout this project.

Contributors

GoConvey is brought to you by SmartyStreets and several contributors (Thanks!).

goconvey's People

Contributors

aaasen avatar adrianduke avatar ajackson-cpi avatar akupila avatar aleksi avatar anpryl avatar bt avatar dancannon avatar dlogvinenko avatar e-max avatar ezk84 avatar m4rw3r avatar markbates avatar mdwhatcott avatar mholt avatar msabramo avatar msteinert avatar mtiller avatar onlyjob avatar paran01d avatar riannucci avatar ruzz311 avatar santosh653 avatar source-gisakulabs avatar srabraham avatar stanim avatar tchajed avatar thomasdezeeuw avatar wallclockbuilder avatar zeffron avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

goconvey's Issues

ShouldResemble output: add 2 more spaces after the word "Expected"

* /Users/mike/work/dev/liveaddress-zipapi/src/pkg.smartystreets.net/liveaddress-zipapi/index/index_test.go 
Line 65:
Expected '{0 [] [] blank Blank lookup (you must provide a ZIP Code and/or City/State combination).}'
to resemble '{0 [] [] invalid_zipcode Invalid ZIP Code.}'
(but it didn't)!

Expand README into a Wiki

The README is getting pretty long. Let's think about pulling most of it out into a wiki with more detail, split into different pages. It'd also be good to cover topics not already mentioned like set-up and clean-up. Maybe also how/why *testing.T only needs to be passed into the first/highest Convey() closure.

Add feature to ignore packages

From the web UI, we should be able to specify fully qualified package names to ignore in case large dependencies (or broken dependencies) start to get in the way or slow things down.

Websockets

Go provides a nice library for using websockets to do long-running communications with the browser (client). So, it looks like we could probably have a go http server that sets up a main "/" handler that serves the html page with some javascript that would setup websockets with another go handler which would serve json or something that would then get templated into the main html. Another goroutine in the server could use fsnotify to get notified of .go file changes in a specified directory and run tests recursively, grabbing (and transforming?) the output and sending it back to the client. Could that work?

Better assertions via oglematchers

github.com/jacobsa/oglematchers is a very thoroughly done project that tackles the assertion problem and in most cases does it much better than what I currently have. It shouldn't be too hard to integrate it with goconvey. The ogle branch has been created for this effort.

Are tests running twice for each save?

I'm not sure if I'm going crazy, but I think when I save a single file (just once!) in my watched directories, the page refreshes twice. Terminal output shows tests running twice, too...

Rename "server"

This is a nitpick I am sure -- but if you want goconvey to live nicely next to other binaries (often named "server") it would if the testing webserver didn't take such a core name under bin by default. Maybe goconvey-server or something?

Bind to loopback (127.0.0.1) for additional privacy

Although goconvey-server doesn't seem to allow an external user to do much other than monitor the status of your tests it would be nice to bind the server's port explicitly to 127.0.0.1 so that nobody beyond the current system, who you can presumably trust, can access the server.

Perhaps this can be done by extending the existing "-port" parameter or adding a new "-host" parameter.

Another option is to use an environment variable such as "GOHOST" (or perhaps a more common one) so that the fully-qualified host name can be set once and used with other tools such as sirnewton01/godev or sirnewton01/godbg that have web front-ends.

Ignore/Skip scope

Is there a difference between skip and ignore (and does it matter)?

Fails to find packages on Windows

I assume most Gophers are using Linux or FreeBSD but Windows is probably still a common platform.
As soon as the goconvey server is started anywhere in the GOPATH this results:


Build Failed
can't load package: package rojects/gocode/src/lib/matrix:
cannot find package "rojects/gocode/src/lib/matrix" in any of:

c:\go\src\pkg\rojects\gocode\src\lib\matrix (from $GOROOT) C:\Projects\gocode\src\lib\src\rojects\gocode\src\lib\matrix (from $GOPATH)

You can see that the path separator char '' on Windows is not being handled correctly. It's also incorrectly prepending the path from the root to the GOPATH directory to the package source directory in its attempt to find the package.
Also it's appending the GOPATH to GOROOT ??? - where no user source exists anyway.

Deleted folder not removed from watch list

After deleting a folder with tests the web UI reported that the build failed because it couldn't see the directory. This shouldn't happen because the watch list should be updated each time the tests are run...

Randomize execution of sibling scopes/resets

This is to drive out temporal couplings between tests. Technically, the setup and teardown methods should be enough to do this but you just never know...

I think this will be a command line option that is enabled by default and can be disabled (which is how some of the unit tests can avoid being broken by this new feature).

Ability to consolidate `So` statements with parent convey text

I'm wondering if--and this is a big i" depending upon the intention and best practices of the library--having the ability to condense the So statement down to a single line along with the parent Convey text. For example, it may be that there is a lot of assertion blocks that take the following form:

Convey("The value should be computed correctly", func() {
    So(a, ShouldEqual, b)
})

Would it be possible to consolidate to rewrite the following in this way or something like it?

So(a, ShouldEqual, b, "The value should be computed correctly")

Perhaps this pollutes the DSL too much and perhaps it's mixing the concerns--that of conveyance of the intention and the underlying assertions. The only possible benefit is the slight reduction in code.

Thoughts?

wiring.go is cluttered

Move the real code into another file. This will allow the documentation to stand but not clutter up the actual code.

Too many open files

fsnotify connects to lots of files and the bigger the watched scope, the more files. So, we should probably explain in the wiki that the user should set the file connection limit higher:

ulimit -n <really-high-number>

Improvements to the readme

  • Why not use := for assignment in top-level Convey scopes?
  • Why use SkipConvey instead of commenting skipped assertions?

Reporter targeted for goconvey-web

Will probably output a json file with:

  • a recursive structure for each scope which contains:
    • the title of the scope
    • the file and line # of the scope
    • the number of passed/failed/skipped assertions (and their line numbers)
    • any panics and their stack traces
  • general statistics across all tested packages

This json structure could then probably be templated using javascript.

Show if outer-most scope is not implemented.

Options:

  • Maybe we allow the tester to pass in null instead of a function to indicate that a scope is not yet implemented.
  • Or we could detect that no assertions were made (would there ever be a time where not having an assertion at the outer-most leaf was intended?).

Run each test package in a separate goroutine

When you have more than 3-4 packages in a project the web ui starts to get laggy--not because of the client side code, but because the server is running each test package in a synchronous, blocking fashion and not delivering the results to the client in a timely manner. Running each test package in a separate goroutine and aggregating the results would be killer.

This seems relevant:

Dependencies not installed using `go get`

$ go get github.com/smartystreets/goconvey/convey does not pull down the necessary dependencies. Neither does $ go get github.com/smartystreets/goconvey -- oglematchers and fsnotify and ogletest are not cloned down.

Also, running these tests on Go 1.1.1 linux 386 gives this result, a build failed:

can't load package: package e/matt/go: cannot find package ...

(My $GOPATH is "/home/matt/go")

When running the server in the gopath.

Debug print statements make the server (parser) crash hard.

When writing some tests today I inserted some debug print statements and it broke the server.

We aren't making any attempt at capturing output from programs or tests and so it gets in the way when trying to deserialize json output in the parser. I think we should encapsulate the json in some sort of unique opening and closing delimiters and count anything else as captured output, which the web GUI could then show in a collapsed area.

Changing root still watches outside new root

When root is adjusted, the server still holds on to watches completely outside the new root.

I've opened an issue with fsnotify.

We may end up rolling our own "fsnotify" that works like somewhat like the python idle.py script (not sure if it would perform across lots of packages though...).

Testing path should be mutable

This means messing with the GOPATH environment variable...

We also need to make sure the server adds ALL nested directories to the watch list.

Separation of concerns - test output generation and display

So, I know that I have to get from goconvey to HTML somehow. The html has to have statistics, stories, filename and line numbers for everything, etc...

Here are the touch points:

  • GoConvey (reporting package, "close to the metal", which is probably where all file/line info should come from...)
  • idle.py script (runs tests and grabs all output from go test)
  • gocoenvey-web HTTP server (reads output from goconvey/idle.py?)
  • AJAX (receives http response from goconvey-web and templates results into html?)

So the big questions are:

  • Where do aggregations and transformations take place?
  • What is the best way for goconvey to communicate with goconvey-web (file system?)
  • What is stored on the file system (if anything)?
  • Is goconvey-web going to read from an output file or could it more directly invoke go test (across a channel on a different goroutine)?

Ideally:

  • we wouldn't have to write a file between processes
  • The format received by the http server would be generic and resilient to UI (html) changes (they should be able to change independently).

Web UI should show test results for old-style go test functions.

Wouldn't it be a great selling-point if our web UI could
show some sort of results for plain old go test functions?
This would make it much more appealing to port an existing package
that uses go test style unit tests into a goconvey style package.
And, (selfish reason) it would allow us to use the web UI to develop
GoConvey itself, which uses the go test style to bootstrap itself.
Right now, the server doesn't support parsing regular go test output
(and would probably fail miserably). We can't really say we integrate fully
with go test if we fail on that kind of test output.

It would mean invoking go test with both the "-json" AND "-v":

exec.Command("go", "test", "-v", "-json").Output()

That way we would get the individual method names for each StoryReport.
Not sure yet if this would require any changes to the reporting package.

Exclusive Convey

Could you please add the ability the run only a specific convey? This would eliminate the need to use SkipConvey on other sections. Perhaps it could be called OnlyConvey or something similar.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.