feature request: gocov test should accept multiple packages to test about gocov HOT 22 CLOSED

I'm working on this now, this issue merely advertises my intentions.

from gocov.

Great, thanks.

from gocov.

Andrew, can you give me some guidence on the following. Most of the code to expand go test ... style path globs is already present in the cmd/go tool. Does the gocov licence permit me to copy those parts into gocov and adjust as necessary ? An alternative would be to ask the symbols be exposed in Go 1.1, but that would make gocov only compatible with Go 1.1+

from gocov.

License wise there's no problem; just ensure correct attribution. Can you
point me at the code? I suspect it's probably not too complicated or
voluminous, so it's probably okay to copy in. If it means copying a heap of
code, then perhaps we could tag the current code as 1.0.3 and then require
1.1 to get new features.

On Tue, Jan 8, 2013 at 6:49 AM, Dave Cheney [email protected]:

Andrew, can you give me some guidence on the following. Most of the code
to expand go test ... style path globs is already present in the cmd/go
tool. Does the gocov licence permit me to copy those parts into gocov and
adjust as necessary ? An alternative would be to ask the symbols be exposed
in Go 1.1, but that would make gocov only compatible with Go 1.1+

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/21#issuecomment-11976056.

Andrew Wilkins
http://awilkins.id.au

from gocov.

License wise there's no problem; just ensure correct attribution. Can you

point me at the code? I suspect it's probably not too complicated or
voluminous, so it's probably okay to copy in. If it means copying a heap
of
code, then perhaps we could tag the current code as 1.0.3 and then require
1.1 to get new features.

Most of the glob code is in cmd/go/main.go. I've copied it into a separate
file (gocov/pkg.go) so we can adjust attribution as required. I'll have a
pull request ready to review soon.

from gocov.

This is what I have so far, it works, but needs cleaning up, pkg.go is
the important parts from cmd/go/main.go

davecheney@b2ca42e

On Tue, Jan 8, 2013 at 12:29 PM, Dave Cheney [email protected] wrote:

License wise there's no problem; just ensure correct attribution. Can you

point me at the code? I suspect it's probably not too complicated or
voluminous, so it's probably okay to copy in. If it means copying a heap
of
code, then perhaps we could tag the current code as 1.0.3 and then require
1.1 to get new features.

Most of the glob code is in cmd/go/main.go. I've copied it into a separate
file (gocov/pkg.go) so we can adjust attribution as required. I'll have a
pull request ready to review soon.

from gocov.

Just another vote for this issue. In understand it was under active development and then was frozen?

from gocov.

I think so - I guess Dave has been busy with other things. I don't have much time to dedicate to gocov at the moment, so if you (or anyone watching) feel like jumping in and Dave isn't working on this, please go ahead.

from gocov.

Yeah, this turned out the a non trivial problem as you can annotate more than one package at a time. Th second problem is feeding multiple annotation output files to the parser caused a huge panic, at that point this sort of went on the back burner.

On 27/05/2013, at 13:11, Andrew Wilkins [email protected] wrote:

I think so - I guess Dave has been busy with other things. I don't have much time to dedicate to gocov at the moment, so if you (or anyone watching) feel like jumping in and Dave isn't working on this, please go ahead.

—
Reply to this email directly or view it on GitHub.

from gocov.

May I suggest a workaround?

There are several problems with running gocov one package at a time:

• You don't count coverage due to tests from a different package.
• Since gocov uses JSON files, it isn't easy to concatenate them together to get a single report for your whole project.
• Gocov has a large setup time, which makes invoking separately for each package an expensive procedure.

Only the first problem requires annotating multiple packages (I think). Now, arguably each package should be self-sufficient, so this could be considered a feature :-) So, how about the following:

• Allow gocov test to accept multiple packages.
• Collect the data as if it was invoked independently, one package at a time - this should greatly simplify the patch, if I correctly understood the issues.
• However only do the expensive setup/teardown only once (creating/removing the temporary directories, etc.).
• Emit a unified JSON file containing all the collected packages.
• This may preclude using -deps, so be it.

from gocov.

@orenbenkiki If the packages are done individually, then the set of files to be instrumented differs for each package being tested. So you're going to have to keep repeating that expensive setup bit. I can't see the directory or symlink creation dominating (the fall back to copying in lieu of symlinks might, though).

I don't think running them all at once is going to be that problematic, it's just not as simple as running through each package in turn. I would just modify gocov test to:

• Accept multiple packages, instrumenting all the packages up front.
• Test each named package in turn, generating multiple coverage files.
• Parse each coverage trace file individually, and then merge the resultant JSON structures. There's code to do the merging already.

EDIT: Re-reading your comment, you may be suggesting the exact same thing as me?

from gocov.

@axw - Copying might explain why it takes less than 10 seconds to test my packages, but more than 2 minutes to collect their coverage :-( Anything I can do to ensure the "fast" setup is used? In fact, why is there a fall back to copying? Symbolic links work across file systems...

At any rate, yes, it seems like we are saying the same thing, up to not reusing all/some of the setup work between testing different packages.

from gocov.

@orenbenkiki Across file systems, but not all operating systems. But anyway, I was referring only to the portion of time spent in copying as a portion of the overall setup phase.

The test execution time can be significantly slowed down by the instrumentation, as gocov inserts another statement before each statement in the original code. If you supply "-v" to "gocov test" then you can eyeball how long code instrumentation vs. test execution is taking.

from gocov.

I don't know of any UNIX OS where symbolic links don't work across file systems... are you referring to Windows?

Anyway, I just run gocov -v on my 11 packages in my project and got:

Total time between running gocov test -v ... and seeing the "instrumenting" message: 42=19+1+1+1+2+1+8+4+2+2+1
Total time between "instrumenting" and "RUN Test": 45=9+3+2+3+3+6+5+3+3+5+3
Total time between "RUN Test" and "ok": 1=0+0+1+0+0+0+0+0+0+0+0

Quite the variability there, possibly having to do with NFS disk accesses?
BTW, this is a multi-core machine, I should use something like "make -j" to parallelize testing the different packages and then merge the final results. In theory this would have cut down the total time to 28 seconds instead of 88. If/when gocov can handle multiple packages, it should probably do the same thing.

from gocov.

I don't know of any UNIX OS where symbolic links don't work across file systems... are you referring to Windows?

Yes. Sorry, I may not have been very clear about what the "fallback" is. gocov will symlink all the files it needs into the temporary GOROOT, and then for any files that need to be instrumented it replaces their symlinks with instrumented regular files. For systems that don't support symlinking (e.g. Windows), we fall back to plain old copying.

Quite the variability there, possibly having to do with NFS disk accesses?

I'm not sure. Caching perhaps? This is getting slightly off topic. If you have some suggestions about how to lower overhead, please file a separate issue or send a pull request.

If/when gocov can handle multiple packages, it should probably do the same thing.

Sounds like a good idea to me.

from gocov.

In theory this would have cut down the total time to 28 seconds instead
of 88. If/when gocov can handle multiple packages, it should probably do
the same thing.

That is my intention, however it turned out to be harder than simply a
matter of typing.

On Tue, May 28, 2013 at 7:05 PM, Oren Ben-Kiki [email protected]:

I don't know of any UNIX OS where symbolic links don't work across file
systems... are you referring to Windows?

Anyway, I just run gocov -v on my 11 packages in my project and got:

Total time between running gocov test -v ... and seeing the
"instrumenting" message: 42=19+1+1+1+2+1+8+4+2+2+1
Total time between "instrumenting" and "RUN Test": 45=9+3+2+3+3+6+5+3+3+5+3
Total time between "RUN Test" and "ok": 1=0+0+1+0+0+0+0+0+0+0+0

Quite the variability there, possibly having to do with NFS disk accesses?
BTW, this is a multi-core machine, I should use something like "make -j"
to parallelize testing the different packages and then merge the final
results. In theory this would have cut down the total time to 28 seconds
instead of 88. If/when gocov can handle multiple packages, it should
probably do the same thing.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/21#issuecomment-18538868
.

from gocov.

Right, we drifted a bit off topic. Back to the original feature request - is it possible (with reasonable effort) to have gocov run multiple independent instances of itself "under the covers", ideally in parallel, then combine the resulting JSON files to emit final overall one. Right now one needs to do that "above" gocov (using Makefiles etc.). I guess one could also write a completely independent "gocov-parallel" tool to do that... but it would be cleaner/easier if it was directly integrated into gocov itself.

from gocov.

@orenbenkiki Yes, it's a reasonable thing to have built in, it's just a matter of someone finding the time to do it. There's a long weekend coming up here, so I may have some time then.

from gocov.

There's a couple of TODOs in gocov.go, but the changes don't make anything worse without doing the TODOs. I'll get to them later.

Support for "std" and "all" isn't all there, because we don't handle packages without tests. I'll open a new issue.

from gocov.

I just tried the latest version, works great. Thanks!

from gocov.

I am unable to get this feature working with the latest code. Am I doing something wrong?

>$ gocov test ./...
cannot use test profile flag with multiple packages
error: exit status 1

>$ gocov test myGopathedPkg/...
cannot use test profile flag with multiple packages
error: exit status 1

from gocov.

@dhoelle Support for this feature was dropped when gocov was rewritten on top of go test. You would currently need to work around it with scripting.

from gocov.