Make Coverage report consistent about plumbing HOT 14 CLOSED

I'm looking into an incorrect coverage report on tektoncd/pipeline#2233. The complete build log is here: https://storage.googleapis.com/tekton-prow/pr-logs/pull/tektoncd_pipeline/2233/pull-tekton-pipeline-go-coverage/1241017694992470016/build-log.txt

Here's what I've observed so far:

1. go test -coverprofile complains about lots of syntax issues, seemingly related more to the version of Go than anything else. Example snippet:

2020/03/20 15:06:18 Error running 'go test -coverprofile ': error='exit status 2'; combined output='# github.com/tektoncd/pipeline/vendor/knative.dev/pkg/reconciler
vendor/knative.dev/pkg/reconciler/events.go:35:15: undefined: errors.Is
vendor/knative.dev/pkg/reconciler/events.go:51:15: undefined: errors.As
vendor/knative.dev/pkg/reconciler/events.go:86:9: undefined: errors.Is
# github.com/tektoncd/pipeline/vendor/github.com/docker/cli/cli/config
vendor/github.com/docker/cli/cli/config/config.go:109:18: undefined: os.UserHomeDir
# github.com/tektoncd/pipeline/vendor/github.com/google/go-containerregistry/pkg/logs
vendor/github.com/google/go-containerregistry/pkg/logs/logs.go:38:10: l.Writer undefined (type *log.Logger has no field or method Writer)
# github.com/tektoncd/pipeline/cmd/entrypoint
cmd/entrypoint/waiter.go:35: Errorf format %w has unknown verb w
# github.com/tektoncd/pipeline/vendor/github.com/jenkins-x/go-scm/scm
vendor/github.com/jenkins-x/go-scm/scm/issue.go:146:9: undefined: strings.ReplaceAll

2. The set of "concerned" files is confusing. The number of changed files in the PR is 10. The number of files reported by github also looks like 10. But the set of files reported by the tool is as follows:

2020/03/20 15:06:18 GetConcernedFiles(...) started
2020/03/20 15:06:19 github file #0: docs/taskruns.md, concerned=true
2020/03/20 15:06:19 github file #1: pkg/apis/pipeline/v1beta1/task_types.go, concerned=true
2020/03/20 15:06:19 github file #2: pkg/reconciler/taskrun/taskrun.go, concerned=true
2020/03/20 15:06:19 github file #3: pkg/reconciler/taskrun/taskrun.go, concerned=true
2020/03/20 15:06:19 github file #4: pkg/remote/oci.go, concerned=true
2020/03/20 15:06:19 github file #5: pkg/remote/oci.go, concerned=true
2020/03/20 15:06:19 github file #6: pkg/remote/resolver.go, concerned=true
2020/03/20 15:06:19 github file #7: test/builder/task.go, concerned=true
2020/03/20 15:06:19 github file #8: test/builder/task.go, concerned=true
2020/03/20 15:06:19 github file #9: test/remote.go, concerned=true
2020/03/20 15:06:19 GetConcernedFiles(...) completed

There are 10 of them but notice the duplicates, for example test/builder/task.go. Looking back at the PR it's clear to see where these dupes come from - they're files that have an _test.go file associated with them. It looks like this is intentional, but it's confusing none the less.

3. Right up until the very end of the coverage tool's log it appears to understand that there are 7 files needing coverage. But then it mysteriously decides to drop 6 of them. See this snippet of log:

7 files changed, reported by github:
docs/taskruns.md	pkg/apis/pipeline/v1beta1/task_types.go	pkg/reconciler/taskrun/taskrun.go	pkg/remote/oci.go	pkg/remote/resolver.go	test/builder/task.go	test/remote.go	

2020/03/20 15:06:20 
Finding joining set of changed files from profile[count=1] & github
checking if this file is in github change list: test/builder/task.go	Yes!	(Coverage low!)
End of Finding joining set of changed files from profile & github

This happens as it's building the report for the Github message. Appears to be that NewGroupChanges is dropping them based on some difference it infers in a "base list" and a "new list" but it's not clear why. I'm still trying to puzzle out what profile is, why it only has 1 file and why it has an influence on the coverage report.

from plumbing.

Recently happened in one more Pipeline PR # 2909.

The stats were produced for just 2 out of 13 go files:

tektoncd/pipeline#2909 (comment)

from plumbing.

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

from plumbing.

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

from plumbing.

@tekton-robot: Closing this issue.

from plumbing.

I've seen this happening again - slack post - so will reopen

/remove-lifecycle rotten
/reopen

from plumbing.

@jerop: Reopened this issue.

from plumbing.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

from plumbing.

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

from plumbing.

/remove-lifecycle rotten

This is still not fixed, noticed in a recent PR tektoncd/pipeline#3700 (comment)

PR #3700 is updating four files but includes coverage report for just one file (pkg/workspace/apply.go):

1. task_types.go
2. workspace_types.go
3. pkg/reconciler/taskrun/resources/apply.go
4. pkg/workspace/apply.go

from plumbing.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

from plumbing.

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

from plumbing.

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen with a justification.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/close

Send feedback to tektoncd/plumbing.

from plumbing.

@tekton-robot: Closing this issue.

from plumbing.