bwplotka / bingo Goto Github PK

View Code? Open in Web Editor NEW

327.0 5.0 29.0 12.09 MB

Like `go get` but for Go tools! CI Automating versioning of Go binaries in a nested, isolated Go modules.

License: Apache License 2.0

Makefile 5.02% Go 92.54% Shell 2.45%

golang go gomodules modules versioning packages pin thanos reproducibility

bingo's People

Contributors

Stargazers

Watchers

bingo's Issues

Consider support .sum files for security purposes.

As per https://blog.golang.org/modules2019#TOC_5. sums can be important.

Support build tags

I have the problem of wanting to import a build tool that has a build tag tools.
https://github.com/observatorium/deployments/blob/f460944fe1a95ff062886c95f04a654d3c26910c/Makefile

Not sure how that is supported right now. I'll add it manually and overwrite the build command in the Makefile.

Consider (optionally) maintaining separate GOPATH for each installation (or at least use tmp one)

The reason is that I found Go modules really problematic if you want to revert or install a various version of the same tools where most of the projects do not use version properly (they do breaking changes in minor versions) it ends up being impossible (you can find buildable version). All until you remove the GOPATH/mod. All suddenly works. I think Go Module prefers already cached items? By not working I mean that Go Modules is always confused by ambigous packages and some other problems.

In Go 1.15 it's super easy: https://twitter.com/inancgumus/status/1266041878379737088 and for Go 1.14 we can use some top one.

Decisions to make here:

Am I right that using new $GOPATH/mod is sometimes necessary for correct use?
Should we always install / download module in separate mod dir? I find this tricky as it consumes extreme amount of disk space sometimes.
Should we always use tmp module dir? This will make our deps to never be cached anywhere
Should we give an option for user?

Warn if some variable is not used from .mk

Error using bingo with github.com/xo/xo

bingo version output: v0.4.3

go version output: go version go1.16.4 darwin/amd64

What happened:

From my project directory:

$ bingo get github.com/xo/xo@9a3ddc1
Bingo not used before here, creating directory for pinned modules for you at .bingo
Error: get command failed: get: xo.mod: getting github.com/xo/xo@9a3ddc1: fallback to local go mod cache resolution failed after go get failure: go: downloading github.com/xo/xo v0.0.0-20210416025017-9a3ddc1e1407
go get: github.com/xo/xo@none updating to
        github.com/xo/[email protected] requires
        github.com/xo/[email protected]: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /Users/panta/.go/pkg/mod/cache/vcs/076de485f6bfc7a8e444ab6a676f8b3bebc02186cfe92720dd78cec82eafcc82: exit status 128:
        ERROR: Repository not found.
        fatal: Could not read from remote repository.
        
        Please make sure you have the correct access rights
        and the repository exists.
: exit 1: no module was cached matching given package github.com/xo/xo

What you expected to happen:

I expected bingo to install github.com/xo/xo@9a3ddc1

How to reproduce it (as minimally and precisely as possible):

Run the command bingo get github.com/xo/xo@9a3ddc1.

Logs (use bingo get -v <thing you do> for verbose output):

$ bingo get -v github.com/xo/xo@9a3ddc1
getting target github.com/xo/xo@9a3ddc1 (module  )
exec 'go mod init -modfile=/Users/panta/prj/gameclub-tsportal-ng/.bingo/xo-e.tmp.mod _'
resolveInGoModCache: Found /Users/panta/.go/pkg/mod/cache/download/github.com/xo/xo/@v directory
panic: runtime error: slice bounds out of range [:12] with length 7

goroutine 20 [running]:
main.resolveInGoModCache(0xc0000b6370, 0x1, 0x0, 0x0, 0xc0000f3160, 0x1, 0x0)
        /Users/panta/.go/pkg/mod/github.com/bwplotka/[email protected]/get.go:513 +0x12a5
main.resolvePackage(0xc0000b6370, 0xc0000b6501, 0xc0000e86c0, 0x39, 0x122bdf0, 0xc0000b6500, 0x0, 0x0, 0xc0000f3160, 0x0, ...)
        /Users/panta/.go/pkg/mod/github.com/bwplotka/[email protected]/get.go:405 +0x18d
main.getPackage(0x122b588, 0xc0000b27e0, 0xc0000b6370, 0xc00009a900, 0xc0000ac180, 0x2c, 0x11ea289, 0x6, 0x0, 0x0, ...)
        /Users/panta/.go/pkg/mod/github.com/bwplotka/[email protected]/get.go:556 +0x5a8
main.get(0x122b518, 0xc0000b27e0, 0xc0000b6370, 0xc00009a900, 0xc0000ac180, 0x2c, 0x11ea289, 0x6, 0x0, 0x0, ...)
        /Users/panta/.go/pkg/mod/github.com/bwplotka/[email protected]/get.go:297 +0x1838
main.main.func2(0x122b518, 0xc0000b4780, 0xc00009a900, 0x0, 0x0)
        /Users/panta/.go/pkg/mod/github.com/bwplotka/[email protected]/main.go:156 +0x2d5
main.main.func5(0x0, 0x0)
        /Users/panta/.go/pkg/mod/github.com/bwplotka/[email protected]/main.go:225 +0xc4
github.com/oklog/run.(*Group).Run.func1(0xc0000b2600, 0xc0000b47c0, 0xc000098870)
        /Users/panta/.go/pkg/mod/github.com/oklog/[email protected]/group.go:38 +0x27
created by github.com/oklog/run.(*Group).Run
        /Users/panta/.go/pkg/mod/github.com/oklog/[email protected]/group.go:37 +0xbb

Anything else we need to know:

Get command fails on retract directive

Today I tried pinning ginkgo (using bingo v0.4.3, go1.16.5), they're using the retract directive in their go.mod file which seems to not work at the moment.

$ bingo get -l github.com/onsi/ginkgo/ginkgo

Error: get command failed: get: ginkgo.mod: getting github.com/onsi/ginkgo/ginkgo: parse target mod file [...]/github.com/onsi/[email protected]/go.mod: parse: [...]/github.com/onsi/[email protected]/go.mod:13: unknown directive: retract

When using -v:

getting target github.com/onsi/ginkgo/ginkgo (module  )
exec 'go mod init -modfile=[...]/.bingo/ginkgo-e.tmp.mod _'
exec 'go get -modfile=[...]/.bingo/ginkgo-e.tmp.mod -d github.com/onsi/ginkgo/ginkgo'
exec 'go env GOPATH'
Error: [...]/go/pkg/mod/github.com/onsi/[email protected]/go.mod:13: unknown directive: retract
parse
github.com/bwplotka/bingo/pkg/bingo.ParseModFileOrReader
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/pkg/bingo/mod.go:349
main.autoFetchReplaceStatements
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/get.go:645
main.getPackage
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/get.go:561
main.get
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/get.go:297
main.main.func2
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/main.go:156
main.main.func5
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/main.go:225
github.com/oklog/run.(*Group).Run.func1
        [...]/go/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
        [...]/sdk/go1.16.5/src/runtime/asm_amd64.s:1371
parse target mod file [...]/go/pkg/mod/github.com/onsi/[email protected]/go.mod
main.autoFetchReplaceStatements
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/get.go:647
main.getPackage
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/get.go:561
main.get
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/get.go:297
main.main.func2
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/main.go:156
main.main.func5
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/main.go:225
github.com/oklog/run.(*Group).Run.func1
        [...]/go/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
        [...]/sdk/go1.16.5/src/runtime/asm_amd64.s:1371
ginkgo.mod: getting github.com/onsi/ginkgo/ginkgo
main.get
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/get.go:298
main.main.func2
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/main.go:156
main.main.func5
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/main.go:225
github.com/oklog/run.(*Group).Run.func1
        [...]/go/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
        [...]/sdk/go1.16.5/src/runtime/asm_amd64.s:1371
get
main.main.func2
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/main.go:157
main.main.func5
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/main.go:225
github.com/oklog/run.(*Group).Run.func1
        [...]/go/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
        [...]/sdk/go1.16.5/src/runtime/asm_amd64.s:1371
get command failed
main.main
        [...]/go/pkg/mod/github.com/bwplotka/[email protected]/main.go:233
runtime.main
        [...]/sdk/go1.16.5/src/runtime/proc.go:225
runtime.goexit
        [...]/sdk/go1.16.5/src/runtime/asm_amd64.s:1371

I'm not sure of any other packages that use this directive to double check, but the error message seems pretty clear.

How to run installed tools?

After I install tools with bingo get what is the intended way to run them?

It would be nice if there was a command where given a tool name it would find the path to the installed tool and execute it.

Example of what this might look like:

bingo run golangci-lint run

Install instruction broken

bingo version output:

go version output:

$ go version
go version go1.15.2 linux/amd64

What happened:

Followed the README.md instruction:

$ go get -u github.com/bwplotka/bingo                                                                   
go: downloading github.com/bwplotka/bingo v0.4.2                                                              
go: github.com/bwplotka/bingo upgrade => v0.4.2                                                               
go: downloading github.com/efficientgo/tools/core v0.0.0-20210201220623-8118984754c2                          
go: downloading github.com/Masterminds/semver v1.5.0                                                          
go: downloading github.com/oklog/run v1.1.0                                                                   
go: downloading github.com/mvdan/sh v1.3.0                                                                    
go: downloading golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898                                       
go: downloading github.com/mvdan/sh v1.3.1                                                                    
go: github.com/pkg/errors upgrade => v0.9.1                                                                   
go: golang.org/x/mod upgrade => v0.4.2                                                                        
go: golang.org/x/xerrors upgrade => v0.0.0-20200804184101-5ec99f83aff1                                        
go: github.com/efficientgo/tools/core upgrade => v0.0.0-20210326193628-425a09c04e05                           
go: github.com/mvdan/sh upgrade => v2.6.4+incompatible                                                        
go: downloading golang.org/x/mod v0.4.2
go: downloading github.com/efficientgo/tools/core v0.0.0-20210326193628-425a09c04e05
go: downloading github.com/mvdan/sh v2.6.4+incompatiblego: downloading golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
go: finding module for package mvdan.cc/sh/expandgo: finding module for package mvdan.cc/sh/syntax
go: downloading golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550go: downloading mvdan.cc/sh v1.3.1
go: downloading golang.org/x/sys v0.0.0-20190624142023-c5567b49c5d0go: downloading mvdan.cc/sh v2.6.4+incompatible
go: found mvdan.cc/sh/expand in mvdan.cc/sh v2.6.4+incompatible
go: found mvdan.cc/sh/syntax in mvdan.cc/sh v2.6.4+incompatiblego: golang.org/x/sys upgrade => v0.0.0-20210514084401-e8d321eab015
go: golang.org/x/sync upgrade => v0.0.0-20210220032951-036812b2e83c
go: golang.org/x/crypto upgrade => v0.0.0-20210513164829-c07d793c2f9a
go: downloading golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
go: downloading golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a
go: downloading golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1
go: downloading golang.org/x/sys v0.0.0-20210514084401-e8d321eab015
go: golang.org/x/term upgrade => v0.0.0-20210503060354-a79de5458b56
go: downloading golang.org/x/term v0.0.0-20210503060354-a79de5458b56
# github.com/bwplotka/bingo/pkg/envars
../../go/pkg/mod/github.com/bwplotka/[email protected]/pkg/envars/eval.go:27:12: undefined: "github.com/mvdan/sh/sy
ntax".Parse
../../go/pkg/mod/github.com/bwplotka/[email protected]/pkg/envars/eval.go:61:23: unknown field 'Context' in struct
literal of type interp.Runner
../../go/pkg/mod/github.com/bwplotka/[email protected]/pkg/envars/eval.go:61:37: unknown field 'File' in struct lit
eral of type interp.Runner
../../go/pkg/mod/github.com/bwplotka/[email protected]/pkg/envars/eval.go:61:46: cannot use envSlice (type []string
) as type expand.Environ in field value:
        []string does not implement expand.Environ (missing Each method)
../../go/pkg/mod/github.com/bwplotka/[email protected]/pkg/envars/eval.go:62:18: not enough arguments in call to ru
.Run
        have ()
        want (context.Context, "mvdan.cc/sh/syntax".Node)

What you expected to happen:

Command to complete without error.

How to reproduce it (as minimally and precisely as possible):

As above

Logs (use bingo get -v <thing you do> for verbose output):

Anything else we need to know:

Thank you for the effort you have put in to making bingo and for making it open source.

We are new to go, so perhaps have done something silly?

NoMeta error getting packages with v>0.2.2

Hello,
I seem to be getting NoMeta errors whenever running bingo get (for any package) with bingo v0.2.2 or greater.
See below for steps to reproduce:

# docker run -it golang:1.14.2-buster /bin/bash
# go version
go version go1.14.2 linux/amd64
# export GO111MODULE=on && export GOBIN="$HOME/.bin" && export PATH="$GOBIN:$PATH"
# go get -u github.com/bwplotka/[email protected]
# bingo get github.com/golangci/golangci-lint/cmd/golangci-lint
Bingo not used before here, creating directory for pinned modules for you at .bingo
go: downloading github.com/golangci/golangci-lint v1.27.0
go: found github.com/golangci/golangci-lint/cmd/golangci-lint in github.com/golangci/golangci-lint v1.27.0
go: downloading github.com/spf13/cobra v0.0.5
go: downloading github.com/fatih/color v1.7.0
go: downloading github.com/mitchellh/go-homedir v1.1.0
go: downloading github.com/go-critic/go-critic v0.4.1
go: downloading github.com/go-lintpack/lintpack v0.5.2
go: downloading github.com/spf13/pflag v1.0.5
...
Error: get command failed: get: 0: getting : get direct package: bingo: Go Module without bingo's metadata


# bingo get -v github.com/golangci/golangci-lint/cmd/golangci-lint
exec 'go mod init -modfile=/go/.bingo/golangci-lint.tmp.mod _'
go: found github.com/golangci/golangci-lint/cmd/golangci-lint in github.com/golangci/golangci-lint v1.27.0
exec 'go get -modfile=/go/.bingo/golangci-lint.tmp.mod -d github.com/golangci/golangci-lint/cmd/golangci-lint'
exec 'go list -modfile=/go/.bingo/golangci-lint.tmp.mod -f={{.Name}} github.com/golangci/golangci-lint/cmd/golangci-lint'
Error: bingo: Go Module without bingo's metadata
github.com/bwplotka/bingo/pkg/bingo.init
        /go/src/github.com/bwplotka/bingo/pkg/bingo/mod.go:40
runtime.doInit
        /usr/local/go/src/runtime/proc.go:5414
runtime.doInit
        /usr/local/go/src/runtime/proc.go:5409
runtime.main
        /usr/local/go/src/runtime/proc.go:190
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373
get direct package
main.getOne
        /go/src/github.com/bwplotka/bingo/get.go:265
main.get
        /go/src/github.com/bwplotka/bingo/get.go:173
main.main.func2
        /go/src/github.com/bwplotka/bingo/main.go:140
main.main.func5
        /go/src/github.com/bwplotka/bingo/main.go:237
github.com/oklog/run.(*Group).Run.func1
        /go/src/github.com/oklog/run/group.go:38
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373
0: getting
main.get
        /go/src/github.com/bwplotka/bingo/get.go:174
main.main.func2
        /go/src/github.com/bwplotka/bingo/main.go:140
main.main.func5
        /go/src/github.com/bwplotka/bingo/main.go:237
github.com/oklog/run.(*Group).Run.func1
        /go/src/github.com/oklog/run/group.go:38
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373
get
main.main.func2
        /go/src/github.com/bwplotka/bingo/main.go:149
main.main.func5
        /go/src/github.com/bwplotka/bingo/main.go:237
github.com/oklog/run.(*Group).Run.func1
        /go/src/github.com/oklog/run/group.go:38
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373
get command failed
main.main
        /go/src/github.com/bwplotka/bingo/main.go:245
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1373

Some files generated by bingo have the execution bit

bingo version output:
v0.4.0

go version output:
go version go1.16.2 linux/amd64

What happened:
Some files generated by bingo are created with the execution bit.

$ ls -lha .bingo/
total 32K
drwxrwxr-x 2 username username 4,0K abr  6 18:47 .
drwxrwxr-x 3 username username 4,0K abr  6 18:47 ..
-rwxrwxr-x 1 username username  116 abr  6 18:47 .gitignore........... ←
-rw-rw-r-- 1 username username  158 abr  6 18:47 golangci-lint.mod
-rwxrwxr-x 1 username username  148 abr  6 18:47 go.mod............... ←
-rwxrwxr-x 1 username username  820 abr  6 18:47 README.md............ ←
-rw-rw-r-- 1 username username  420 abr  6 18:47 variables.env
-rw-rw-r-- 1 username username 1,2K abr  6 18:47 Variables.mk

What you expected to happen:
Generated files don't have the execution bit when unneeded.

How to reproduce it (as minimally and precisely as possible):

Start new Go project,
Run bingo get github.com/golangci/golangci-lint/cmd/golangci-lint
Run ls -lha .bingo

Logs (use bingo get -v <thing you do> for verbose output):
N/A

Anything else we need to know:
N/A

Consider setting $GOPATH explicitly on Variables.mk

See: https://cloud-native.slack.com/archives/CL25937SP/p1591005765413900

Ensure better experience of upgrading tool that has need for `replace` statements in mod file.

Name conflict

The name bingo was previously used in the Go community for https://github.com/saibing/bingo, a very early LSP implementation.

Given that this project is now flagged deprecated as gopls matures, I don't think this is too much of an issue, only users that historically still have bingo binaries will be affected by this.

Feel free to close immediately, just a heads up :)

Use github.com/bwplotka/mdox/pkg/clilog for logging.

Similar to `replace` fetch `exclude` statements.

Kind of forgotten thing (:

Support Binaries with Non-Standard Names

bingo version output:

v0.3.0

go version output:

go version go1.15.3 linux/amd64

What happened:

Attempting to use bingo to fetch github.com/githubnemo/CompileDaemon failed

What you expected to happen:

The binary should be appended to the other tools installed via bingo get

How to reproduce it (as minimally and precisely as possible):

bingo get github.com/githubnemo/[email protected]
# or
bingo get github.com/githubnemo/CompileDaemon

Logs (use bingo get -v <thing you do> for verbose output):

getting target github.com/githubnemo/CompileDaemon (module  )
exec 'go mod init -modfile=/code/.bingo/CompileDaemon-e.tmp.mod _'
exec 'go get -modfile=/code/.bingo/CompileDaemon-e.tmp.mod -d github.com/githubnemo/CompileDaemon'
exec 'go env GOPATH'
Error: open /go/pkg/mod/github.com/githubnemo/[email protected]/go.mod: no such file or directory
read
github.com/bwplotka/bingo/pkg/bingo.ParseModFileOrReader
        /go/pkg/mod/github.com/bwplotka/[email protected]/pkg/bingo/mod.go:304
main.getPackage
        /go/pkg/mod/github.com/bwplotka/[email protected]/get.go:492
main.get
        /go/pkg/mod/github.com/bwplotka/[email protected]/get.go:293
main.main.func2
        /go/pkg/mod/github.com/bwplotka/[email protected]/main.go:156
main.main.func5
        /go/pkg/mod/github.com/bwplotka/[email protected]/main.go:247
github.com/oklog/run.(*Group).Run.func1
        /go/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1374
parse target mod file /go/pkg/mod/github.com/githubnemo/[email protected]/go.mod
main.getPackage
        /go/pkg/mod/github.com/bwplotka/[email protected]/get.go:494
main.get
        /go/pkg/mod/github.com/bwplotka/[email protected]/get.go:293
main.main.func2
        /go/pkg/mod/github.com/bwplotka/[email protected]/main.go:156
main.main.func5
        /go/pkg/mod/github.com/bwplotka/[email protected]/main.go:247
github.com/oklog/run.(*Group).Run.func1
        /go/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1374
CompileDaemon.mod: getting github.com/githubnemo/CompileDaemon
main.get
        /go/pkg/mod/github.com/bwplotka/[email protected]/get.go:294
main.main.func2
        /go/pkg/mod/github.com/bwplotka/[email protected]/main.go:156
main.main.func5
        /go/pkg/mod/github.com/bwplotka/[email protected]/main.go:247
github.com/oklog/run.(*Group).Run.func1
        /go/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1374
get
main.main.func2
        /go/pkg/mod/github.com/bwplotka/[email protected]/main.go:157
main.main.func5
        /go/pkg/mod/github.com/bwplotka/[email protected]/main.go:247
github.com/oklog/run.(*Group).Run.func1
        /go/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1374
get command failed
main.main
        /go/pkg/mod/github.com/bwplotka/[email protected]/main.go:255
runtime.main
        /usr/local/go/src/runtime/proc.go:204
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1374

Anything else we need to know:

My guess is that this issue is caused by the non-standard capital-casing of the package name. Looking at my /go/pkg directory, it looks like this:

$ ls ~/go/pkg/mod/github.com/githubnemo/ 
[email protected]

Normally package names are required to use lower-case names without special characters according to the package naming standards, but I guess that binaries don't necessarily follow this restriction. Rather, the packages are downloaded to the package name with ! preceding the non-standard characters.

I'm not sure if this corner case is worth pursuing since it probably only affects a small handful of Golang tools.

Consider moving to `go get -b` once ready.

Thanks @kakkoyun for sharing this:

https://go-review.googlesource.com/c/proposal/+/243077/2/design/30515-go-get-b.md

missing go.sum entry error

$ go version
go version go1.16.2 linux/amd64
$ mkdir testbingo
$ cd testbingo/
$ go get -u github.com/bwplotka/bingo
$ bingo get -l github.com/bwplotka/bingo
Bingo not used before here, creating directory for pinned modules for you at .bingo
Error: get command failed: get: bingo.mod: getting github.com/bwplotka/bingo: install: go: github.com/bwplotka/[email protected]: missing go.sum entry; to add it:
	go mod download github.com/bwplotka/bingo
: exit 1
$ go mod download github.com/bwplotka/bingo
go mod download: module github.com/bwplotka/bingo: not a known dependency

Add -ucommit to get latest commit

I always need it and always do that manually ): Let's automate!

Weird empty `go.mod` leftover in .bingo dir

We don't need this - remove
Add to .gitignore
Assert in tests

Incorrectly failing unsupported version check

It looks like the version check may be being a little overzealous, perhaps expecting a patch version where my installed version of Go reports none.

ja@vecna ~/git/galvatron/services (git)-[master] % bingo get github.com/go-bindata/go-bindata/...
Error: get command failed: found unsupported go version: go version go1.14 linux/amd64
. Requires go1.14.x
1 ja@vecna ~/git/galvatron/services (git)-[master] % go version                                                                                                                                                                                                                                                             :(
go version go1.14 linux/amd64
ja@vecna ~/git/galvatron/services (git)-[master] % head -n 5 go.mod
module rainbird.ai/galvatron

go 1.14

require (
ja@vecna ~/git/galvatron/services (git)-[master] %

go build errors after upgrading go from 1.15 to 1.16

bingo version output: v0.4.3

go version output: go version go1.16.5 linux/amd64

What happened:
The usage instructions list 2 different ways to use the tool:

go build -modfile .bingo/<tool>.mod -o=$GOBIN/<tool>-<version>
or
bingo get <tool>

We prefer to use the first way (go build) but it abruptly stopped working after we upgraded go from 1.15 to 1.16.
The second way does still work.

What you expected to happen:
go build -modfile .bingo/tool.mod -o=$GOBIN/<tool>-<version> will build the specified tool.

How to reproduce it (as minimally and precisely as possible):

λ bingo get [email protected]

λ cat .pprof.mod
module _ // Auto generated by https://github.com/bwplotka/bingo. DO NOT EDIT

go 1.16

require github.com/google/pprof v0.0.0-20180301010423-e8d30eee164b

λ go build -modfile ./pprof.mod -o=$GOBIN/pprof-foo
go: github.com/google/[email protected]: missing go.sum entry; to add it:
        go mod download github.com/google/pprof

Logs (use bingo get -v <thing you do> for verbose output):
Here are the logs from bingo get -v which does work, if they are helpful:

λ bingo get -v [email protected]
getting target github.com/google/[email protected] (module github.com/google/pprof )
exec 'go list -modfile=/home/ubuntu/co/backend/.bingo/pprof.tmp.mod -mod=mod -f={{.Name}} github.com/google/pprof'
exec 'go build -modfile=/home/ubuntu/co/backend/.bingo/pprof.tmp.mod -o=/home/ubuntu/co/backend/go/bin/pprof-v0.0.0-20180301010423-e8d30eee164b github.com/google/pprof'

Document Usage in go:generate Directives

In my Go project, I have some go generate directives that look like this:

//go:generate go run github.com/maxbrunsfeld/counterfeiter/v6 -generate

Unfortunately, this bypasses bingo's installation and pinning process. After a bunch of trial and error, I discovered that you can use use use the bingo-pinned version of the tool like so:

Makefile:

include .bingo/Variables.mk
export

gen: $(COUNTERFEITER)
    go generate ./...

Source Code:

//go:generate $COUNTERFEITER -generate

This might be a bit of a hack, but I wasn't able to find any other examples of using bingo tools in go:generate directives. This might be worth documenting for others who want to use bingo for their code-generation tools.

Can't install Go tool through bingo

bingo version output: v0.4.3

go version output: go version go1.16.3 linux/amd64

What happened:

➜  godoc git:(main) ✗ bingo get github.com/golang/tools/cmd/godoc
Bingo not used before here, creating directory for pinned modules for you at .bingo
Error: get command failed: get: godoc.mod: getting github.com/golang/tools/cmd/godoc: install: list: go: github.com/golang/[email protected]: parsing go.mod:
	module declares its path as: golang.org/x/tools
	        but was required as: github.com/golang/tools
: exit 1
➜  godoc git:(main) ✗ bingo get github.com/x/tools/cmd/godoc
Error: get command failed: get: godoc.mod: getting github.com/x/tools/cmd/godoc: fallback to local go mod cache resolution failed after go get failure: go get: module github.com/x/tools/cmd/godoc: reading https://proxy.golang.org/github.com/x/tools/cmd/godoc/@v/list: 410 Gone
	server response:
	not found: module github.com/x/tools/cmd/godoc: git ls-remote -q origin in /tmp/gopath/pkg/mod/cache/vcs/24db76dba5946ce3c81ae0780e53eb9e14e3680e100ebbc32b730eadb32b5601: exit status 128:
		fatal: could not read Username for 'https://github.com': terminal prompts disabled
	Confirm the import path was entered correctly.
	If this is a private repository, see https://golang.org/doc/faq#git_https for additional information.
: exit 1: no module was cached matching given package github.com/x/tools/cmd/godoc

What you expected to happen:

godoc installed

error cannot access runtime `cgo` lib during build time

As reported here: https://gophers.slack.com/archives/C01535DF3L2/p1595522898001500

go: found github.com/golangci/golangci-lint/cmd/golangci-lint in github.com/golangci/golangci-lint v1.27.0
exec 'go get -modfile=/Users/example/.local/src/github.com/caarlos0/promfmt/.bingo/golangci-lint.tmp.mod -d github.com/golangci/golangci-lint/cmd/[email protected]'
exec 'go list -modfile=/Users/example/.local/src/github.com/caarlos0/promfmt/.bingo/golangci-lint.tmp.mod -f={{.Name}} github.com/golangci/golangci-lint/cmd/golangci-lint'
go build runtime/cgo: copying /Users/example/Library/Caches/go-build/6c/6ca5db57043262ad56fd407655908d7c87beebaeab956fd7f09b67eec162187c-d: open /usr/local/go/pkg/darwin_amd64/runtime/cgo.a: permission denied
Error: error while running command 'go build -modfile=/Users/example/.local/src/github.com/caarlos0/promfmt/.bingo/golangci-lint.mod -i -o=/Users/example/.local/bin/golangci-lint-v1.27.0 github.com/golangci/golangci-lint/cmd/golangci-lint'; out: go build runtime/cgo: copying /Users/example/Library/Caches/go-build/6c/6ca5db57043262ad56fd407655908d7c87beebaeab956fd7f09b67eec162187c-d: open /usr/local/go/pkg/darwin_amd64/runtime/cgo.a: permission denied
; err: exit status 1
github.com/bwplotka/bingo/pkg/runner.(*Runner).exec
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/pkg/runner/runner.go:111
github.com/bwplotka/bingo/pkg/runner.(*Runner).execGo
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/pkg/runner/runner.go:96
github.com/bwplotka/bingo/pkg/runner.(*runnable).Build
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/pkg/runner/runner.go:207
main.getOne
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/get.go:273
main.get
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/get.go:173
main.getAll
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/get.go:63
main.get
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/get.go:107
main.main.func2
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/main.go:140
main.main.func5
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/main.go:237
github.com/oklog/run.(*Group).Run.func1
	/Users/example/.local/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1357
0: getting v1.27.0
main.get
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/get.go:174
main.getAll
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/get.go:63
main.get
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/get.go:107
main.main.func2
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/main.go:140
main.main.func5
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/main.go:237
github.com/oklog/run.(*Group).Run.func1
	/Users/example/.local/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1357
get
main.main.func2
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/main.go:149
main.main.func5
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/main.go:237
github.com/oklog/run.(*Group).Run.func1
	/Users/example/.local/pkg/mod/github.com/oklog/[email protected]/group.go:38
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1357
get command failed
main.main
	/Users/example/.local/pkg/mod/github.com/bwplotka/[email protected]/main.go:245
runtime.main
	/usr/local/go/src/runtime/proc.go:203
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1357

Handle reliably name clashes

Currently, if you install goimports and then you install totally different command with the same name (last segment in package path, or clashing -name), weird things can happen (most likely rewrite).

Let's avoid this with meaningful error and test cases to avoid this (:

Use local https://github.com/gomods/athens in tests to improve unit test timing.

Currently, it is on the edge of 10m. https://github.com/bwplotka/bingo/pull/27/checks?check_run_id=754080749

Warn about need for Go 1.14+

Advice for integrated versions

github.com/envoyproxy/protoc-gen-validate is an example of something used both as a binary (codegen) and a library. It is very version sensitive, so bingo version has to match go.mod. bingo is still nice as it pins the version in protoc. what advice do you have for coupled versions between bingo and go.mod?

`bingo get` breaks array versioning.

Something to fix and write tests for; help wanted.

Repro:
bingo get <some package>@<version1,version2>
bingo get

Now see array is broken into multiple things. We expect the version array to be maintained instead.

Main branch still called `master`, and linter broken for PRs

See

Error on sourcing .bingo/variables.env files

.bingo/variables.env is using local outside a function it causes an error on certain systems.

$ source .bingo/variables.env
.bingo/variables.env: line 4: local: can only be used in a function

Potential collaboration with gobin (:

See: myitcv/gobin#96

errorcapture error on bingo install

bingo version output: none, not installed yet

go version output: go version go1.15.5 darwin/amd64

What happened:

When I try to install bingo, I get the following error.

go get github.com/bwplotka/bingo      
# github.com/bwplotka/bingo/pkg/bingo
go/src/github.com/bwplotka/bingo/pkg/bingo/mod.go:86:4: undefined: errcapture.Close
go/src/github.com/bwplotka/bingo/pkg/bingo/mod.go:328:8: undefined: errcapture.Close

Indeed, the latest code in the errorcapture module doesn't have a Close func (there is a Do func). I see the bingo go.mod is targeting some specific commit of errorcapture, but maybe its not the right one? Not sure what's going on exactly.

What you expected to happen:

I expected to install bingo successfully.

Install location different between bingo get and makefile

The install location is different when installing tools with bingo get vs using the makefile variables. When running bingo get tools are installed at .bingo. When using the makefile variables, tools are installed at $GOPATH/bin. I'm wondering what the reason for this is? Would it be possible to make both methods install to the same location?

For context, I want to run bingo get in CI to install all required tools, and then cache all the built tools. However, if I then go to use them in my makefile they will be rebuilt, which defeats the point of caching them.

Weird log line: found malformed module file, removing

Found malformed module file, removing: /home/bwplotka/Repos/bingo/.bingo/mdox.mod - but actually nothing was removed 🔥 (:

bingo should build non versioned command for latest `get` as well.

For convenience! (:

Initial install in new env

First off, thank you for this tool!

I was curious what the best practice is for installing the initial bingo binary for a project with an existing bingo configuration?

My first thought (and the process described in the docs) was to just go get github.com/bwplotka/bingo in my project. This of course works, but subsequent go mod tidy runs (somewhat obviously) remove bingo from my project's go.mod file since it's not actually used in code anywhere.

I saw in the docs that you can use bingo to manage bingo itself :-) but when I go build using the module file in my project I get:

go: github.com/bwplotka/[email protected]: missing go.sum entry; to add it:
        go mod download github.com/bwplotka/bingo

Which I think is the same issue mentioned here, as I am also using go 1.16.

In the interim I've been installing bingo as an initial OS dependency w/ the recently added go install tool which works without issue and avoids adding extra entries to my go.mod file:

go install github.com/bwplotka/[email protected]

Apologies in advance if this is just a duplicate GitHub issue, but was wondering if I was maybe just thinking about this all wrong, and if so, if there's a documentation improvement I could submit to assist others.

Thanks!

Release binaries

Getting this is installed is problematic:

issue #84
issue #85

Is bingo suited to being installed as a single binary?

`-i` build flag can cause permissions issues for go installations owned by `root`

Various issues suggest that using the -i build flag is no longer necessary - e.g. golang/go#37962 (comment). Further, there is a pending issue that will deprecate the flag in go1.16, removing it in 1.17: golang/go#41696.

Would it be possible to remove that build flag to make the out-of-the-box experience with bingo get smoother?

For now I have worked around the issue with chown -R <user> /usr/local/go (or wherever the installation lives).

https://github.com/bwplotka/bingo/blob/master/pkg/runner/runner.go#L212

Go version doesn't update

bingo version output: v0.4.0

go version output: go version go1.16.3 darwin/amd64

What happened: updated Go to 1.16 from 1.15 and then bingo to v0.4.0 from v0.2.3 via bingo get -u github.com/bwplotka/bingo but .bingo/bingo.mod still reads:

module _ // Auto generated by https://github.com/bwplotka/bingo. DO NOT EDIT

go 1.15

require github.com/bwplotka/bingo v0.4.0

What you expected to happen: for the file to read go 1.16

How to reproduce it (as minimally and precisely as possible): Bump the go version of a project that already has Bingo in it. Then update one of the tool dependencies. The .mod file will have the previous go version still.

Logs (use bingo get -v <thing you do> for verbose output):

bingo get -v -u github.com/bwplotka/bingo
getting target github.com/bwplotka/bingo (module github.com/bwplotka/bingo )
exec 'go mod init -modfile=/Users/matiasanaya/code/src/github.com/kaddyteam/api/.bingo/bingo-e.tmp.mod _'
exec 'go get -modfile=/Users/matiasanaya/code/src/github.com/kaddyteam/api/.bingo/bingo-e.tmp.mod -d -u github.com/bwplotka/bingo'
exec 'go env GOPATH'
exec 'go list -modfile=/Users/matiasanaya/code/src/github.com/kaddyteam/api/.bingo/bingo.tmp.mod -mod=mod -f={{.Name}} github.com/bwplotka/bingo'
exec 'go build -modfile=/Users/matiasanaya/code/src/github.com/kaddyteam/api/.bingo/bingo.tmp.mod -o=/Users/matiasanaya/go/bin/bingo-v0.4.0 github.com/bwplotka/bingo'

Move testing to some fake package

Currently, we install real modules, let's craft something lightweight in git for test purposes instead. This will make test much faster and will not require things like #28

Add windows support

AC:

Add Github Action windows runner to test against windows
Decide how Makefile / variables.env should work on Windows platform, and adjust for windows (e.g. should we support PowerShell, any other requirements?)

Improve -u and -upatch options

Right now we rely on go get -u and -u=patch

The truth is that in practice the go get logic is totally surprising and illogical (to me).

Some examples:

v0.3.0 module is released.

If you do go get -u or go get -u=patch to module that was previously installed at v0.2.4 nothing literally change....

I think there is need for something better here. ):

Error running with Go pre-releases

bingo version output:

v0.3.1

go version output:

go version go1.16rc1 linux/amd64

What happened:

bingo quits with the following error, returned by *Runner:

Error: list command failed: Invalid Semantic Version

What you expected to happen:

The underlying error details are shared to ease troubleshooting, and
parseGoVersion does not fail to capture X.Y.Z string used to build the *semver.Version

How to reproduce it (as minimally and precisely as possible):

Install go1.16rc1
Run bingo list

Logs (use bingo get -v <thing you do> for verbose output):

Error: Invalid Semantic Version
get command failed
main.main
	/home/username/go/src/github.com/bwplotka/bingo/main.go:255
runtime.main
	/home/username/sdk/go1.16rc1/src/runtime/proc.go:225
runtime.goexit
	/home/username/sdk/go1.16rc1/src/runtime/asm_amd64.s:1371
exit status 1

Anything else we need to know:

I'd be happy to try to fix it. This initial attempt worked for me but feel free to suggest changes.

There may be more issues when using go1.16, e.g. I've seen this:

bingo get github.com/bufbuild/buf/cmd/[email protected]
Error: get command failed: get: buf.mod: getting github.com/bufbuild/buf/cmd/[email protected]: install: go: github.com/bufbuild/[email protected]: missing go.sum entry; to add it:
	go mod download github.com/bufbuild/buf
: exit 1

Remove need for fake `go.mod`

Unfortunately as per Go 1.14 --moddir feature: https://golang.org/doc/go1.14#go-flags:

A file named go.mod must still be present in order to determine the module root directory, but it is not accessed.

This means that we need to maintain in .bingo dir a fake empty go.mod. Let's create some issue on upstream why this is needed in Go itself and if we can remove that need.

Running bingo on air-gaped environments

For certain use cases, in our stack, we have to run air-gaped builds for our projects. This poses a problem when we want to use bingo on those projects.

Would you consider having some type of vendor directory support or module cache support?
How challenging Is it to implement?

-upatch and -u options does not work as expected; bring them back

We had to skip test even:

	{
			name: "Use -u=patch to upgrade thanos package",
			do: func(t *testing.T, g *goEnv, p *testProject) {
				if !goVersion.LessThan(version.Go116) {
					// TODO(bwplotka): Fix. It's not critical feature though to block release.
					t.Skip("From Go 1.16 behavior changed: get: can't query version \"patch\" of module github.com/thanos-io/thanos/cmd/thanos: no existing version is required")
				}
				fmt.Println(g.ExecOutput(t, p.root, bingoPath, "get", "--upatch", "thanos"))
			},
			expectRows: []row{
				// TODO(bwplotka) This will be painful to maintain, but well... improve it
				{name: "thanos", binName: "thanos-v0.21.1", pkgVersion: "github.com/thanos-io/thanos/cmd/[email protected]"},
			},
			expectBinaries: []string{"thanos-v0.21.1"},
		},

Next steps:

Potentially remove u and upatch options with something home-made that tries to get literally latest commit/tag of provided repo.

Complete install instructions

We find it nearly impossible to read the GIF embedded in README.md showing some bingo setup from scratch.

Squinting with tilted heads we think we can make out some variables being set before the install is attempted.

However in the written instructions there is no reference to env variables to be set?

Unable to get kustomize due to version tag

To reproduce:

bingo get sigs.k8s.io/kustomize/kustomize/[email protected]

The generated file becomes

V3 := $(GOBIN)/v3-v3.5.4
$(V3): .bingo/v3.mod
	@# Install binary/ries using Go 1.14+ build command. This is using bwplotka/bingo-controlled, separate go module with pinned dependencies.
	@echo "(re)installing $(GOBIN)/v3-v3.5.4"
	@cd .bingo && $(GO) build -modfile=v3.mod -o=$(GOBIN)/v3-v3.5.4 "sigs.k8s.io/kustomize/kustomize/v3"

How shall I change the name to the expected kustomize ? Thanks.

bwplotka / bingo Goto Github PK

bingo's People

Contributors

Stargazers

Watchers

Forkers

bingo's Issues

Decisions to make here:

Recommend Projects

Recommend Topics

Recommend Org