robfig / glock Goto Github PK

(showing up as a yext internal issue)

Monorepo ref: 00ff9c4

glock sync yext && glock sync yext && glock save yext leads to a cannot find package error during the save operation. This works as expected prior to #33

When tests define their own package, i.e. lib_test, test dependencies are not found by the save command.

The problem lies with the usage of the go list command in save.go. At line 160, where testImportOutput is computed, the template uses the TestImports field, which only lists imports for in-package tests: it also needs to use the XTestImports field, which lists imports for outside-package tests.

This did not come up yet because no Glock save test defines pkg.importPath as ending with _test.

Presently glock only reads the currently checked-out commit, and it reports [OK] if there are uncommitted changes. Instead, it should report uncommitted changes as an error that the user should resolve

Hello & happy NYE!

glock is a very slick project. I was wondering why the choice of GLOCKFILE for the filename? why not, Glockfile ?

.. perhaps the filename can be case insensitive, with the following support:

• glock save github.com/my/project will generate GLOCKFILE as a default (to your preference) - as is. However, if a "glockfile" exists of any case, then it will write the contents in the filename of that case.
• glock sync will look for any "glockfile" type-case

I have a repo (git.encryptio.com/slime) which has many subpackage imports under vanity import paths. It takes nearly a minute to run glock save git.encryptio.com/slime. glock is sending remote requests for vanity URL metadata, yet the information used by glock save to generate its output is completely local (I've written a basic script to do this for me, it finishes in 0.3 seconds and generates bit-for-bit identical output.)

This seems to stem from a non-optimized use of vcs.go's repoRootForImportPath, which runs various remote verifications to ensure proper behavior for go get (which is completely irrelevant for glock save.)

Hello,

Having some trouble adding a dependency on googlemaps.github.io/maps. In a folder ("googlemapstest") in the root of my $GOPATH with a main.go as follows:

package main

import (
	"fmt"
	"googlemaps.github.io/maps"
)

func main() {
	fmt.Println(maps.AvoidFerries)
}

Running go get googlemaps.github.io/maps and glock save googlemapstest which work as expected. Then, attempting to glock sync:

admin$ glock sync googlemapstest
golang.org/x/net                                   66f0418ca412	[OK]
googlemaps.github.io/maps                          0a3b50a8f265	[OK]
failed to get: googlemaps.github.io/maps/internal

> go get -v -d googlemaps.github.io/maps/internal
(success)

> import googlemaps.github.io/maps/internal
no repo found: /Users/admin/repo/alpha/gocode/src/googlemaps.github.io/maps/internal

I'm wondering if it isn't related to the import path (googlemaps.github.io/maps) being different that then underlying repo path (github.com/googlemaps/google-maps-services-go)?

The git hooks installed by glock install apply to pulls but not to rebases (consider git fetch; git rebase origin//master). A post-rewrite hook is needed for more comprehensive coverage. (For that matter, I'd also argue that switching branches with git checkout should trigger a glock sync)

I decided to go with a fresh checkout for the new command stuff:

$ glock save github.com/joeshaw/envdecode
open /Users/joeshaw/go/src/github.com/joeshaw/envdecode/GLOCKFILE: no such file or directory

If I touched GLOCKFILE before I ran, things worked fine.

(As an aside, it would be nice to run glock save . or glock save ./...)

Great tool you've made. I really like the single file listing and the cmd feature.
A problem I'm having is when I have to patch a 3rd party dependency (i.e. fork, fix and update).
To check out the right repo, I need to change the path in the GLOCKFILE and that will further necessitate a change in all my import paths. Anyway to alias the import path to the original library, even if I work with a fork?

(Warning: This is mostly a copy of a similar feature request filed for godep: tools/godep#43)

glock and other Go dependency management tools share a strong opinion about how the source tree of a project is managed. Our repository (and the repository of the last two startups I worked at) contains the entire source tree (and more). This allows us to have a company-wide Makefile that implements various policies and procedures useful to our workflow, and allows stuff related to the project to be stored in the same git repo (ex: Vagrantfiles). Also, we have a policy where all of our code must use the same dependencies. This layout is similar to Google's internal layout, or any other company that has a single-repository policy.

I recognize that this is not the "recommended" layout as described in the Go documentation but it is a layout that offers many benefits and is common in corporate environments.

In summary, our single GOPATH git repository structure:

/
/ops
/go      # this is $GOPATH
/go/src/company.com/proj1
/go/src/company.com/proj2

glock seems incompatible with this structure because it expects the .git directory to be at the same level of the package. However, in the example above, / contains .git and the src/company.com/proj1 directory does not.

I would be happy to use a single GLOCKFILE for all Go projects in the repository, but glock as implemented doesn't support this behavior.

Thanks.

Fetching a package via go get creates a repository clone or working directory (svn).
It would be nice to have an option for removing the .git, .svn and the like directories below $GOPATH/src to get something like this

cd $GOPATH/src && find . -name .git -print0 | xargs -0 rm -rf

If the $GOPATH is the project repository (see #13), this would allow the dependencies inside $GOPATH/src to be checked in just like vendoring.

If glock sync could then rerun go get, then doing the sync and then remove the .gitetc folders again a very nice workflow would be possible where not just the GLOCKFILE diffs are tracked inside the repo, but also the changes inside the 3rd party packages.

Also the main repo would be self contained.

I have a GLOCKFILE containing:

github.com/RangelReale/osin 2f2bec974bb61e7b0c47a054dd671a5b5d73fb72

I created a new $GOPATH and used glock to populate it:

github.com/RangelReale/osin                        0624bbd49aca [get checkout 2f2bec974bb6]

When I look in $GOPATH/src/github.com/RangelReale/osin it is the correct 2f2bec974bb6 revision. However, the library that was built seems to be from a different revision, I assume 0624bbd49aca, because my project no longer compiles due to an interface change.

When I delete $GOPATH/pkg/darwin_amd64/github.com/RangelReale/osin/osin.a and re-run glock sync or go get the correct version is built and my project compiles again.

This repository doesn't currently specify a license; please choose one and add it to the repository to make its legal status unambiguous. If you don't already have a preference I'd encourage you to choose the MIT or Apache licenses. See https://choosealicense.com/ for more about your options.

Thanks!

Just had a puzzling build failure where glock silently failed to install dependencies coming from code.google.com. After trying to manually install with go get, I realized mercurial wasn't installed on the build machine. glock should have complained about it.

From
https://groups.google.com/d/msg/golang-nuts/dc0hIrbWX_U/3nUD7Hb8Xq0J

Evidently, Git is fully supported by Glock.

After a quick scan through the source code, it looks like there is some incomplete support for other VCS. The vcs.go file mentions git, hg, bzr and svn.

However, I got the feeling that the hook install only works for Git. Is that correct?

Are you planning to add support for other VCS? (I'm using Mercurial as it happens)

Some dependencies will come in their own VCS other than Git. Is the rest of Glock (excluding the Git hook) going to work ok?

Perhaps the readme could summarise the current status.

Thanks,
Rick

I'm trying to get itsabot/abot setup but I am getting glock: command not found on step 3 of abot's setup:

You can install Abot via Go:

$ go get github.com/itsabot/abot
$ cd $GOPATH/src/github.com/itsabot/abot
$ cmd/setup.sh
$ abot server

When I run cmd/setup.sh I get this:

mikeumus:~/workspace/src/github.com/itsabot/abot (master) $ cmd/setup.sh
    [ok] checking for go binary
    [ok] checking GOPATH
    [ok] installing dependency manager
   [err] syncing dependencies

failed cmd:
         glock sync 'github.com/itsabot/abot'

bash: glock: command not found

This is likely some env var for go or glock that's off. I did run go get glock to no avail:

mikeumus:~/workspace $ go get github.com/robfig/glock
mikeumus:~/workspace $ glock
bash: glock: command not found

I'm attempting to get this setup in Cloud 9 IDE and the workspace is here:

• https://c9.io/mikeumus/abot-trial

Please feel free to provide me your Cloud9 username and I'll grant you access to the workspace.
Thank you for any feedback or ideas.

Glock sync fails when checking out a ref that corresponds to a tag.

If the repo already exists locally, glock sync only fetches origin which does not fetch any tags. This means if new tags are pushed to a repository and an entry in the glockfile is manually updated to the corresponding ref, glock sycn will fail.

It should be possible to make glock sync very fast in the case where everything is already up to date. That would reduce the cost of running it more frequently, for example as part of a build or deploy process.

Presently, it takes about 7 seconds for my config (from work) on a perfectly synced GOPATH.

After doing this:
go get github.com/robfig/glock

Terminal still says command not found.

I'm trying to install Abot.

I would like a Go dependency management tool to be able to handle installing specific versions of binaries. Is there a way to do this with glock?

For instance, I'd like to be able to lock a specific revision of github.com/jteeuwen/go-bindata/go-bindata. It is installed into $GOPATH/bin, and I can't import it directly into my code.

I've tried creating a deps.go file with // +build ignore at the top, but glock seems to ignore the file in that case. If I remove the build ignore flag it gets github.com/jteeuwen/go-bindata, but not the go-bindata binary. (Then my program also fails to build because I am trying to import something that isn't a package.)

If it's not possible, would you consider adding something like this to glock?

How does one update a dependency? It's a little ambiguous in the README - does one just glock save github.com/acme/project again to update a dependency?

In 1.5 the go command will read the environment variable GO15VENDOREXPERIMENT. If enabled, it will use the local "/vendor/" folder for dependencies. This replaces the need to manage GOPATH variables as dependencies can be fetched directly into the vendor folder and resolved there when built. This allows the GOPATH to not be modified. The contents of the vendor folder can be ignored.

Will you consider supporting this approach?

If I am working offline, I would expect to be able to trace back through previous history, even though I cannot fetch new deps from the internet.

This doesn't work because git fetch --ff-only is executed and if this fails, the sync fails.

However, the failure condition should be more nuanced. If the git fetch and the git checkout both fail, then glock should abort. But if the checkout succeeds, then the sync should proceed anyway.

Sometimes running glock sync fails:

Step 7 : RUN glock sync github.com/redacted/redacted
 ---> Running in 2baed2746ee2
�cannot find package "code.google.com/p/snappy-go" in any of:
    /usr/local/go/src/pkg/code.google.com/p/snappy-go (from $GOROOT)
    /gopath/src/code.google.com/p/snappy-go (from $GOPATH)
�time="2015-06-02T15:43:43-04:00" level=info msg="The command [/bin/sh -c glock sync github.com/redacted/redacted] returned a non-zero code: 1" 

The actual package it cannot find varies from failure to failure. I'd say it happens about 10% of the time across all of our Go repositories.

This comes from our continuous integration system, in which we build every commit within a GitHub pull request. We use Docker in the CI system to build a container that can later be pushed and deployed to production servers. The CI system runs on Linux.

I don't think I've seen this on my local development box, which runs OS X, but I also run glock sync very infrequently compared to the CI systems.

When glock is being called from a git hook, the env vars GIT_DIR and GIT_WORK_TREE are set.
However, if glock tries to go get a new dependency (which in turn tries to call git clone), those vars will cause go get to fail with a message that looks like this:

> go get -v -d github.com/mattn/go-isatty
github.com/mattn/go-isatty (download)
# cd .; git clone https://github.com/mattn/go-isatty /Users/david/code/go/src/github.com/mattn/go-isatty
fatal: working tree '.' already exists.
package github.com/mattn/go-isatty: exit status 128
exit status 1

As a workaround, unsetting the vars when calling glock avoids the issue:
env -u GIT_DIR -u GIT_WORK_TREE glock ...

EDIT: env -u isn't standard apparently. unset in a subshell works though: (unset GIT_WORK_TREE; glock...).

That said, running glock via git hook seems like it might be a common enough use-case that sanitizing those internally when calling go get makes sense.

Overview in README.md says: Glock is similar to "godep -copy=false". However, godep no longer supports the -copy=false parameter:

$ godep save -copy=false
godep: flag unsupported: -copy=false
Usage: godep save [-r] [packages]

Run 'godep help save' for help.

See also:

• https://github.com/tools/godep/blob/master/save.go#L65
• "Making godep save always copy code" proposal: https://docs.google.com/document/d/1Dxo9PHfNVETus0UCAuviysZZAr2Hv_vPBMK1-9jBU4M/edit#heading=h.x62avzkxk8v8
• tools/godep#118

It would be good to update the overview (possibly just remove the sentence and reference to godep).

Any chance that glock save could utilize standard out so we can pipe to an file of our choosing? Additional, during glock apply/install/sync operations, any chance it can read from stdin for the same reason?

For example:

glock save > .deps.txt
cat .deps | glock apply

If a user hasn't set up any githooks, the .git/hooks/ directory won't exist. Perhaps

If I have some conditional dependencies (i.e. ones that are defined in files that are build only on e.g. linux), and I run glock save on a system where those conditional dependencies are not required, my GLOCKFILE will not include all possible dependencies of my project.

This has implications when developing on OS X and running CI on linux.

A cursory look at the glock code reveals that we're currently shelling out to go list, which I don't think allows specifying build tags, but we should really be using the go/build package anyway, and specifying the union of (at least) all OS build tags when resolving dependencies.