sgotti / glide-vc Goto Github PK

The glide.lock file may contain unneeded packages. This can happen with unneeded entries in glide.yaml or with not fine grained ones (for example having just github.com/coreos/etcd when only the github.com/coreos/etcd/client pkg is needed will have a glide.lock entry with a lot of etcd packages not needed by the project)

A way to get the real needed pkgs list will be to use the glide list json output. But, since we want to also keep the base package license files, there's the need to also get the base package path. Actually this isn't exported so a solution will be to recycle the glide functions that do this, but this is suboptimal.

Probably the nested vendor directories should be kept.

If so, how to clean them from unneeded packages/files?

Issue:

Projects sometime vendor main packages no directly imported by any code in the project. For example one of my projects vendors a go protoc plugin[0].

However, because these aren't imported directly, glide-vc removes them from the vendor directory,

Proposal:

If mains are explicitly declared as packages or sub packages, include them when determining if a package is imported.

[0] https://github.com/coreos/dex/blob/3c9ab50ddf05b07b74cda1777df6addf9ff44dfe/glide.yaml#L125

Ex given a glide.lock (not complete, trimmed to demonstrate):

hash: a92787bfb8b070e832ea4315aa20b5220ac3c530383ee531462dc75c1478c7fc
updated: 2016-08-29T11:04:56.268632692-07:00
- name: github.com/coreos/pkg
  version: 160ae6282d8c48a33d8c150e4e4817fdef8a5cde
  subpackages:
  - flagutil

Glide-vc will clean all things that are not
github.com/coreos/pkg/flagutil as expected:

glide vc --dryrun --only-code | grep coreos\/pkg
Removing unused file: github.com/coreos/pkg/.gitignore
Removing unused file: github.com/coreos/pkg/.travis.yml
Removing unused file: github.com/coreos/pkg/CONTRIBUTING.md
Removing unused file: github.com/coreos/pkg/DCO
Removing unused file: github.com/coreos/pkg/MAINTAINERS
Removing unused file: github.com/coreos/pkg/README.md
Removing unused file: github.com/coreos/pkg/build
Removing unused dir: github.com/coreos/pkg/capnslog
Removing unused dir: github.com/coreos/pkg/cryptoutil
Removing unused dir: github.com/coreos/pkg/dlopen
Removing unused dir: github.com/coreos/pkg/health
Removing unused dir: github.com/coreos/pkg/httputil
Removing unused dir: github.com/coreos/pkg/multierror
Removing unused dir: github.com/coreos/pkg/netutil
Removing unused dir: github.com/coreos/pkg/progressutil
Removing unused file: github.com/coreos/pkg/test
Removing unused dir: github.com/coreos/pkg/timeutil
Removing unused dir: github.com/coreos/pkg/yamlutil

Given the following glide.lock I would expect every subpackage in github.com/coreos/pkg to be retained:

hash: a92787bfb8b070e832ea4315aa20b5220ac3c530383ee531462dc75c1478c7fc
updated: 2016-08-29T11:04:56.268632692-07:00
- name: github.com/coreos/pkg
  version: 160ae6282d8c48a33d8c150e4e4817fdef8a5cde

However, it won't keep any packages within github.com/coreos/pkg in this case:

glide vc --dryrun --only-code | grep coreos\/pkg
Removing unused file: github.com/coreos/pkg/.gitignore
Removing unused file: github.com/coreos/pkg/.travis.yml
Removing unused file: github.com/coreos/pkg/CONTRIBUTING.md
Removing unused file: github.com/coreos/pkg/DCO
Removing unused file: github.com/coreos/pkg/MAINTAINERS
Removing unused file: github.com/coreos/pkg/README.md
Removing unused file: github.com/coreos/pkg/build
Removing unused dir: github.com/coreos/pkg/capnslog
Removing unused dir: github.com/coreos/pkg/cryptoutil
Removing unused dir: github.com/coreos/pkg/dlopen
Removing unused dir: github.com/coreos/pkg/flagutil
Removing unused dir: github.com/coreos/pkg/health
Removing unused dir: github.com/coreos/pkg/httputil
Removing unused dir: github.com/coreos/pkg/multierror
Removing unused dir: github.com/coreos/pkg/netutil
Removing unused dir: github.com/coreos/pkg/progressutil
Removing unused file: github.com/coreos/pkg/test
Removing unused dir: github.com/coreos/pkg/timeutil
Removing unused dir: github.com/coreos/pkg/yamlutil

Using --keep has no useful effect here.

What license is this available under? Without a license it's technically not open source. Glide itself is under the MIT.

Thanks for putting this together. I know it will be useful.

I have a few thoughts that may help you along.

1. Don't strip non-Go files by default. Instead opt-in to that with a flag. The reason is those documents can contain license information. Sometimes it's in a license file. Sometimes it's in a readme. You never can be sure.

A good default for licenses is important. Take the BSD license. If someone removes the license and someone redistributes the parent project on GitHub that person is in violation of the BSD license. There are a bunch of things like that.

Would you be open to opting in on that rather than opting out? That way those who can do it, such as a proprietary codebase, can opt-in. It spares accidents for open source projects.

2. This project deletes things. So, can you add some testing in? Maybe a testdata directory and some tests. Maybe even cross platform (POSIX and Windows)? If you're looking for an example you can see what I did on rmvcsdir. I'm happy to talk more about this.

It's reassuring to know the tests make sure it works right and for those looking to use it.

3. If you're not going to keep the dependencies in the vendor/ folder, consider adding vendor/ to the .gitignore file. I was considering a pull request but I ran out of time when I was in my dev environment.

4. This is entirely optional but, glide has a plugin system like git. If this were named glide-vc (or something else) you could use the command glide vc when the plugin is installed. Just a thought. I'm not all that opinionated on this. I just like to raise awareness.

Nice work. I'm happy to see you working on this.

I know it's possible to not use this option, but it does cause problems in this case, so it might be worth thinking about how to deal with cgo.

tried this: glide vc --keep 'Makefile' '**/*.h' '**/*.cpp*'

but it still appears to be deleting my necessary cpp and such files...any hints?