Local repository / package cache about composer HOT 17 CLOSED

Added a prototype of cached downloads #62

from composer.

Any comments for prototype? Need some discussion to finish with this necessary feature

from composer.

And how does this cache know when to expire upon a package being changed?

from composer.

@CodeOtter If the commit reference is the same, it cannot be a different code. A package being changed means a different version number or a different reference for the branch, and they are used when generating the cache file name.

from composer.

Hrmm... so that means I have to make a brand new tag for every single code change, even in development?

from composer.

@CodeOtter No you don't. The reference for branches is the commit sha, which changes when you do a code change. See the dev versions listed on Packagist for instance. They display the reference: https://packagist.org/packages/symfony/symfony

from composer.

@Seldaek why was this one closed? Was an alternative solution implemented, or you're just rejecting the request? Does this not fit into the grand architectural vision of composer or something?

from composer.

@cognifloyd doesn't composer already cache downloaded packages? When I run composer, I often have the message "installing from cache", expect for VCS packages.

from composer.

@tristanlins That's what I thought too. However, I've only noticed this when installing versions of packages that have releases though (ie: packages that have zip archives associated with them). A quick search in the composer book shows that for git packages, only metadata is downloaded.

from composer.

@tristanlins said:

expect for VCS packages.

Exactly. During development, I want to rapidly create a bunch of fresh installations of my package. That means that for every test installation, I'm downloading the entire git repository over the network. That's very inefficient considering (A) I already have a local mirror of the repository, and (B) Composer already downloaded the entire repository 5 (or 50) times on my machine. I want composer to git clone to ~/.composer/[name of repo].git or something, and then just copy from there. Then composer only has to update the one git repo instead of hammering the open source repos that are kindly & generously hosted by one of the companies in the community. Just because someone is kindly hosting something without charging for access, does not give me the right to abuse their services. That's what composer is doing by constantly redownloading (a natural part of development) VCS repos.

from composer.

@cognifloyd yeah I definitely would love to see a feature where composer saves an archive of a bare copy of the repository into its cache. It makes sense...I just don't know what the core devs would think about this.

/cc @stof @Seldaek -- any thoughts?

from composer.

There are other issues regarding git caching. Zip caching has been done over a year ago, so I closed this since it didn't add any value. I went through over 300 issues yesterday and didn't really have the nerves to explain everywhere why what and cross-link things.

from composer.

@Seldaek I would like an explanation please. Is there an gissue somewhere where you've already explained this that I can read? Not only do I want to understand, I would like to help explain to others (not just on github) why composer does not, or should not, do this.

from composer.

Sorry @Seldaek, I didn't remember about the other issue (which I commented on).
Anyway, for posterity here are the relevant issues:
Issue: #1323
PR: #1338

from composer.

@chr0n1x there is an open PR for it, and I think overall it might be good but I never had enough time to try it in depth and complete it. Lots of work, not enough time, but we are getting there slowly.

from composer.

i also would prefer to be able to set a global cache path for repos, which is shared across projects to limit the transfered data and the time to build

from composer.

Not sure if this is the best place to comment, but it does seem to be the root of my current troubles.

I'm using Composer with Docker, which means I'll only ever be installing packages for this one project. When I run my build, Composer builds a local cache from source of every package (even when --prefer-dist is set). This takes a very long time and often fails due to Github throttling. To make matters worse, because I'm using Docker, this is going to happen every time I rebuild.

My main question is this: Does this caching system trigger additional downloads beyond what would be required for a minimal installation or am I misunderstanding something?

Thanks!

from composer.