Execution time improvements about guix-install-action HOT 17 CLOSED

I don’t think we can solve this fundamental problem since Guix works best if you have a permanent store, whereas GitHub’s workers are designed to be ephemeral. If you want fast Guix-based building you’d better use Cuirass or a long-running machine with guix installed and just while true; do guix build …; sleep 1m; done.

I tried using the worker cache to store /gnu/store, but it does not work properly and usually it’s too big to be cached.

from guix-install-action.

Hmm, was not aware of Cuirass, will take a look. Thanks!

from guix-install-action.

Maybe we could cache merely the closure of the pulled Guix, or store a pack of Guix and unpack that instead of pulling every time?

from guix-install-action.

I’m not sure how you’d cache a closure of the pulled Guix. Would guix gc -R $(which guix) be enough?

Since we’re installing the lastest guix tarball from cuirass we could also offer to disable pulling completely for those, who are fine with a “reasonably recent” Guix. Afaik it uses the latest Guix package.

from guix-install-action.

I, as original author of this issue, recommend you to move to Guix-powered CI 🙂

from guix-install-action.

@tribals Realistically, how would that work? We don't have a hosted cuirass service. The appeal of using Github workflow is that we don't need to take care of the resources and infrastructure needed to deploy CI.

from guix-install-action.

@PromyLOPh not running "guix pull" each time is indeed what I'm thinking about. I don't think we can avoid the initial installation of Guix -- we still need to create build users, launch the daemon, etc. Starting from a more recent Guix could be enough to reduce setup time.

The Github cache lets us store up to 10GB. We cannot snapshot /gnu/store completely, but we might be able to store substitutes corresponding to our environment there, so instead of fetching them from ci.guix.gnu.org we'd import them from the cache. This fails gracefully as we would fall back to substitutes from ci.guix.gnu.org on a cache miss.

from guix-install-action.

Realistically, how would that work? We don't have a hosted cuirass service.

That is exactly the issue.

Bacisally, we have two options:

• hack aroud GitHub CI in order it to better support Guix
• use other project hosting service where Guix already supported, so integrating Cuirass (or something other) would be trivial.

I don't want to spent my time to make GitHub more Guix-friendly. So, only option is second option?

I don't think either.

Although, there is project hosting services who support Guix natively (https://capsul.org/, https://sr.ht/), only support is "you get that Guix machine, you do what you want to it". That means you need setup everything yourself - choose CI, install it, mantain it. (Funny thing is that with help of Guix, it will be simplier, but still). This leads us to ultimate reason of the problem: in the end you need some resources to run your CI. Clouds make you think they are "just there", but this is not the case. In the end, you need to run that executable, no matter where.

I prefer to turn it other way around. First, I set up CI on my localhost and make it "just work". So-mentioned Cuirass fits there nicely. And only then, I start to think how to move that setup to some "project hosting provider". At this time, I already have CI working, mostly with help of Guix. Replicating it to other machine is a matter of wrintig "operating system definiton". This way, when I start to looking for "publicly available project hosting provider", all I need is just "server runnig Guix system".

from guix-install-action.

I think this falls outside the scope of a project that provides a Github action to use Guix as part of a Github workflow.

from guix-install-action.

Exactly. But what do you want? Running CI, or running Github CI?

from guix-install-action.

Moving hosters is not an option for the project in question, so I do want Github actions. The alternative is not moving to another hoster but not using Github actions. (Our packages are already part of Guix, so we are using ci.guix.gnu.org, just not for day to day development.)

from guix-install-action.

@rekado This action is already using the most recent Guix available.

How could we “cache” substitutes? Set up an nginx reverse proxy for ci.guix.gnu.org, which caches file to disk and put that directory into GitHub’s cache?

from guix-install-action.

@PromyLOPh oh, I missed that. I guess when using the default channels we could do without running "guix pull", thus reducing setup time significantly.

Substitute caching does seem a little gnarly. Perhaps it would be easier to use "guix archive" to export an environment and cache/import that on subsequent uses.

from guix-install-action.

Another issue with caching is that composite actions cannot depend on other actions it seems. So every user of this action would have to implement his own caching (as already suggested by this action’s workflow).

edit: Actually, looking at the documentation it should be possible now: https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runsstepsuses I was looking at my old notes on why I never implemented this.

from guix-install-action.

Anyway, I added an option to disable pulling with v1.3.

from guix-install-action.

Thank you, this is very useful!

from guix-install-action.

@PromyLOPh this may be of interest wrt caching /gnu/store: https://github.com/rikhuijzer/cache-install

from guix-install-action.