nix-community / flake.nix Goto Github PK

In preparation for my require.nix talk, I implemented the functionality as it's own set of libraries following the conventions I was hoping to have in place. Now that there is clear interest in incorporating this kind of functionality in Nix once it's validated, we should just have a self-contained entry point (to be fetched by e.g. builtins.fetchGit) to handle implementing the specifications while we're in the prototype phase.

The initial POC for require.nix did not include any systematic way to do testing. We should figure out what test suites look like, whether they allow only pure evaluation or also builds or possibly even arbitrary runs, how to test failures, etc.

Clean up and formalize the type representation for type annotations on definitions. In particular, we probably want some way to distinguish types that are structurally identical in some cases.

In particular, the lock file format described during the talk does not support multiple imports (e.g. multiple nixpkgs sets) well.

It would be nice if we could expect various flakes to all be laid out in the same way (e.g. do we use a src directory, is there a common examples directory, etc.)

"Flakes" get mentioned a lot lately. I've read the contents of the branches of this repo and discussions in PRs, but I still don't understand what is the issue in the first place and where is the end goal of this?

According to @edolstra at NixOS/nixpkgs#51109 (comment) the idea behind flakes is to replace the NIX_PATH with them, according to other quotes general modularization of Nixpkgs. The specification and discussions here talk about versions and version resolution a lot. The specification itself looks like cabal/setup.py files for Nix.

Which takes me to my stupid question. So you have a bunch of packages, each has a bunch of versions and you want to make a consistent set. What is the problem with just using version control for that?

E.g. I store a bunch of git controlled checkout of Nixpkgs/SLNOS/other nix-y things (some of which follow channels, some of which don't) and a bunch of configuration.nix files referring to them under a single git controlled meta-repository, which is my NIX_PATH. If I need some constraint resolution I just write it in my nix files in the meta-repository, most of the time I just update git submodules, commit, bulild and test. Why would I want flakes in such a setup?

In my experience things are pretty tightly coupled everywhere and I usually wish more things were in a single repository, not less. E.g. I wish nixos-homepage was a part of nixpkgs (maybe not literally, but via some meta-repository where I could make a meta-commit changing both at the same time) because every time I think about editing it I need to change both.

Similarly, Stackage is essentially a big consistent repository made out of Hackage (that constantly breaks), TeXlive is a similar thing for TeX (made for similar reasons), etc etc. Nixpkgs is a similar thing trying to make a consistent set of everything that can be built with Nix. Nixpkgs itself is already more or less consistent. So why would I want to break Nixpkgs itself into a set of inconsistent repositories?

Sure, with plain version control meta-repository you can't simply transparently use multiple sets of compatible versions, but I don't really see the point of having that functionality most of the time anyway. Think Nixpkgs, most of the time you want a single version of every package, older versions, ideally, are temporary artifacts. Similarly, with version control, if you need a different set of versions, you can just add a new checkout to your meta-repository (or, if its really major, make a new branch in the meta-repository).

Introducing generic constraint solvers just complicates things, IMHO. I'm yet to see a single Cabal, Node or Python package with more than a handful of dependencies that correctly describes its own version requirements. If you really really need some constraint solving, why not make the meta-repository do it like I do in my setup?

/cc @shlevy @zimbatm

P.S. On meta-repository thing: to be even more extremist, let me say I dream of the time when I could have true transparency from Nix to the version control. Think stdenv.mkDerivation, I dream to have no src = fetchurl lines ever, but, essentially, src = ./source where ./source would be a git submodule (or similar) pointing to the source repository, but so that Nix wouldn't copy it into the store every time on eval. That is to say, I want "version controlled repository" to be a first-class citizen in Nix, and I want to share git objects (or similar) between version control and /nix/store.

We should settle on expectations for documentation of flakes during the prototyping phase. This should include things like

1. Should description strings be some kind of markup (docbook?)
2. What does the top-level flake documentation look like?
3. How do we give examples, both on a per-definition level and on a flake-wide level?
4. Should we have additional annotations (per flake, flake member, or definition) besides human-oriented description and type annotation?

etc.