Hi, im trying to create an ansible project using nix language. I have an entire directory of .nix files that are referencing each other through self, super, root and are structured in ansible compatible hierarchy. Now i would like to save these imported nix expressions to .yaml files in a new directory while keeping the hierarchy structure intact, so i can run ansible-playbook on it.

I was hoping there is some way to iterate over the files that haumena has loaded, with access to their relative filepath and the evaluated expression so i could do something like this (pkgs.formats.yaml {}).generate "path/filename.yml" nixexpr

Help much appreciated and thanks for your work on this project.

Hi Everyone,

first of all, thank you for this library. It provides me with a simple solution for what has annoyed me the most since I've started writing my NixOS config about 4 weeks ago. During that time, I've been trying to find a more ergonomic way to manage NixOS module files and I think I now managed to find a decent way using Haumea. If you are at all interested, the dotfiles are available here, with a small preliminary write-up of how it works, but it's by no means required to understand what follows.

With that as context, now to my actual issue/proposal:
For the project I wrote a custom loader, which modifies the self, super and root attributes such that their functionality is preserved in a way that is actually useful in the context of NixOS modules. For this loader I'm actually only interested in a "cursor" relative to the root, so e.g. [ "base" "feature" ] for a file at <load path>/base/feature.nix. I then use this cursor together with nixpkgs.lib.getAttrFromPath to navigate the attribute tree.

However, the API currently passes only the absolute loading path to a loader, so e.g. /nix/store/<hash>-source/<load path>/base/feature.nix. This leads to the awkward situation that one needs to reverse engineer the cursor from this absolute path. In general, this is tricky, especially considering that the information is already available within the code of this library, namely in tree.pov in load.nix.

The original code of src/load.nix in lines 78-85 looks currently as follows:

{
   content = fix (self:
     (head matches).loader
       # inputs arg
       (inputs // {
         inherit self;
         super = getAttrFromPath tree.pov (view tree);
         root = view tree;
       }) 
       # path arg
       (src + "/${path}"));
}

My current patch to Haumea just adds a third argument to the API in load.nix:

{
   content = fix (self:
     (head matches).loader
       # inputs arg
       (inputs // {
         inherit self;
         super = getAttrFromPath tree.pov (view tree);
         root = view tree;
       })
       # path arg
       (src + "/${path}")
       # cursor arg
       (tree.pov ++ [name]));
}

There are also probably other ways of approaching this, but this one is the most straight forward in my opinion. Since I have the patch already, I could easily extend this to a pull request, but I would probably need some help with testing, as I currently don't use any other loaders than my own and the default one.

I realize this might not be worth the break in the API or even a use case you want to support. But for me it would go a long way to increase the ease of use in these edge cases where you only want to do some relative changes within the structure of the loaded files.

Thank you again for your work, best regards.

This may be a bit of a controversial feature suggestion, but if we use the nix feature to declare __functor in attrs, we can make attribute sets callable. This way we can make the haumea flake itself callable. Since we have a go-to function that most people would want to be using with haumea.lib.load, we could make that the default function of the flake. If we write in the main flake:

outputs = { self, nixpkgs }: rec {
  # ...
  __functor = _system: args: self.lib.load args;
};

Then, in a using flake we could write:

outputs = { self, haumea, nixpkgs }: {
  lib = haumea {
    src = ./src;
    inputs = { inherit (nixpkgs) lib; };
  };
};

I feel similarly to https://discourse.nixos.org/t/haumea-filesystem-based-module-system-for-nix/26902/15, aka I have trouble making sense of the documentation. I find the readme great but the git book useless.
I had to look at @figsoda 's dotfiles to see how to use it.

I cloned the repo to get a better understanding and was surprised to see that the comments exist in the doc book but not in the code. Moving the doc to src/load.nix the doc for the load function with what it expects as input would help greatly IMO.

I am not sure whether this is a problem with how I originally set up my config, haumea, or a combination of the two. In certain spots, especially when referring to config I get infinite recursion just when migrating a line from my original config to a folder that haumea is loading. Oddly, this recurision is solved by lib.mkForce but it feels hacky.

Some examples:

Just specifying which package vscodium module should use.
Lyndeno/nix-config@4add99c

+ # FIXME: Why do I need mkForce to avoid recursion?
+ package = lib.mkForce pkgs.vscodium;

Referencing config to get the proper kernel.
Lyndeno/nix-config@c98095a

# Haumea file
+  # FIXME: For some reason mkForce is needed to prevent infinite recursion
+  kernelPackages = lib.mkForce config.boot.zfs.package.latestCompatibleLinuxPackages;

# Old file
-  boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;

And in this case, not even referencing config, just changing the kernel.
Lyndeno/nix-config#dda0cc2

# FIXME: For some reason mkForce is needed to prevent infinite recursion
 kernelPackages = lib.mkForce pkgs.linuxPackages_rpi4; # Raspberry pies have a hard time booting on the LTS kernel.

In all these cases, the original line worked fine before migrating it to my haumea layout. I am not sure if maybe I am loading the folder in a non-ideal way, or if the way haumea works makes recursion more likely with the way I am configuring my system. The most puzzling cases to me are the ones where I am just changing the kernel package or the vscode package, not something I would expect to cause recursion. I am wondering if anyone has any idea how to fix these errors without resorting to mkForce.

Maybe this is not the right spot to post this problem, but I thought since I am running into this when converting my configs with haumea, maybe someone here could provide valuable insight.

Thank you for the very nice tool! It is allowing me to clean and organize my configs very nicely.

Hi! I was trying to use a matcher and was confused by the

... ({ self, super, root, ... } -> Path -> a ) ...

everywhere in the doc. Only after I read the source did I realize that this is simply a loader. I believe it would be easier to define:

Loader<a> := { self, super, root, ... } -> Path -> a

... at the top of e.g. loaders and use that instead. Do you think this is a good idea? I am not sure if Loader<a> := is the nix way to annotate a new type, though; the angled brackets are simply borrowed from rust. Also, replacing some of the a with e.g. NixExpr would make it feel more descriptive to me.

If you all agree with this proposal but don't have time to implement this, I can try to submit a pull request, some time (I am very slow though, haha).

Hi there, I think I have a misunderstanding on how Haumea works.

I have:

tree src/.
src/.
├── archetype
│   └── workstation.nix
├── modules
│   ├── betterbird.nix
│   └── firefox.nix
└── suites
    └── office.nix

flake.nix:

{
<snipped>

  inputs = {

    haumea = {
      url = "github:nix-community/haumea/v0.2.2";
      inputs.nixpkgs.follows = "nixpkgs";
    };
<snipped>

  };

  outputs = inputs@{ self, nixpkgs, home-manager, nixos-hardware, haumea, ... }:
    with inputs;
    let
      nixpkgsConfig = { overlays = [ ]; };
     
    in {

      lib = haumea.lib.load {
        src = ./src;
        inputs = { inherit (nixpkgs) lib; };
      };

      nixosConfigurations = {
      <snipped>
      };
    };
}

Then for the workstation.nix I have something like:

{ config, pkgs, lib, ... }:
let cfg = config.archetype.workstation;
in {
  config = lib.mkIf cfg.enable {

    suites = {
      office.enable = true;
    };

  };
}

and for office.nix I have:

{ config, pkgs, lib, ... }:
let cfg = config.suites.office;
in {
  config = lib.mkIf cfg.enable {

    programs = {
      firefox.enable = true;
      betterbird.enable = true;
    };

  };
}

and just as an example, for firefox.nix:

{ config, pkgs, lib, ... }:
let
    cfg = config.programs.firefox;
in
 {
  config = lib.mkIf cfg.enable {
    environment.systemPackages = with pkgs; [ firefox ];
  };
}

So my impression was that all of my files in my src folder would be imported.

And then by enabling the configs I had created using lib.mkIf I would be able to enable the configurations I wanted.

So my flow is essentially:

flake.nix --> imports all files in src --> workstation.nix --> office.nix --> firefox.nix

But when I rebuild I get:

error: The option `archetype' does not exist. Definition values:
       - In `/nix/store/i9czzb50vfnr1l1y7l6jzm89f8xiwnzp-source/systems/rembot/config':
           {
             workstation = true;
           }

Which makes me think the auto importing works different than I thought. Any tips? Is there another way I should be calling the options?

and with regards to the src file, how many folders deep will nix files be imported? Just one layer deep?
Thank you for your time.

Besides using liftDefault?

For example, _1password-gui.

I will use default.nix for now just wondering if there is any other way to handle this?

Would it be possible to specify a list of directory names to be ignored while scanning?

It seems that the judgment of introducing the value of config in hoisted will cause a problem of infinite recursion encountered

Notice: This problem is only triggered in evalModules and when the value of config for conditional judgment is introduced at the beginning of _imports(hoisted APIs).

Issue output:

       … while calling the 'head' builtin

         at /nix/store/bwps706g6ywl1811r7bwv32py12129ci-source/lib/attrsets.nix:780:11:

          779|         || pred here (elemAt values 1) (head values) then
          780|           head values
             |           ^
          781|         else

       … while calling the 'foldl'' builtin

         at /nix/store/yf26s7734ql7q05zlq6nq4y5q3rjyhj7-source/lib/trivial.nix:63:8:

           62|     let reverseApply = x: f: f x;
           63|     in builtins.foldl' reverseApply val functions;
             |        ^
           64|

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: infinite recursion encountered

       at /nix/store/yf26s7734ql7q05zlq6nq4y5q3rjyhj7-source/lib/modules.nix:261:21:

          260|           (regularModules ++ [ internalModule ])
          261|           ({ inherit lib options config specialArgs; } // specialArgs);
             |                     ^
          262|         in mergeModules prefix (reverseList collected);

Env:

• lib.evalModule
• haumea.load

 haumea.lib.load {
=   1       src = ./recipes/nixosModules;
= 101       inputs = modulesArgs;
=   1       transformer = with haumea.lib.transformers; [
=   2         liftDefault
=   3         (hoistLists "_imports" "imports")
=   4         (hoistAttrs "_options" "options")
=   5       ];
=   6     };

Since this problem is slightly confusing, I provided several causes that will help you understand easily.
Cases:

{config,lib}:

_imports =  ([
{
  # worked
  services.emacs.test = config.services.emacs.enable;
}
]);
#  error: infinite recursion encountered
_imports =  lib.optionals config.services.emacs.enable ([ {}
]);

I want Haumea to automatically convert directoryToImport imports to directoryToImport.default imports IF the directory contains a default.nix.

Reason is as follows:

Let's say I have two types of directories:

1. Directories without default.nix in them
2. Directories withdefault.nix in them.

In the first scenario, it's a case where I want all files in the dir without question (which is why I started using Haumea in the first place). But the second scenario is more for when I want a more portable directory structure and/or want to be more precise in what modules I want to import in the dir (example in this case being NixOS configuration). This second scenario becomes a bit clunky to solve. I have to write directoryToImport.default to actually import the directory, which is unneccessary, ugly and removes the dynamic ability in going back and forth between importing a loose nix file and a directory without changing its file path.

How do I do this?

Is there a way to do the same by just using a haumea transformer and/or loader?

# My package definitions are organized in the same way as in nixpkgs.
# Given /some/long/descriptive/path/to/packageName/default.nix
# I would like the resulting attributes to be
{ packageName = "/some/long/descriptive/path/to/packageName/default.nix"; }

At the moment, the following code does it for me

{ inputs, ... }:

let
  inherit (inputs) haumea;
  inherit (inputs.nixpkgs-lib.lib)
    foldl'
    attrNames
    concatStringsSep
    isPath
    isString
    isAttrs
    mapAttrs'
    nameValuePair
    last
    splitString;

  flattenTree = tree:
    let
      recurse = sum: path: val:
        foldl'
          (sum: key: op sum (path ++ [ key ]) val.${key})
          sum
          (attrNames val);

      op = sum: path: val:
        let
          pathStr = concatStringsSep "." path;
        in
        if (isPath val || isString val)
        then (sum // { "${pathStr}" = val; })
        else
          if isAttrs val
          then (recurse sum path val)
          else sum;
    in
    recurse { } [ ] tree;

  importPackages = src:
    let
      attrTree = haumea.lib.load {
        inherit src;
        loader = haumea.lib.loaders.path;
        transformer = [ (_: v: v.default or v) ];
      };

    in
    mapAttrs'
      (k: v: nameValuePair (last (splitString "." k)) v)
      (flattenTree attrTree);
in
importPackages

Hello,

I would like to use haumea in inputs.home-manager.lib.homeManagerConfiguration { } for the modules attributes.

From:

inputs.home-manager.lib.homeManagerConfiguration {
  modules = [
    ./modules/module1
    ./modules/module2
  ];
}

to

inputs.home-manager.lib.homeManagerConfiguration {
  modules = inputs.haumea.lib.load { src = ./modules; };
}

I tried many things, but I have the feeling that it is not working.

Maybe I'm wrong? Can you help ?