uthar / nix-cl Goto Github PK
View Code? Open in Web Editor NEWUtilities for packaging ASDF systems using Nix
License: BSD 2-Clause "Simplified" License
Utilities for packaging ASDF systems using Nix
License: BSD 2-Clause "Simplified" License
i.e. introduction of ${lisp}Attrs
(program
,pkg
,flags
,faslExt
) instead of a lisp command and introduction of asdf
parameter
Probably can wait, because this api is extremely experimental as well, will probably go with lisp.pkgs
/lisp.withPackages
/lisp.buildASDFSystem
style.
How could we automate discovery of the libraries and programs from nixpkgs that each Lisp system depends on?
I suspect this will become more important over time if we want to support custom Lisp distributions (#25) because then we can't entirely rely on hand-crafted dependency declarations based on the latest Quicklisp e.g. on kons-9 they are forking a bunch of dependencies and requiring many new foreign libraries on the forks.
I don't know what the best approach is but here is a first idea that I'm toying around with. This doesn't involve screen-scraping the failing build logs (yet...)
Algorithm to discover dependencies of a Lisp package:
Builds with no libraries?
yes ->
Done: no dependencies.
no ->
Builds with "jumbo" set of all potential anticipated dependencies?
yes ->
Progress: we are able to build the package!
Recommended next step:
Iteratively search for minimal subset of packages that are truly needed.
no ->
Broken package: human needs to check for reason e.g.
- completely broken
- unsupported on the lisp/arch
- depends on unanticipated package from nixpkgs (must add to "jumbo" set)
- depends on something outside nixpkgs (must bundle somehow)
I'd like to test this idea by writing down a "jumbo" set of all the packages that Lisp systems might depend on, based on browsing build logs, and then see what effect this has e.g. does it make a lot more systems work.
If it does have a big effect then we could think about how to "shrink" the dependencies for each package so they don't all depend on everything. (Or if it doesn't improve the package situation much maybe we needn't bother.)
Here's the first tiny snippet of libraries that I've seen referenced in failing build logs:
ncurses, renderdoc, geoip, xorg.libX11, libcerf, libblas, libdrm, libfann, libfarmhash, glpk, gsl, libiio, tokyocabinet, leveldb, lmdb.
If the total ends up being less than ~50 that might be reasonable to maintain as our "jumbo" collection.
Could we get an update to the March dist update of Quicklisp? If I am not mistaken, the last update in this repo is from December. Thanks.
http://blog.quicklisp.org/2022/04/march-2022-quicklisp-dist-update-now.html
TODO: use :output-file argument to cl:compile-file to avoid having to specify faslExt
Hi,
eventually, we should add some metadata to the generated packages, like "description" and "license" ( anything else ? ). Currently, this is what I get with the old and the new package set:
nix-repl> lispPackages.cl-async.meta
{ available = true; broken = false; description = "Asynchronous operations for Common Lisp."; insecure = false; name = "lisp-cl-async-20211020-git"; outputsToInstall = [ "out" ]; position = ".../pkgs/development/lisp-modules/define-package.nix:115"; unfree = false; unsupported = false; version = "20211020-git"; }
nix-repl> lispPackages_new.sbclPackages.cl-async.meta
{ available = true; broken = false; insecure = false; name = "cl-async-20211020-git"; outputsToInstall = [ "out" ]; position = ".../pkgs/development/lisp-modules-new/imported.nix:6488"; unfree = false; unsupported = false; }
Metadata is also useful, if we want to be able to make the package set discoverable and eventually make them searchable on https://search.nixos.org.
For that though, we need to call recurseIntoAttrs
onto the package set that is bound to the lisp implementation, like sbclPackages
. As you can see in you PR NixOS/nixpkgs#172234, ofborg is not seeing anything to rebuild.
Seems like setup-hook should be somewhere else. This is how python works. It detects libraries when entering a dev-shell and puts them in PYTHONPATH (See toPythonPath
). Lisps only detect it in a build.
I am looking for a way to customize the set of Lisp package derivations before they are built. Is this supported in the API? Or else how might we support it?
For example I would like to add a failureHook
so that broken packages will be "built" into debug information.
I tried to do it like this:
let pkgs = import nixpkgs {};
baseLispPackages = pkgs.lispPackages_new.sbclPackages;
updateDerivation = d: d.overrideAttrs (o: {
failureHook = ''
...
'';
});
lispPackages = builtins.mapAttrs (n: v: updateDerivation v) baseLispPackages;
pkg = builtins.getAttr lispPackage lispPackages;
in
but I don't think this works because the modified derivations will include references to old unmodified definitions of their dependencies. Somehow I need to have the updated derivations all refer to each other.
I have a few ideas of how to approach this...
build-asdf-system
.callPackage
to resolve their dependencies before creating their derivation.But I'm not sure which approach makes sense so I'd be glad for some advice.
refactor everything to use generic hashes instead of hard coded sha256
It would be nice to run tests (asdf:test-system
) alongside each package to see if it works.
Now test systems are built separately, but with (asdf:load-system
), and it doesn't fit nicely into mkDerivation checkPhase
.
Maybe could default to a package/test
or package/tests
system name?
Thanks for the great work on this project!
I'm trying to use this to set up a small development environment for use from within Emacs. To that end, I've ended up with the code at the end of this post (but I'm not the most technically skilled when it comes to hacking on Nix or Emacs, and I'm completely new to CL), which works. With echo 'use flake' > .envrc
, I get a Nix shell within which I can evaluate
(require :hunchentoot)
(hunchentoot:start (make-instance 'hunchentoot:easy-acceptor :port 4242))
or similar in Emacs without issues.
However, I had some questions.
ps: ...
functions doesn't exactly seem idiomatic, but I also don't want to fork this repo and patch packages.nix
every time I want to use something that's not in the default package set. Is there an overlay feature?slynk-quicklisp
couldn't be loaded. I imagine that's of no use if I don't…use Quicklisp, which is the whole point of using nix-cl. Indeed, dismissing that condition and continuing seems to produce no ill effects. Is that right?slynk-stepper
package definition perhaps be added to nix-cl?{
description = "cl-dev";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
nix-cl.url = "github:Uthar/nix-cl";
};
outputs =
inputs@{
flake-utils,
nixpkgs,
nix-cl,
...
}:
flake-utils.lib.eachDefaultSystem (
system:
let
pkgs = import nixpkgs {
inherit system;
config.allowUnfree = true;
overlays = [ ];
};
sbcl = nix-cl.packages."${system}".sbcl;
local-slynk =
ps:
ps.slynk.overrideLispAttrs (o: {
systems = o.systems ++ [ "slynk/mrepl" ];
});
local-slynk-stepper =
ps:
sbcl.buildASDFSystem {
pname = "slynk-stepper";
version = "9999";
src = pkgs.fetchFromGitHub {
owner = "joaotavora";
repo = "sly-stepper";
rev = "da84e3bba8466c2290c2dc7c27d7f4c48c27b39e";
sha256 = "sha256-gMbmG42gxaQrgDUVySbGGpayxDM3pKAgDvYpd9KZ4B4=";
};
systems = [ "slynk-stepper" ];
lisp = sbcl;
lispLibs = [
(local-slynk ps)
ps.agnostic-lizard
];
};
local-sbcl-pkgs =
ps: with ps; [
hunchentoot
(local-slynk ps)
(local-slynk-stepper ps)
];
local-sbcl = sbcl.withPackages local-sbcl-pkgs;
in
{
devShells.default = pkgs.mkShell rec {
name = "cl-dev";
LD_LIBRARY_PATH = "${nixpkgs.lib.strings.makeLibraryPath buildInputs}";
buildInputs = with pkgs; [
local-sbcl
nixfmt-rfc-style
];
};
}
);
}
Hey is there a vision for how we can use nix-cl to define custom Lisp package distributions e.g. capturing the exact dependencies of a Lisp-based application?
I recently discovered a project called Qlot. This looks pretty convenient for writing a list of packages and having flexibility about where each one should be fetch from. Seems to also have functionality to generate a lock
file to capture exact versions. Maybe it would make sense to support building package sets from qlock
files in addition to Quicklisp distribution files?
I have been wondering about this in the context of kons-9 (kaveh808/kons-9#191) where the project started off using Quicklisp for everything but sometimes needs more specific versions of things. I wrote Nix derivations for the special dependencies by hand but that was a bit laborious e.g. writing down the list of dependent packages, writing down the list of systems, etc, which nix-cl automates so nicely when sourcing from quicklisp.
Is there an easy way to override the source of a Lisp package when it includes many sub-packages?
For example I'd like to replace the source to mcclim
but we have many derivations (mcclim
, mcclim-bezier
, ...) and they each have a separate src
attribute that references an external url/sha256. The best option I can think of is to override all of those derivations together.
It might be neat though if the sources themselves were a named derivation that could be overridden...? Or what's the least-bad way to do it today? :)
Will require un-hardcoding from LD_LIBRARY_PATH
. Maybe use DYLD_LIBRARY_PATH
depending on the system
currently that's done manually in nix code (flattenedDeps
)
normally, there are setup hooks in nixpkgs that e.g. add all jars from /share/java to the class path.
maybe a similar mechanism with share/lisp could be used to reduce the amount of hackery in nix code
These are broken as of 60dbe8f:
There's a patch to make them work:
From d7ea3d708bf0a430ac7e23b4f5cbff7160d4b8d0 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Kasper=20Ga=C5=82kowski?= <[email protected]>
Date: Sun, 9 Oct 2022 19:34:49 +0200
Subject: [PATCH 1/3] smokegen: init at v4.14.3
---
maintainers/maintainer-list.nix | 6 ++++++
.../libraries/smokegen/default.nix | 21 +++++++++++++++++++
pkgs/top-level/all-packages.nix | 2 ++
3 files changed, 29 insertions(+)
create mode 100644 pkgs/development/libraries/smokegen/default.nix
diff --git a/maintainers/maintainer-list.nix b/maintainers/maintainer-list.nix
index 33b6269fcbd..4ff47316875 100644
--- a/maintainers/maintainer-list.nix
+++ b/maintainers/maintainer-list.nix
@@ -13980,6 +13980,12 @@
githubId = 928084;
name = "Utku Demir";
};
+ uthar = {
+ email = "[email protected]";
+ github = "uthar";
+ githubId = 15697697;
+ name = "Kasper Gałkowski";
+ };
uvnikita = {
email = "[email protected]";
github = "uvNikita";
diff --git a/pkgs/development/libraries/smokegen/default.nix b/pkgs/development/libraries/smokegen/default.nix
new file mode 100644
index 00000000000..643ae1064db
--- /dev/null
+++ b/pkgs/development/libraries/smokegen/default.nix
@@ -0,0 +1,21 @@
+{ pkgs, lib, ... }:
+
+pkgs.stdenv.mkDerivation rec {
+ pname = "smokegen";
+ version = "v4.14.3";
+ src = pkgs.fetchzip {
+ url = "https://invent.kde.org/unmaintained/${pname}/-/archive/${version}/${pname}-${version}.tar.gz";
+ hash = "sha256-finsoruPeJZLawIjNUJ25Pq54eaCByfALVraNQJPk7c=";
+ };
+ buildInputs = [ pkgs.cmake pkgs.qt4 ];
+ buildPhase = ''
+ cmake .
+ '';
+ meta = with lib; {
+ description = "A general purpose C++ parser with a plugin infrastructure";
+ homepage = "https://invent.kde.org/unmaintained/smokegen";
+ license = licenses.gpl2Only;
+ platforms = platforms.unix;
+ maintainers = with maintainers; [ uthar ];
+ };
+}
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 0586a6575fd..3e40a0b4cec 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -4733,6 +4733,8 @@ with pkgs;
simg2img = callPackage ../tools/filesystems/simg2img { };
+ smokegen = callPackage ../development/libraries/smokegen {};
+
snazy = callPackage ../development/tools/snazy { };
snippetpixie = callPackage ../tools/text/snippetpixie { };
--
2.34.1
From fd3646f5177ec74430218f1f24c1bc40be4990a2 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Kasper=20Ga=C5=82kowski?= <[email protected]>
Date: Sun, 9 Oct 2022 19:36:54 +0200
Subject: [PATCH 2/3] smokeqt: init at v4.14.3
---
.../development/libraries/smokeqt/default.nix | 21 +++++++++++++++++++
pkgs/top-level/all-packages.nix | 2 ++
2 files changed, 23 insertions(+)
create mode 100644 pkgs/development/libraries/smokeqt/default.nix
diff --git a/pkgs/development/libraries/smokeqt/default.nix b/pkgs/development/libraries/smokeqt/default.nix
new file mode 100644
index 00000000000..145a9f243bb
--- /dev/null
+++ b/pkgs/development/libraries/smokeqt/default.nix
@@ -0,0 +1,21 @@
+{ pkgs, lib, ... }:
+
+pkgs.stdenv.mkDerivation rec {
+ pname = "smokeqt";
+ version = "v4.14.3";
+ src = pkgs.fetchzip {
+ url = "https://invent.kde.org/unmaintained/${pname}/-/archive/${version}/${pname}-${version}.tar.gz";
+ hash = "sha256-8FiEGF8gduVw5I/bi2wExGUWmjIjYEhWpjpXKJGBNMg=";
+ };
+ cmakeFlags = [
+ "-DCMAKE_CXX_STANDARD=98"
+ ];
+ buildInputs = [ pkgs.cmake pkgs.qt4 pkgs.smokegen ];
+ meta = with lib; {
+ description = "Bindings for the Qt libraries";
+ homepage = "https://invent.kde.org/unmaintained/smokeqt";
+ license = licenses.gpl2Only;
+ platforms = platforms.unix;
+ maintainers = with maintainers; [ uthar ];
+ };
+}
diff --git a/pkgs/top-level/all-packages.nix b/pkgs/top-level/all-packages.nix
index 3e40a0b4cec..6362b923c37 100644
--- a/pkgs/top-level/all-packages.nix
+++ b/pkgs/top-level/all-packages.nix
@@ -4735,6 +4735,8 @@ with pkgs;
smokegen = callPackage ../development/libraries/smokegen {};
+ smokeqt = callPackage ../development/libraries/smokeqt {};
+
snazy = callPackage ../development/tools/snazy { };
snippetpixie = callPackage ../tools/text/snippetpixie { };
--
2.34.1
From 422f1c24bd9ebb0c413d74bf11f28eb82441e525 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Kasper=20Ga=C5=82kowski?= <[email protected]>
Date: Sun, 9 Oct 2022 19:38:32 +0200
Subject: [PATCH 3/3] sbclPackages: fix build of qt, qt-libs and qtools
---
.../development/lisp-modules-new/packages.nix | 91 +++++++++++++++++++
.../patches/qt-libs-dont-download.patch | 36 ++++++++
.../patches/qtools-use-nix-libs.patch | 19 ++++
3 files changed, 146 insertions(+)
create mode 100644 pkgs/development/lisp-modules-new/patches/qt-libs-dont-download.patch
create mode 100644 pkgs/development/lisp-modules-new/patches/qtools-use-nix-libs.patch
diff --git a/pkgs/development/lisp-modules-new/packages.nix b/pkgs/development/lisp-modules-new/packages.nix
index 97c1cf0fca5..1ad06f615b5 100644
--- a/pkgs/development/lisp-modules-new/packages.nix
+++ b/pkgs/development/lisp-modules-new/packages.nix
@@ -11,6 +11,9 @@ let
optionals
hasSuffix
splitString
+ remove
+ optionalString
+ stringLength
;
# Used by builds that would otherwise attempt to write into storeDir.
@@ -42,6 +45,11 @@ let
# Patches are already applied in `build`
patches = [];
src = build;
+ # TODO(kasper): handle this with a setup hook
+ LD_LIBRARY_PATH =
+ build.LD_LIBRARY_PATH
+ + (optionalString (stringLength build.LD_LIBRARY_PATH != 0) ":")
+ + "${build}";
});
# A little hacky
@@ -332,6 +340,89 @@ let
version = "f19162e76";
});
+ qt = let
+ rev = "dffff3ee3dbd0686c85c323f579b8bbf4881e60e";
+ in build-with-compile-into-pwd rec {
+ pname = "commonqt";
+ version = builtins.substring 0 7 rev;
+
+ src = pkgs.fetchFromGitHub {
+ inherit rev;
+ owner = pname;
+ repo = pname;
+ hash = "sha256-GAgwT0D9mIkYPTHfCH/KxxIv7b6QGwcxwZE7ehH5xug=";
+ };
+
+ buildInputs = [ pkgs.qt4 ];
+ nativeBuildInputs = [ pkgs.smokegen pkgs.smokeqt ];
+ nativeLibs = [ pkgs.qt4 pkgs.smokegen pkgs.smokeqt ];
+
+ systems = [ "qt" ];
+
+ lispLibs = with ql; [
+ cffi named-readtables cl-ppcre alexandria
+ closer-mop iterate trivial-garbage bordeaux-threads
+ ];
+ };
+
+ qt-libs = build-with-compile-into-pwd {
+ inherit (ql.qt-libs) pname version src;
+ patches = [ ./patches/qt-libs-dont-download.patch ];
+ prePatch = ''
+ substituteInPlace systems/*.asd --replace ":qt+libs" ":qt"
+ echo "LD Path: $LD_LIBRARY_PATH"
+ '';
+ lispLibs = ql.qt-libs.lispLibs ++ [ qt ];
+ systems = [
+ "qt-libs"
+ "commonqt"
+ # "phonon"
+ # "qimageblitz"
+ # "qsci"
+ "qt3support"
+ "qtcore"
+ "qtdbus"
+ "qtdeclarative"
+ "qtgui"
+ "qthelp"
+ "qtnetwork"
+ "qtopengl"
+ "qtscript"
+ "qtsql"
+ "qtsvg"
+ "qttest"
+ "qtuitools"
+ # "qtwebkit"
+ "qtxml"
+ "qtxmlpatterns"
+ # "qwt"
+ "smokebase"
+ ];
+ };
+ commonqt = qt-libs;
+ qt3support = qt-libs;
+ qtcore = qt-libs;
+ qtdbus = qt-libs;
+ qtdeclarative = qt-libs;
+ qtgui = qt-libs;
+ qthelp = qt-libs;
+ qtnetwork = qt-libs;
+ qtopengl = qt-libs;
+ qtscript = qt-libs;
+ qtsql = qt-libs;
+ qtsvg = qt-libs;
+ qttest = qt-libs;
+ qtuitools = qt-libs;
+ qtxml = qt-libs;
+ qtxmlpatterns = qt-libs;
+ smokebase = qt-libs;
+
+ qtools = build-with-compile-into-pwd {
+ inherit (ql.qtools) pname version src nativeLibs;
+ lispLibs = [ qt ] ++ remove ql.qt_plus_libs ql.qtools.lispLibs ++ [ qt-libs ];
+ patches = [ ./patches/qtools-use-nix-libs.patch ];
+ };
+
};
in packages
diff --git a/pkgs/development/lisp-modules-new/patches/qt-libs-dont-download.patch b/pkgs/development/lisp-modules-new/patches/qt-libs-dont-download.patch
new file mode 100644
index 00000000000..e21a44c2a17
--- /dev/null
+++ b/pkgs/development/lisp-modules-new/patches/qt-libs-dont-download.patch
@@ -0,0 +1,36 @@
+--- a/qt-libs.asd
++++ b/qt-libs.asd
+@@ -17,5 +17,4 @@
+ :components ((:file "qt-libs"))
+ :depends-on (:qt-lib-generator
+ :cl-ppcre
+- :cffi)
+- :perform (asdf:load-op :after (op c) (uiop:symbol-call :qt-libs :ensure-standalone-libs)))
++ :cffi))
+--- a/qt-libs.lisp
++++ b/qt-libs.lisp
+@@ -94,16 +94,14 @@
+ standalone-dir)
+
+ (defun %ensure-lib-loaded (file)
+- (let ((file (etypecase file
+- (pathname file)
+- (string (installed-library-file file))))
+- (name (intern (string-upcase (pathname-name file))))
+- #+sbcl(sb-ext:*muffled-warnings* 'style-warning))
+- (cffi::register-foreign-library
+- name `((T ,file))
+- :search-path (to-directory file))
+- (unless (cffi:foreign-library-loaded-p name)
+- (cffi:load-foreign-library name))))
++ (let ((name (make-pathname :name (format nil "lib~a" file)
++ :type #+unix "so"
++ #+darwin "dylib")))
++ (or (find-if (lambda (lib)
++ (equal (cffi:foreign-library-pathname lib)
++ (namestring name)))
++ (cffi:list-foreign-libraries))
++ (cffi:load-foreign-library name))))
+
+ (defun ensure-lib-loaded (file)
+ (cond ((pathnamep file)
\ No newline at end of file
diff --git a/pkgs/development/lisp-modules-new/patches/qtools-use-nix-libs.patch b/pkgs/development/lisp-modules-new/patches/qtools-use-nix-libs.patch
new file mode 100644
index 00000000000..b380894b04a
--- /dev/null
+++ b/pkgs/development/lisp-modules-new/patches/qtools-use-nix-libs.patch
@@ -0,0 +1,19 @@
+Dont use the qt+libs system for managing Qt dependencies, because Nix provides
+them already.
+Don't build the deploy.lisp helper file, because Nix also can handle deployment.
+--- a/qtools.asd
++++ b/qtools.asd
+@@ -33,10 +33,9 @@
+ (:file "generate")
+ (:file "dynamic")
+ (:file "precompile")
+- (:file "deploy")
+ (:file "fast-call")
+ (:file "documentation"))
+- :depends-on (:qt+libs
++ :depends-on (:qt
+ :deploy
+ :cl-ppcre
+ :closer-mop
+
+Diff finished. Sun Oct 2 14:38:06 2022
\ No newline at end of file
--
2.34.1
be more in line with other things in nixpkgs
Glad my issue about reusing quicklisp-to-nix was noticed.
I have started writing an overlay for Common Lisp in December but there were questions I could not answer myself. Now I think that I overcomplicated stuff. Anyway, here are the things I could not decide on:
Wonder what your thoughts on these are.
Also, I thought that CL in nix should not depend on quicklisp, having its own package set. But earlier today I had a thought that that could have been too much.
For constructing CL_SOURCE_REGISTRY I had a function similar to the one I found in your code. I have just written a setup-hook to replace that here.
With this mess I wanted to generate package definitions without quicklisp.
Overall my nix knowledge is not really great, and I would love to contribute instead of writing from scratch.
I attempt to start a slynk server from nix-configuration only. For this I need the slynk-mrepl packages and others. When attempting to build them, I get the following. Is there any way I can help with this? Thank you.
$ nix-build https://github.com/Uthar/nix-cl/archive/master.tar.gz -A sbclPackages.slynk_slash_mrepl
this derivation will be built:
/nix/store/ypbnbsw51vp25wz160647wnjkf91sp52-slynk_mrepl-20220707-git.drv
building '/nix/store/ypbnbsw51vp25wz160647wnjkf91sp52-slynk_mrepl-20220707-git.drv'...
unpacking sources
unpacking source archive /nix/store/bn2hyl4659b8rj24i9xp1jscnv7mcn0f-source
source root is source
patching sources
configuring
no configure script, doing nothing
building
SLYNK's ASDF loader finished.
Unhandled SB-INT:SIMPLE-FILE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
{1004C181B3}>:
Error opening #P"/nix/store/wxcdkva1kp8isnadn5rvgp6rvlaayg55-slynk-20220707-git/contrib/slynk-mrepl-tmpGHU3ALSV.fasl":
Permission denied
Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {1004C181B3}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<SB-INT:SIMPLE-FILE-ERROR "Error opening ~S" {10033A7E33}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK *INVOKE-DEBUGGER-HOOK* #<SB-INT:SIMPLE-FILE-ERROR "Error opening ~S" {10033A7E33}>)
2: (INVOKE-DEBUGGER #<SB-INT:SIMPLE-FILE-ERROR "Error opening ~S" {10033A7E33}>)
3: (ERROR SB-INT:SIMPLE-FILE-ERROR :PATHNAME #P"/nix/store/wxcdkva1kp8isnadn5rvgp6rvlaayg55-slynk-20220707-git/contrib/slynk-mrepl-tmpGHU3ALSV.fasl" :MESSAGE "Permission denied" :FORMAT-CONTROL "Error opening ~S" :FORMAT-ARGUMENTS (#P"/nix/store/wxcdkva1kp8isnadn5rvgp6rvlaayg55-slynk-20220707-git/contrib/slynk-mrepl-tmpGHU3ALSV.fasl"))
4: (SB-IMPL::FILE-PERROR #P"/nix/store/wxcdkva1kp8isnadn5rvgp6rvlaayg55-slynk-20220707-git/contrib/slynk-mrepl-tmpGHU3ALSV.fasl" 13 "Error opening ~S" #P"/nix/store/wxcdkva1kp8isnadn5rvgp6rvlaayg55-slynk-20220707-git/contrib/slynk-mrepl-tmpGHU3ALSV.fasl")
5: (SB-IMPL::%OPEN-ERROR #P"/nix/store/wxcdkva1kp8isnadn5rvgp6rvlaayg55-slynk-20220707-git/contrib/slynk-mrepl-tmpGHU3ALSV.fasl" 13 NIL :CREATE)
6: (OPEN #P"/nix/store/wxcdkva1kp8isnadn5rvgp6rvlaayg55-slynk-20220707-git/contrib/slynk-mrepl-tmpGHU3ALSV.fasl" :DIRECTION :IO :ELEMENT-TYPE :DEFAULT :IF-EXISTS NIL :IF-DOES-NOT-EXIST :CREATE :EXTERNAL-FORMAT :UTF-8 :CLASS SB-SYS:FD-STREAM)
7: (UIOP/STREAM:CALL-WITH-TEMPORARY-FILE #<FUNCTION (FLET "BEFORE234" :IN UIOP/STREAM::GET-TEMPORARY-FILE) {5346555B}> :WANT-STREAM-P NIL :WANT-PATHNAME-P T :DIRECTION :IO :KEEP T :AFTER NIL :DIRECTORY #P"/nix/store/wxcdkva1kp8isnadn5rvgp6rvlaayg55-slynk-20220707-git/contrib/" :TYPE "fasl" :PREFIX "slynk-mrepl-tmp" :SUFFIX NIL :ELEMENT-TYPE NIL :EXTERNAL-FORMAT NIL)
8: (UIOP/LISP-BUILD:COMPILE-FILE* #P"/nix/store/wxcdkva1kp8isnadn5rvgp6rvlaayg55-slynk-20220707-git/contrib/slynk-mrepl.lisp" :OUTPUT-FILE #P"/nix/store/wxcdkva1kp8isnadn5rvgp6rvlaayg55-slynk-20220707-git/contrib/slynk-mrepl.fasl" :EXTERNAL-FORMAT :UTF-8 :WARNINGS-FILE NIL)
9: (ASDF/LISP-ACTION:PERFORM-LISP-COMPILATION #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "slynk/mrepl" "../contrib/slynk-mrepl">)
10: ((SB-PCL::EMF ASDF/ACTION:PERFORM) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "slynk/mrepl" "../contrib/slynk-mrepl">)
11: ((LAMBDA NIL :IN ASDF/ACTION:CALL-WHILE-VISITING-ACTION))
12: ((:METHOD ASDF/ACTION:PERFORM-WITH-RESTARTS :AROUND (T T)) #<ASDF/LISP-ACTION:COMPILE-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "slynk/mrepl" "../contrib/slynk-mrepl">) [fast-method]
13: ((:METHOD ASDF/PLAN:PERFORM-PLAN (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1002516553}>) [fast-method]
14: ((FLET SB-C::WITH-IT :IN SB-C::%WITH-COMPILATION-UNIT))
15: ((:METHOD ASDF/PLAN:PERFORM-PLAN :AROUND (T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1002516553}>) [fast-method]
16: ((:METHOD ASDF/OPERATE:OPERATE (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "slynk/mrepl"> :PLAN-CLASS NIL :PLAN-OPTIONS NIL) [fast-method]
17: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "slynk/mrepl">)
18: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
19: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/SYSTEM:SYSTEM "slynk/mrepl">) [fast-method]
20: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> ASDF/LISP-ACTION:LOAD-OP SLYNK/MREPL)
21: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
22: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP SLYNK/MREPL) [fast-method]
23: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {10017A464B}> :OVERRIDE T :KEY NIL :OVERRIDE-CACHE T :OVERRIDE-FORCING NIL)
24: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
25: (ASDF/SESSION:CALL-WITH-ASDF-SESSION #<FUNCTION (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {10030E0D6B}> :OVERRIDE NIL :KEY NIL :OVERRIDE-CACHE NIL :OVERRIDE-FORCING NIL)
26: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP SLYNK/MREPL) [fast-method]
27: (ASDF/OPERATE:LOAD-SYSTEM SLYNK/MREPL)
28: ((LAMBDA NIL :IN "/nix/store/15ywyl8x06j5xmi9c51pchhhnm160mlv-build-slynk_mrepl.lisp"))
29: (SB-INT:SIMPLE-EVAL-IN-LEXENV (DOLIST (S (QUOTE (SLYNK/MREPL))) (ASDF/OPERATE:LOAD-SYSTEM S)) #<NULL-LEXENV>)
30: (EVAL-TLF (DOLIST (S (QUOTE (SLYNK/MREPL))) (ASDF/OPERATE:LOAD-SYSTEM S)) 1 NIL)
31: ((LABELS SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (DOLIST (S (QUOTE (SLYNK/MREPL))) (ASDF/OPERATE:LOAD-SYSTEM S)) 1)
32: ((LAMBDA (SB-KERNEL:FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) (DOLIST (S (QUOTE (SLYNK/MREPL))) (ASDF/OPERATE:LOAD-SYSTEM S)) :CURRENT-INDEX 1)
33: (SB-C::%DO-FORMS-FROM-INFO #<FUNCTION (LAMBDA (SB-KERNEL:FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) {10016567DB}> #<SB-C::SOURCE-INFO {10016567A3}> SB-C::INPUT-ERROR-IN-LOAD)
34: (SB-INT:LOAD-AS-SOURCE #<SB-SYS:FD-STREAM for "file /nix/store/15ywyl8x06j5xmi9c51pchhhnm160mlv-build-slynk_mrepl.lisp" {1001650E73}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
35: ((LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) #<SB-SYS:FD-STREAM for "file /nix/store/15ywyl8x06j5xmi9c51pchhhnm160mlv-build-slynk_mrepl.lisp" {1001650E73}> NIL)
36: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<FUNCTION (LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) {7FFFF784F80B}> #<SB-SYS:FD-STREAM for "file /nix/store/15ywyl8x06j5xmi9c51pchhhnm160mlv-build-slynk_mrepl.lisp" {1001650E73}> NIL #<SB-SYS:FD-STREAM for "file /nix/store/15ywyl8x06j5xmi9c51pchhhnm160mlv-build-slynk_mrepl.lisp" {1001650E73}>)
37: (LOAD #<SB-SYS:FD-STREAM for "file /nix/store/15ywyl8x06j5xmi9c51pchhhnm160mlv-build-slynk_mrepl.lisp" {1001650E73}> :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST :ERROR :EXTERNAL-FORMAT :DEFAULT)
38: ((FLET SB-IMPL::LOAD-SCRIPT :IN SB-IMPL::PROCESS-SCRIPT) #<SB-SYS:FD-STREAM for "file /nix/store/15ywyl8x06j5xmi9c51pchhhnm160mlv-build-slynk_mrepl.lisp" {1001650E73}>)
39: ((FLET SB-UNIX::BODY :IN SB-IMPL::PROCESS-SCRIPT))
40: ((FLET "WITHOUT-INTERRUPTS-BODY-11" :IN SB-IMPL::PROCESS-SCRIPT))
41: (SB-IMPL::PROCESS-SCRIPT "/nix/store/15ywyl8x06j5xmi9c51pchhhnm160mlv-build-slynk_mrepl.lisp")
42: (SB-IMPL::TOPLEVEL-INIT)
43: ((FLET SB-UNIX::BODY :IN SB-IMPL::START-LISP))
44: ((FLET "WITHOUT-INTERRUPTS-BODY-3" :IN SB-IMPL::START-LISP))
45: (SB-IMPL::%START-LISP)
unhandled condition in --disable-debugger mode, quitting
;
; compilation unit aborted
; caught 1 fatal ERROR condition
error: builder for '/nix/store/ypbnbsw51vp25wz160647wnjkf91sp52-slynk_mrepl-20220707-git.drv' failed with exit code 1;
last 10 log lines:
> 41: (SB-IMPL::PROCESS-SCRIPT "/nix/store/15ywyl8x06j5xmi9c51pchhhnm160mlv-build-slynk_mrepl.lisp")
> 42: (SB-IMPL::TOPLEVEL-INIT)
> 43: ((FLET SB-UNIX::BODY :IN SB-IMPL::START-LISP))
> 44: ((FLET "WITHOUT-INTERRUPTS-BODY-3" :IN SB-IMPL::START-LISP))
> 45: (SB-IMPL::%START-LISP)
>
> unhandled condition in --disable-debugger mode, quitting
> ;
> ; compilation unit aborted
> ; caught 1 fatal ERROR condition
For full logs, run 'nix log /nix/store/ypbnbsw51vp25wz160647wnjkf91sp52-slynk_mrepl-20220707-git.drv'.
cl-ana is a "Free (GPL) Common Lisp data analysis library with emphasis on modularity and conceptual clarity. "
Build is broken, can be repaired by this patch:
--- a/pkgs/development/lisp-modules-new/ql.nix
+++ b/pkgs/development/lisp-modules-new/ql.nix
@@ -127,6 +127,15 @@ let
nativeLibs = [ libfixposix ];
systems = [ "iolib" "iolib/os" "iolib/pathnames" ];
};
+ "cl-ana.hdf-cffi" = pkg: {
+ nativeBuildInputs = [ hdf5 pkg-config ];
+ nativeLibs = [ hdf5 ];
+ NIX_LDFLAGS = [ "-lhdf5" ];
+ };
+ gsll = pkg: {
+ nativeBuildInputs = [ gsl ];
+ nativeLibs = [ gsl ];
+ };
};
qlpkgs =
I noticed that Nix crashes with a stack overflow when I try to build the full set of packages for sbcl
like this:
$ nix-build . -A lispPackages_new.sbclPackages
error: stack overflow (possible infinite recursion)
Does anyone understand what the problem is? (Is it a true infinite recursion problems or just a deep recursion due to coding style that would need to be changed?)
My use case is that I would like to test building all packages via a Hydra CI instance. For this I need to evaluate a Nix expression that returns a complete (or at least ample) set of package derivations.
It's a 3d graphics development tool
see cfd4a3d
Consider whether ASDF should be pinned and load
'ed between each build run and during start of lisp wrappers.
Essentially overriding the implementation-dependent (require 'asdf)
- Could lead to more consistent results, like in SBCL where ASDF doesn'y support package-inferred-system
Porting nix-cl to another project means making the flake more accessible, so if there is any work/comments on this already, it would be useful for a PR which will come in the following days. Much love and thanks in advance. Much gratitude for this project. Lisp will win, in the end.
Ressources already found:
PS: Land survey, gyroscopes and celestial navigation, each individually proves Earth is level.
UI toolkit bindings https://github.com/bohonghuang/cl-gtk4
Might be cool for docker images, to limit their size and in general to limit closure size.
I have a Lisp package that needs to add a directory to XDG_DATA_DIRS. What's the right way to do that?
(It's vk
that needs to find vulkan-validation-layers
for some data files loaded by the native libraries at runtime.)
When attempting to run the example from the readme, I get the following error:
$ HOME=$PWD nix-shell -p 'with import ./. {}; sbclWithPackages (ps: [ps.hunchentoot ps.sqlite])' --run "sbcl --noinform --eval '(require :asdf)' --eval '(asdf:load-system :hunchentoot)'"
debugger invoked on a SB-INT:SIMPLE-FILE-ERROR in thread
#<THREAD "main thread" RUNNING {1004B78323}>:
Can't create directory /nix/store/wywjn6x9hcn5qd5z8nmiw49r0agniqak-babel-20200925-git/build
Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.
restarts (invokable by number or by possibly-abbreviated name):
0: [RETRY ] Retry directory creation.
1: [RETRY ] Retry
compiling #<CL-SOURCE-FILE "asdf" "build" "asdf">.
2: [ACCEPT ] Continue, treating
compiling #<CL-SOURCE-FILE "asdf" "build" "asdf">
as having been successful.
3: Retry ASDF operation.
4: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the
configuration.
5: [CONTINUE ] Ignore runtime option --eval "(asdf:load-system :hunchentoot)".
6: [ABORT ] Skip rest of --eval and --load options.
7: Skip to toplevel READ/EVAL/PRINT loop.
8: [EXIT ] Exit SBCL (calling #'EXIT, killing the process).
(SB-KERNEL::%FILE-ERROR #P"/nix/store/wywjn6x9hcn5qd5z8nmiw49r0agniqak-babel-20200925-git/build/asdf.fasl" "Can't create directory ~A~:[~;,~%a file with ~
the same name already exists.~]" "/nix/store/wywjn6x9hcn5qd5z8nmiw49r0agniqak-babel-20200925-git/build" NIL)
0]
;
; compilation unit aborted
; caught 1 fatal ERROR condition
*
Does this maybe have something todo with asdf's output path translations?
Don't use execv flags to load ASDF, because for SBCL it makes it impossible to use C runtime flags.
sbcl --load /nix/store/.../asdf.fasl --dynamic-space-size 4096
fatal error before reaching READ-EVAL-PRINT loop:
C runtime option --dynamic-space-size in the middle of Lisp options.
Don't know how to do it, I tried with a pipe to stdin, but got really weird results... It worked but then the repl wouldn't catch EOF from Ctrl+D, and it would't catch SIGINT from Ctrl+C.
This is experimental work-in-progress but I am running a Hydra (Nix CI) instance with builds of the Lisp packages: https://hydra.nuddy.co/
I started last month with one 16-core Ryzen3 CPU running Linux/x86-64 builds.
This week I added an 80-core Ampere ARM64 build machine and just this moment kicked off an experimental larger build of {sbcl,ecl,abcl} on {x86-64, i686, arm64} at NixOS/nixpkgs#193754 (comment).
The next machine I add will be a Mac M1. Then we should have quite good cross-platform test coverage for Lisp packages.
The tests are very basic right now, and only served up in the raw Hydra webUI, but the intention is to also generate some human-readable reports and e.g. to directly monitor some key projects like SBCL to test changes before they are released.
If anyone wants to collaborate on this, e.g. to have Hydra test some branches of their own for Lisp packages, just leave a comment and I will try to help. The big idea is just to efficiently find and fix problems in Lisp libraries (or their Nix packagings.)
McCLIM build is not working. I don't understand exactly why yet.
There seem to be a couple of issues:
mcclim-bezier
package is being flagged as broken. (I see now this was done manually, possibly due to problem below.)mcclim-bezier
package fails because it tries to compile a file from mcclim-render
into read-only nix store.Reproduce on master branch:
$ lwp sbcl mcclim-bezier
error: builder for '/nix/store/1nhzagh6w25zsy2vzfmf128yvs654jfv-mcclim-bezier-20221106-git.drv' failed with exit code 1;
last 10 log lines:
> ; compiling file "/nix/store/8612lj65srfmy9a59l0l0f2w4igz0ir8-source/Extensions/fonts/ttf-medium-mixin.lisp" (written 01 JAN 1970 12:00:01 AM):
>
> ; wrote /build/source/Extensions/fonts/ttf-medium-mixin-tmpJAIDFZTC.fasl
> ; compilation finished in 0:00:00.033
> ;
> ; compilation unit aborted
> ; caught 1 fatal ERROR condition
> BUILD FAILED: Error opening #P"/nix/store/k7kcqlpc7a7p20bcarfxn7alyvryw98p-mcclim-render-20221106-git/Extensions/render/package-tmp8V3J6PE9.fasl":
>
> Permission denied
For full logs, run 'nix log /nix/store/1nhzagh6w25zsy2vzfmf128yvs654jfv-mcclim-bezier-20221106-git.drv'.
error: 1 dependencies of derivation '/nix/store/p07vdqm699c6di1irnlp7skalwnnfhhm-sbcl-with-packages.drv' failed to build
Just a small tracking issue, that the July update is available.
http://blog.quicklisp.org/2022/07/july-2022-quicklisp-dist-update-now.html
there should be something akin to buildPythonApplication
from nixpkgs, but for ASDF lisp, perhaps using asdf:make
combined with asdf:program-op
.
Hey I don't quite understand which nixpkgs version nix-cl is supposed to use for libraries?
In practice the chain of dependencies seems to be github:uthar/nix-cl
-> github:uthar/dev
-> nixpkgs
i.e. that the nixpkgs version will be determined by the contents of https://github.com/Uthar/dev/blob/master/flake.lock.
That seems a bit random, though? Is it just that using flakes means pinning everything by default anyway?
I ask because I need to make a not-quite-trivial update to the Vulkan shaderc
library in nixpkgs (lukego/nixpkgs@059fb38) in order to fix an obscure runtime incompatibility in the Lisp vk
library that only shows up when running real application code (not during build.) I'm planning to upstream that change to nixpkgs but I don't really understand the flow for landing it in nix-cl where it is needed e.g. when I make a PR to package vk.
Tips welcome, I don't know how understood all this flakes stuff is really in general :)
To be able to specify something like this in build-asdf-system
https://github.com/nuddyco/lispnix/blob/f7ecdd252fa91268f7bb0d1b8b67d918c1cbe457/kons-9.nix#L16
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.