xach / buildapp Goto Github PK
View Code? Open in Web Editor NEWBuildapp makes it easy to build application executables with SBCL
Home Page: http://www.xach.com/lisp/buildapp/
Buildapp makes it easy to build application executables with SBCL
Home Page: http://www.xach.com/lisp/buildapp/
Take the version from the system file and display it when given a --version option.
When I make buildapp with an absolute pathname for SBCL, like this:
make DESTDIR=/home/rpg/.local/bin/ LISP=/opt/sbcl/current/bin/sbcl
I get the wrong command line arguments to SBCL (--no-init
instead of --no-userinit
and --no-sysinit
):
/opt/sbcl/current/bin/sbcl --quiet --no-init \
--eval "(require 'asdf)" \
--eval "(push \"$(pwd)/\" asdf:*central-registry*)" \
--eval "(require 'buildapp)" \
--eval "(buildapp::build-buildapp)" \
--eval "#+sbcl (exit) #+ccl (quit)"
When I use simply sbcl
I do not get this problem:
$ make DESTDIR=/home/rpg/.local/bin/ LISP=sbcl
sbcl --noinform --no-userinit --no-sysinit --disable-debugger \
--eval "(require 'asdf)" \
--eval "(push \"$(pwd)/\" asdf:*central-registry*)" \
--eval "(require 'buildapp)" \
--eval "(buildapp::build-buildapp)" \
--eval "#+sbcl (exit) #+ccl (quit)"
This is due to a bug in the Makefile:
ifeq ($(LISP),sbcl)
FLAGS=--noinform --no-userinit --no-sysinit --disable-debugger
else
FLAGS=--quiet --no-init
endif
The check for sbcl
should be a substring or basename check, not an absolute check.
I will try to get you an MR for this; I don't have time at the moment, but am filing this issue so I don't forget.
As far as I can tell, sbcl supports 1-9 as a compression level if built with :SB-CORE-COMPRESSION feature, but buildapp does not allow for setting the compression level.
Could the --compress-core
flag accept an (optional) compression level?
I tested building buildapp on CCL and it failed with "Permission denied: '/buildapp'". Complete output here: http://paste.lisp.org/display/138728
To me it seems that NATIVE-NAMESTRING binds default-pathname-defaults to #P"/".
When there is no default entry in a set of dispatched-entries, buildapp errors out without building.
I believe that it's going wrong at this line:
Line 123 in dd00f18
macroexpand-1
on (quit 1)
causes (quit 1)
to be executed?
Here's an example of the error:
BUILDAPP(3): (dispatched-entry-form (dispatched-entries *dumper*))
rpg@rpg-mbp182: ~/lisp/shop/shop3/buildapp $
To replicate, do rlwrap sbcl --eval '(ql)' --eval '(ql:quickload "buildapp")' --eval '(in-package :buildapp)'
(cl-user::ql
is my function that load setup) and then
(defparameter *args* (list "--logfile" "/tmp/buildapp-shop-app.log" "--output" "shop-app" "--dispatched-entry" "shop/shop-app::main" "--dispatched-entry" "ess-shop/shop-app::ess-main" "--load" "/Users/rpg/lisp/asdf/build/asdf.lisp" "--asdf-path" "/Users/rpg/lisp/shop/shop3/buildapp/../" "--asdf-tree" "/Users/rpg/lisp/shop/shop3/buildapp/../../jenkins/ext/" "--eval" "(declaim (optimize speed space safety))" "--load-system" "shop3" "--load" "/Users/rpg/lisp/shop/shop3/buildapp/shop-app-entrypoints.lisp" "--dumpfile-copy" "/tmp/dumpfile-copy.lisp"))
(defparameter *dumper* (command-line-dumper *args*))
(dispatched-entry-form (dispatched-entries *dumper*))
If you add --dispatched-entry "/shop-app::main"
to the arguments, all is well.
It'd be nice if buildapp could run some (declaim (optimize))
depending on the build.
Thoughts?
When I do (ql:quickload "buildapp")
I get the following error:
The value #P"buildapp.exe" is not of the expected type SEQUENCE.
[Condition of type TYPE-ERROR]
It looks like it is caused by this form in buildapp.lisp
(defparameter *output-type-pathname* ....)
I am using CCL "Version 1.10-r16196 (WindowsX8632)" on Windows XP 32b.
There seems to be a --sbcl option to set which program is invoked for sbcl, but no corresponding --ccl option to set the ccl binary. It seems hardcoded to "ccl".
This is a problem if you really want to use a different binary to e.g. produce 64 vs 32 bit executables.
(buildapp:build-buildapp) fails with
Error #<BUILDAPP::SILENT-EXIT-ERROR #x302002BECB5D>
[Condition of type BUILDAPP::SILENT-EXIT-ERROR]
Restarts:
0: [RETRY] Retry SLIME REPL evaluation request.
1: [*ABORT] Return to SLIME's top level.
2: [ABORT-BREAK] Reset this thread
3: [ABORT] Kill this thread
Backtrace:
0: ((:INTERNAL BUILDAPP::MAIN) #<BASIC-FILE-CHARACTER-OUTPUT-STREAM ("dumper-3SdqQGqO.lisp"/4 UTF-8) #x302002BF220D> #P"dumper-3SdqQGqO.lisp")
1: (BUILDAPP::CALL-WITH-TEMPORARY-OPEN-FILE "dumper.lisp" #<COMPILED-LEXICAL-CLOSURE (:INTERNAL BUILDAPP::MAIN) #x302002BF2ACF>)
2: (BUILDAPP::MAIN ("sbcl" "--asdf-path" "/Users/joshuakordani/quicklisp/dists/quicklisp/software/buildapp-1.5.4/" "--load-system" "buildapp" "--entry" ...))
3: (CCL::CALL-CHECK-REGS BUILDAPP:BUILD-BUILDAPP)
4: (CCL::CHEAP-EVAL (BUILDAPP:BUILD-BUILDAPP))
5: (SWANK::EVAL-REGION "(buildapp:build-buildapp)\n")
6: ((:INTERNAL SWANK-REPL::REPL-EVAL))
7: (SWANK-REPL::TRACK-PACKAGE #<COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK-REPL::REPL-EVAL) #x302002A706BF>)
8: (SWANK::CALL-WITH-RETRY-RESTART "Retry SLIME REPL evaluation request." #<COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK-REPL::REPL-EVAL) #x302002A7073F>)
9: (SWANK::CALL-WITH-BUFFER-SYNTAX NIL #<COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK-REPL::REPL-EVAL) #x302002A7077F>)
10: (SWANK-REPL::REPL-EVAL "(buildapp:build-buildapp)\n")
11: (CCL::CALL-CHECK-REGS SWANK-REPL:LISTENER-EVAL "(buildapp:build-buildapp)\n")
12: (CCL::CHEAP-EVAL (SWANK-REPL:LISTENER-EVAL "(buildapp:build-buildapp)\n"))
13: (SWANK:EVAL-FOR-EMACS (SWANK-REPL:LISTENER-EVAL "(buildapp:build-buildapp)\n") "COMMON-LISP-USER" 144)
14: (SWANK::PROCESS-REQUESTS NIL)
15: ((:INTERNAL SWANK::HANDLE-REQUESTS))
16: ((:INTERNAL SWANK::HANDLE-REQUESTS))
17: (SWANK/BACKEND:CALL-WITH-DEBUGGER-HOOK #<Compiled-function SWANK:SWANK-DEBUGGER-HOOK #x3020008CA55F> #<COMPILED-LEXICAL-CLOSURE (:INTERNAL SWANK::HANDLE-REQUESTS) #x302000A88B7F>)
18: (SWANK::CALL-WITH-BINDINGS ((STANDARD-OUTPUT . #<SWANK/GRAY::SLIME-OUTPUT-STREAM #x302000A87B1D>) (STANDARD-INPUT . #<SWANK/GRAY::SLIME-INPUT-STREAM #x302000A87E8D>) ..))) #<COMPILED-LEXICAL-CLOSU..
19: (SWANK::HANDLE-REQUESTS #<MULTITHREADED-CONNECTION #x30200098D27D> NIL)
20: (CCL::RUN-PROCESS-INITIAL-FORM #<PROCESS repl-thread(14) [Active] #x302000A76D6D> (#<COMPILED-LEXICAL-CLOSURE (:INTERNAL CCL::%PROCESS-RUN-FUNCTION) #x302000A76AFF>))
21: ((:INTERNAL (CCL::%PROCESS-PRESET-INTERNAL (PROCESS))) #<PROCESS repl-thread(14) [Active] #x302000A76D6D> (#<COMPILED-LEXICAL-CLOSURE (:INTERNAL CCL::%PROCESS-RUN-FUNCTION) #x302000A76AFF>))
22: ((:INTERNAL CCL::THREAD-MAKE-STARTUP-FUNCTION))
I was having trouble with an application I built with this command line:
buildapp --dynamic-space-size 100000 --output strike-force-one --load game-player.lisp --entry 'cl-user::main' --dumpfile-copy 'strike-force-one-dumpfile.lisp' --logfile 'build-game-player.log'
When I ran the resulting application, it ran out of memory quickly. But when I ran sbcl with --dynamic-space-size 100000
and passed it a script that did the equivalent, it did not run out of memory.
Tracking it down, I found that run-program
for SBCL, invoked in main
was not getting the --dynamic-space-size 100000
argument:
(RUN-PROGRAM "sbcl"
'("--noinform" "--disable-debugger" "--no-userinit" "--no-sysinit"
"--load"
"/home/rpg/projects/scepter/shop-mcts/strike-force-one/dumper-2SKVI5f7.lisp"))
Further debugging shows that buildapp
does not see the --dynamic-space-size
argument:
Constructing a dumper from (--output strike-force-one --load game-player.lisp
--entry cl-user::main --dumpfile-copy
strike-force-one-dumpfile.lisp --logfile
build-game-player.log)
After processing the command line arguments, we have the following dumper object:
#<BUILDAPP::DUMPER strike-force-one cl-user::main>
[standard-object]
Slots with :INSTANCE allocation:
PACKAGE = #:DUMPER124
ACTIONS = ((:LOAD "game-player.lisp"))
ENTRY = cl-user::main
DISPATCHED-ENTRIES = NIL
ASDF-DIRECTIVES = NIL
LOAD-PATHS = NIL
SBCL = "sbcl"
CCL = "ccl"
OUTPUT = "strike-force-one"
LOGFILE = "build-game-player.log"
DUMPFILE-COPY = "strike-force-one-dumpfile.lisp"
CORE-ONLY = NIL
COMPRESS-CORE = NIL
DYNAMIC-SPACE-SIZE = NIL
The "constructing a dumper" message above formats the arguments that are passed to command-line-dumper
.
I conjecture that SBCL is stripping out this argument before the buildapp executable can see it.
I will attempt to verify this.
It is documented at https://ccl.clozure.com/manual/chapter16.1.html and is comparable to --dynamic-space-size
in SBCL I think. Following up on problems to build pgloader with CCL on dockerhub and travis environments I got the following piece of advice:
rme: The second thing might be to try running ccl with "--heap-reserve 150g". Perhaps the container environment doesn't like ccl's default reservation of 512 gigabytes of address space.
It looks like the following patch would allow buildapp to support --heap-reserve
?
modified buildapp.lisp
@@ -400,7 +400,8 @@ ARGV. See *USAGE* for details."
(quit))
(let* ((dumper (command-line-dumper (rest argv)))
(*package* (find-package :buildapp))
- #+sbcl (dynamic-space-size (dynamic-space-size dumper)))
+ #+sbcl (dynamic-space-size (dynamic-space-size dumper))
+ #+ccl (heap-reserve (heap-reserve dumper)))
(with-tempfile (stream ("dumper.lisp" file))
(write-dumpfile dumper stream)
(force-output stream)
@@ -415,6 +416,11 @@ ARGV. See *USAGE* for details."
(list "--dynamic-space-size"
(princ-to-string
dynamic-space-size)))
+ #+ccl
+ (when heap-reserve
+ (list "--heap-reserve"
+ (princ-to-string
+ heap-reserve)))
#+sbcl "--noinform"
#+ccl "--quiet"
#+sbcl "--disable-debugger"
modified command-line.lisp
@@ -160,6 +160,8 @@
(push entry (dispatched-entries plan))))
(:dynamic-space-size
(setf (dynamic-space-size plan) (parse-integer value)))
+ (:heap-reserve
+ (setf (heap-reserve plan) (parse-integer value)))
(t
(error 'unknown-argument :flag argument)))))))
modified dumper.lisp
@@ -86,6 +86,10 @@
(dynamic-space-size
:initarg :dynamic-space-size
:accessor dynamic-space-size
+ :initform nil)
+ (heap-reserve
+ :initarg :heap-reserve
+ :accessor heap-reserve
:initform nil)))
(defgeneric needs-asdf-p (dumper)
Buildapp-generated binaries should accept runtime arguments that limit memory footprint, just like a normal sbcl session does.
Easy to reproduce:
$ echo "(" > bug.lisp
$ buildapp --output bug --load bug.lisp
The subprocess gets stuck in a debugger that isn't attached to the terminal.
osx 10.10 ccl64 build from trunk, make line was make DESTDIR="path/to/buildapp" LISP="path/to/dx86cl64" install
note hardcoded use of sbcl at frame 14.
(20E7C6A0) : 0 (>-2 NIL 0) 2293
(20E7C6D0) : 1 (<=-2 NIL 0) 29
(20E7C6F0) : 2 (FUNCALL #'#<#<CCL::STANDARD-KERNEL-METHOD PRINT-OBJECT (STANDARD-OBJECT T)>> #<SILENT-EXIT-ERROR #x302000D5F3FD> #<STRING-OUTPUT-STREAM #x302000D5EC5D>) 69
(20E7C710) : 3 (%CALL-NEXT-METHOD ((#) #<CCL::STANDARD-KERNEL-METHOD PRINT-OBJECT #> #<SILENT-EXIT-ERROR #x302000D5F3FD> #<STRING-OUTPUT-STREAM #x302000D5EC5D>)) 869
(20E7C790) : 4 (FUNCALL #'#<(:INTERNAL (PRINT-OBJECT (CONDITION T)))>) 221
(20E7C7B8) : 5 (%CALL-NEXT-METHOD ((#) #<CCL::STANDARD-KERNEL-METHOD PRINT-OBJECT #> #<SILENT-EXIT-ERROR #x302000D5F3FD> #<STRING-OUTPUT-STREAM #x302000D5EC5D>)) 869
(20E7C838) : 6 (FUNCALL #'#<(:INTERNAL (PRINT-OBJECT :AROUND (T T)))>) 221
(20E7C860) : 7 (%%CNM-WITH-ARGS-COMBINED-METHOD-DCODE #<SIMPLE-VECTOR 5> 69007642 NIL) 1029
(20E7C8E8) : 8 (FUNCALL #'#<#<CCL::STANDARD-KERNEL-METHOD CCL::REPORT-CONDITION (CONDITION T)>> #<SILENT-EXIT-ERROR #x302000D5F3FD> #<STRING-OUTPUT-STREAM #x302000D5EC5D>) 237
(20E7C928) : 9 (%BREAK-MESSAGE "Error" #<SILENT-EXIT-ERROR #x302000D5F3FD> 69007728 #>) 1101
(20E7CAF0) : 10 (BREAK-LOOP-HANDLE-ERROR #<SILENT-EXIT-ERROR #x302000D5F3FD> 69007728) 701
(20E7CB58) : 11 (%ERROR #<SILENT-EXIT-ERROR #x302000D5F3FD> NIL 69007728) 333
(20E7CB80) : 12 (FUNCALL #'#<(:INTERNAL MAIN)> #<BASIC-FILE-CHARACTER-OUTPUT-STREAM ("dumper-f0dIhgSY.lisp"/4 UTF-8) #x302000D76E5D> #P"dumper-f0dIhgSY.lisp") 597
(20E7CBB0) : 13 (CALL-WITH-TEMPORARY-OPEN-FILE "dumper.lisp" #<CCL:COMPILED-LEXICAL-CLOSURE (:INTERNAL MAIN) #x302000D777AF>) 781
(20E7CC48) : 14 (MAIN ("sbcl" "--asdf-path" "/Users/joshuakordani/Source/buildapp/" "--load-system" "buildapp" ...)) 461
(20E7CC88) : 15 (CALL-CHECK-REGS BUILD-BUILDAPP) 221
(20E7CCC0) : 16 (CHEAP-EVAL (BUILD-BUILDAPP)) 101
(20E7CCF8) : 17 (FUNCALL #'#<(:INTERNAL CCL::EVAL-STRING CCL::STARTUP-CCL)> "(buildapp::build-buildapp)") 485
(20E7CD40) : 18 (STARTUP-CCL NIL) 1541
(20E7CDA8) : 19 (FUNCALL #'#<(:INTERNAL (CCL:TOPLEVEL-FUNCTION (CCL::LISP-DEVELOPMENT-SYSTEM T)))>) 61
(20E7CDC8) : 20 (FUNCALL #'#<(:INTERNAL CCL::MAKE-MCL-LISTENER-PROCESS)>) 661
(20E7CE60) : 21 (RUN-PROCESS-INITIAL-FORM #<TTY-LISTENER listener(1) [Active] #x3020004476CD> (#<CCL:COMPILED-LEXICAL-CLOSURE # #x3020004471FF>)) 669
(20E7CEF0) : 22 (FUNCALL #'#<(:INTERNAL (CCL::%PROCESS-PRESET-INTERNAL (CCL:PROCESS)))> #<TTY-LISTENER listener(1) [Active] #x3020004476CD> (#<CCL:COMPILED-LEXICAL-CLOSURE # #x3020004471FF>)) 573
(20E7CF98) : 23 (FUNCALL #'#<(:INTERNAL CCL::THREAD-MAKE-STARTUP-FUNCTION)>) 277
Hi, I'm trying to build a skeleton Caveman app with the following:
(caveman.skeleton:generate #p"~/Lisp/test")
buildapp --output test-build \
--asdf-tree ~/quicklisp/dists/quicklisp/software/ \
--asdf-path ~/Lisp/test/
--load-system test \
--entry test:start
but get the following error:
;; loading system ...
;; ...
Fatal SIMPLE-PACKAGE-ERROR:
The name "QUICKLISP-CLIENT" does not designate any package.
I can't seem to find where this comes from (neither asdf-path/tree include a mention of grep -nr quicklisp-client .
)
Buildapp's dumper script currently sets sb-ext:*invoke-debugger-hook*
to NIL as one of its final steps. The intent is that user code should initialize the value at application startup. However, SBCL can fail and enter the debugger before any of the init hooks or custom toplevel functions are called.
One strategy to avoid this problem is to wrap all loading operations in a form that binds *invoke-dynamic-hook*
, and checks to see whether it's modified as a result of the load, and then saves that modified value at the end of the dumper script.
See the following debug output, it almost looks like the working directory isn't being set in sbcl.
$ sbcl --version
SBCL 1.3.15
$ make clean
rm -f buildapp *~ *.fasl *.lx32fsl
$make
sbcl --noinform --no-userinit --no-sysinit --disable-debugger \
--eval "(require 'asdf)" \
--eval "(push \"$(pwd)/\" asdf:*central-registry*)" \
--eval "(require 'buildapp)" \
--eval "(buildapp::build-buildapp)" \
--eval "#+sbcl (exit) #+ccl (quit)"
fatal error encountered in SBCL pid 51157(tid 0x7ffff7fd5fc0):
can't find core file at NIL/sbcl.core
make: *** [Makefile:11: buildapp] Error 1
Allow the use of a manifest file, where each system file is listed, one per line, with preferred versions of a library listed first.
This should go hand-in-hand with a command in Quicklisp to emit a manifest file.
After performing a write test, delete the write test file. Otherwise it can look like a successful build.
On CCL1.10 on Windows XP 32b, (buildapp:build-buildapp)
fails. run-program is called as follows:
(run-program "ccl" '(...))
If run-program is called like below, it works.
(run-program "cmd" '("/c" "ccl" ...))
Running make
gives the following:
make
sbcl --noinform --no-userinit --no-sysinit --disable-debugger \
--eval "(require 'asdf)" \
--eval "(push \"$(pwd)/\" asdf:*central-registry*)" \
--eval "(require 'buildapp)" \
--eval "(buildapp::build-buildapp)" \
--eval "#+sbcl (exit) #+ccl (quit)"
Unhandled SB-INT:EXTENSION-FAILURE in thread #<SB-THREAD:THREAD "main thread" RUNNING
{10013D0613}>:
Don't know how to REQUIRE BUILDAPP.
Explicitly loading the ASDF in Makefile fixes it: --eval "(progn (load \"buildapp.asd\") (asdf:load-system :buildapp))"
.
Per tcr, it would be nice to have buildapp store fasls in a given directory to be blown away at will, separate from the user global ~/.cache directory.
I may have broken something in my branch (see #39). Now when I compile dispatched-entry-form
, I am getting this compilation note:
; file: /Users/rpg/lisp/buildapp/dumper.lisp
; in: DEFUN DISPATCHED-ENTRY-FORM
; (MACROEXPAND-1 (BUILDAPP::QUIT 1))
;
; note: The first argument never returns a value.
; (LIST
; `(FORMAT *ERROR-OUTPUT* "Unknown dispatch name '~A', quitting~%"
; BUILDAPP::BINARY-NAME)
; (MACROEXPAND-1 (BUILDAPP::QUIT 1)))
;
; note: deleting unreachable code
Any idea what's causing this? Is it a problem? I'm not even sure what "The first argument" refers to:
macroexpand-1
: (BUILDAPP::QUIT 1)
?list
: `(FORMAT *ERROR-OUTPUT* "Unknown dispatch name '~A', quitting~%" BUILDAPP::BINARY-NAME)
?Or is it just that we're missing a quote in front of the argument to macroexpand-1
?
Additional flag for verbose compilation log output?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.