Giter Club home page Giter Club logo

larsbrinkhoff / lbforth Goto Github PK

View Code? Open in Web Editor NEW
408.0 31.0 113.0 19.36 MB

Self-hosting metacompiled Forth, bootstrapping from a few lines of C; targets Linux, Windows, ARM, RISC-V, 68000, PDP-11, asm.js.

License: GNU General Public License v3.0

Makefile 1.01% Forth 96.53% Shell 0.73% C 0.41% Awk 0.05% Emacs Lisp 0.83% Batchfile 0.13% GDB 0.09% HTML 0.21%
forth compiler interpreter metacompiler pdp11 m68k self-hosted linux arm programming-language

lbforth's Introduction

( Subset of Forth94 )

This is a self-hosted implementation of Forth, which can regenerate itself from Forth source code. The bootstrapping process uses a metacompiler written in Lisp to target a small inner interpreter and a handful of code words written in C. A new metacompiler written in Forth generates an x86 executable using using assembly language code words.

There are also ARM, RISC-V, Motorola 68000, PDP-11, and asm.js targets. There is a cross compiler for 6502, 8051, AVR, Cortex-M, MSP430, PDP-8, PIC, and STM8.

( Continuous integration )

The code is continuously built and tested in Linux, MacOS X, and Windows using several cloud-based continuous integration services. This is documented in build.md.

( Further reading )

INSTALL \ How to build.
doc \ Classic (and recent) texts not related to this project.
lib/README \ Information about libraries.
targets/README.md \ Information about current and possibly future targets.

( Implementation guide )

The Forth kernel contains everything needed to read and compile the rest of the system from source code, and not much else. It's composed of two parts: a target-specific file nucleus.fth containing all primitive CODE words, and a target-independent kernel.fth. These two are compiled by the metacompiler.

The C target nucleus used for bootstrapping has only twelve proper primitives. There is also the COLD word which compiles to main(), and four I/O words.

When the kernel starts, it jumps to the word called WARM. This is responsible for loading the rest of the system and entering the text interpreter. The first file loaded by WARM is core.fth, which implements the CORE wordset. Because the kernel only has a bare minimum of words, the start of core.fth looks a little strange.

lbforth's People

Contributors

johanlindberg avatar larsbrinkhoff avatar liorst4 avatar pipcet avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lbforth's Issues

(number) strips digits from undefined symbols

When entering "2dummy", the error message is "Undefined: dummy". It's even worse when base is 36, as all letters and digits will be stripped from the error message:

36 base !
 ok
abcdef-ghij
Undefined: -ghij

Help targeting macOS

I'd like to make a port to macOS, but it's not really viable to do that without access to a macOS machine.

If I could get ssh access to a machine for a month or so, that'd be really helpful.

Windows test fails

make check fails in Windows. There are two failing tests:

  • test-exe
  • test-save-image

Metacompiler overhaul

The metacompiler is due for an overhaul.

  • The current implementation is closely tied to building the kernel.

    • The kernel-specific code should be split off.
    • The compiler should be made a generic cross compiler.
  • Everything is written to a single memory space.

    • Many microcontrollers use a Harvard architecture and need separate code and data spaces.
    • Some applications may want to have separate read-only and read-write data spaces.
    • The word headers can be written to a separate area.
  • Building and saving the image is constrained by the single memory space.

    • Must write everything sequentially from start to end.
    • It would be better to be able to write many separate sections and assemble them together in the end.

search-path entries are truncated to NAME_LEN

lbForth$ strace -e open ./b-forth
[...]
s" /this/is/a/really/really/long/path" searched
 ok
include foo.fth
open("/this/is/a/realfoo.fth", O_RDONLY) = -1 ENOENT (No such file or directory)

calling windows system calls

Is there currently a high level way to call system calls?

I look in io.fth and see you setting up the system calls but you only do them from assembly below.

For example, in FreeForth there is the word:

lib:` ( @ # <name> -- ; a n -- funEntry ) loads the dynamic-link-library
  with filename starting at address "@" for "#" bytes, and defines a word
  which will lookup in this library for the function matching the name
  starting at address "a" for "n" bytes, and if found will return its entry,
  for use with "#call" or "fun:".
  If the library is not found, or if later a function is not found,
  an exception is thrown with corresponding error message.
  : lib:` :` #lib lit` #fun ' call, ;` ;
  see also: lib: #call fun: libc k32  ff.boot

and:

#call ( argN ... arg1 N funEntry -- funResult ) calls the library function
  with entry at "funEntry", which consumes "N" arguments from DATAstack and
  returns a single result on top of DATAstack. The N arguments must be pushed
  on DATAstack in the same order as in C, i.e. C-prototype (see Linux man or
  Windows win32.hlp) rightmost argument first, leftmost last.

Ensure x86 performance at least on par with Gforth

See e.g. https://groups.google.com/d/msg/comp.lang.forth/YVjtNQwSIH0/YHduzDG4c9EJ

Some may believe that a threaded-code engine written in assembly language would have led to better performance than that exhibited by Gforth. However, looking at http://www.complang.tuwien.ac.at/forth/performance.html, Gforth 0.4.9 performed faster than any of the threaded-code Forth systems written in assembly language (Win32Forth, NT Forth, and eforth) on all of the benchmarks on a 486DX2/66:

The interpretive systems written in assembly language (except eforth opt) are, surprisingly, slower than Gforth. One important reason for the disappointing performance of these systems is probably that they are not written optimally for the 486 (e.g., they use the lods instruction).

Switching search order between interpretation and compilation

The search order is different in interpretation state and compilation state. When interpreting, only forth is searched, but when compiling there's an implicit also compiler-words. This is currently done in a buggy and nongeneral way.

The solution might also consider metacompilation. Inside the metacompiler in interpretation state, the search order is host-interpreter, meta-interpreter, forth. In compilation state, the search order is host-compiler, meta-interpreter, forth.

When metacompiling in interpretation state, the search order is: meta-interpreter, forth. When metacompiling in compilation state, the search order is: meta-compiler, then target dictionary.

PDP-10 target

Get up and running in ITS.

It's a word-adressed machine. But there are also special "byte pointers". Consider what to do about C@ and C! etc.

Accomodate asm.js changes in metacompiler

The asm.js target added some [IF] sections to the metacompiler. See #33 and f832c94. These should be fixed to use code from some target file instead.

The target also abuses cold.fth to insert special code for dumping the image. This doesn't hurt the overall structure of the metacompiler, but there is room for improvement.

min vs umin for size

Why do you use min but not umin in the code like the following:

: name, ( a u -- ) #name min dup c, tuck move, offset, ;

Here min protects against too long u, but not from negative u (or very long positive ;).

Use save-image to write a new executable

Use the save-image to write a new executable with everything pre-loaded.

  • It will make Forth start faster.
  • It will test the save-image library.
  • There will be an incentive to make save-image work in more targets.

Xtensa target

ESP8266 and ESP32 are everywhere. Need that Xtensa target.

Fedora Core 25 x86 64 bits target

I thought it would be interesting to try out lbForth. Target is Fedora Core 25 x86 64 bits and here are my notes:

  • gnu/stubs-32.h is missing (it‘s not installed by default on 64 bits FC25), but it seems to work by simply removing -m32 in targets/c/forth.mk (see compiler log below).
  • Make warns about “circular kernel.c <- b-forth dependency dropped” (see compiler log below).
  • lbForth prints “File not found” but it would be helpful if the failed file path is printed too. Strace reveals the details so it was relatively easy to figure out the cause.
  • I used s" /some/dir/..." searched to make lbForth find source files of interest. Not sure if this is correct but it did seem to work.
  • The search path seems to be limited to 23 characters which caused some problems with mangled file paths.
  • require subdirectories seems to be lost, so that foo in require foo/bar.fth is ignored?
  • The code I tried makes use of Forth locals to some extent, and it seems src/locals.fth doesn’t implement it so I didn‘t get further after the error message Undefined: {.
make -ftargets/c/bootstrap.mk
make[1]: Entering directory '/usr/local/src/lbForth'
make[1]: Circular kernel.c <- b-forth dependency dropped.
gcc -m32 -O2 -fomit-frame-pointer -fno-unit-at-a-time -Itargets/c targets/c/params.c -o params
In file included from /usr/include/features.h:392:0,
                 from /usr/include/string.h:25,
                 from targets/c/params.c:1:
/usr/include/gnu/stubs.h:7:27: fatal error: gnu/stubs-32.h: No such file or directory
 # include <gnu/stubs-32.h>
                           ^
compilation terminated.
targets/c/forth.mk:25: recipe for target 'params' failed
make[1]: *** [params] Error 1
make[1]: Leaving directory '/usr/local/src/lbForth'
Makefile:27: recipe for target 'b-forth' failed
make: *** [b-forth] Error 2

overload of EXECUTE in cygwin/clisp causes failure to metacompile

$ make
make -ftargets/c/bootstrap.mk
make[1]: Entering directory '/home/kevin/lbForth'
make[1]: Circular kernel.c <- b-forth dependency dropped.
gcc -m32 -O2 -fomit-frame-pointer -fno-unit-at-a-time -Itargets/c targets/c/params.c -o params
targets/c/run.sh ./params -lisp > params.lisp
cp targets/c/jump.fth jump.fth
cp targets/ctc.fth threading.fth
cp targets/c/target.fth target.fth
echo ": sysdir s" src/" ;" >> target.fth
./lisp/lisp.sh '(load "lisp/meta.lisp") (compile-forth "targets/c/nucleus.fth" "src/kernel.fth")'
;; Loading file lisp/meta.lisp ...
;; Loading file params.lisp ...
;; Loaded file params.lisp
WARNING: DEFUN/DEFMACRO(EXECUTE): # is locked
Ignore the lock and proceed
WARNING: DEFUN/DEFMACRO: redefining function EXECUTE in /home/kevin/lbForth/lisp/meta.lisp, was defined in C
;; Loading file lisp/words.lisp ...
;; Loaded file lisp/words.lisp
;; Loaded file lisp/meta.lisp
gcc -m32 -O2 -fomit-frame-pointer -fno-unit-at-a-time -Itargets/c -c -o kernel.o kernel.c
gcc -m32 kernel.o -o b-forth
rm -f kernel.c kernel.o params.lisp jump.fth threading.fth target.fth
make[1]: Leaving directory '/home/kevin/lbForth'
cp b-forth forth
rm -f *-stamp
touch x86-windows-stamp
cat targets/x86/params.fth targets/x86/windows/params.fth > params.fth
cp targets/x86/jump.fth jump.fth
cp targets/itc.fth threading.fth
cat targets/x86/target.fth targets/x86/windows/target.fth > target.fth
echo ": sysdir s" /usr/local/share/lbForth/" ;" >> target.fth
echo include targets/x86/meta.fth | targets/c/run.sh ./forth | tail -n+3 > x86-forth
mv image x86-forth
mv: cannot stat 'image': No such file or directory
make: [targets/x86/forth.mk:6: x86-forth] Error 1 (ignored)
chmod a+x x86-forth
rm -f forth.exe
cp x86-forth forth
0;~/lbForth

Assembler overhaul

There's a lot of copy-pasting between the different assemblers. Much of the assembler implementation should go into a generic library, and target-specific parts should built on top of that.

Clean up asm.js target.fth

The asm.js target.fth has two definitions for header,, which is probably a mistake.

Also, docol, etc compile a zero cell before the code field proper. I'm not sure that's by design or not. Is that the place for the "codeid", or the does field? I think the latter. But then the zero could probably move to header, like in the C target.

@pipcet, do you have any comment on this?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.