mitra42 / webforth Goto Github PK

There are a number of places where code and data are mixed, preventing them being separated - e.g. in Rom.

This is a feature of many other Forth's.

Specific known issues ...

• VARIABLE
• Vocabulary words - plus the hand-constructed "FORTH"
• Anything built with CREATE
• What else?

Now its working the testing needs fixing.

test()

• check not used

• Pull any appropriate tests from it to code('TEST')

• Enable some way to test the PAD (might be hard in pure Forth ?) see padTestLength and code in test()

• Review various test code ... L17-20 variables etc

(depends on #5)

• refactor names[] to point at na, and check offset from there xt = @(na-2*CELLL)
  • check usage of names
• immediate to set lex
• Have check of immediate use lex

THEN

• find - refactor maybe should it return (xt f) or (ca na)|(a 0) as eForth does? Check users #9
• & rework find to use the dictionary in JS, might not be too hard while single dictionary.

Provide an optional extra file that has definitions of words in ANS forth as a dictionary, and overrides words with different signatures. A program could then start with "ANS" to switch dictionary to use ANS and then FORTH

I'm at the point in the code where i need to transition to the Forth text interpreter from the JS one.

I think this requires the following steps ....

• See if can get js interpret to use the TIB
  • investigate if regexp and other string operations work esp using slice on the Buffer
  • See if can copy input into TIB
• Define replacements for [ ( ] : ;
• IF can use TIB then can probably transition gradually.
• ELSE will need to define (but not use) replacements, then switch in one go.

Should check users - "find" in code currently returns (xt f) which I think came from ANS Forth
"find" in eFORTH is (ca na) | (a 0)

"find" isn't currently working - so debug first

The eForthAndZen version of PARSE is clearly broken - it doens't increment the pointer and eForthOverviewV5 version matches the spec, but clearly the consumers e.g. "(" assume >IN is moved over the parsed string, which it isnt.

For now it is commented out, and the Javascript version kept in use

Maybe a "?" next to the title that goes to README.md for now, then to HELP.md

Also a source link and link to mitra.biz

Get the I/O words working - see TODO-IO

Some variations would be ...

• Node stdin and stdout
• Browser via a field on a web page see #22
• RS232 line for embedding (not currently relevant)

To get find to work, it needs to be able to explore dictionaries

Each name entry consists of CodePointer (to start of code dict entry); LinkAddress (to NA of previous); count+name

• Need to understand CURRENT and CONTEXT
  • Make sure CURRENT and CONTEXT are set to point to vocab really really early
• [ ]Assumption is that CONTEXT @ points to a variable space in the "FORTH" definition
• $,n and code equiv have to:
  • store CURRENT @@ (na of last word) in Link
  • Store pointer to top of dic (HERE) in Code
  • Store na in LAST
  • check for duplicates here (or maybe ok in callers)
  • needs lastStore
• OVERT pg 95
  • Store LAST in CURRENT@
  • Store in names[]
  • needs to be called by anything that calls dollarCommaN (callers of $CODE or in ;)
• Replace dollarCommN in callers
  • needs na on the stack
  • Need to remove all use of _LINK
  • check for uses of lastDefinition
  • $CODE to write name before calling dollarCommaN
• Start using LAST as well as NP

THEN

• refactor names[] and immediate #6
  Should check users - "find" in code currently returns (xt f) which I think came from ANS Forth
  "find" in eFORTH is (ca na) | (a 0)

• "find" isn't currently working - so debug first ( see below)

• Decide if refactor js version

• If refactor then make it search dictionary.

• [ ] See TODO-30-BYE

We need a way to add new Javascript when extending since jsFunctionAttributes is read once at startup.

Hi - if you are contributing or posting issues, it would be great to introduce yourself in the comments.

Basic goal would be

• a single web page
• loads a script
• Pulls in another JS file with more code
• Pulls in a Forth file
• Displays a DOM including a button with a Forth action attached (e.g. Foo @ 1 + Foo ! )and some link to display it in the DOM

Each of these steps is likely to be an issue in itself, but this issue is for the overall design.

eForth has provision for data structures for vocabularies but none of the required words.

See https://www.taygeta.com/forth/dpans16.htm for the ANS versions of them, there may be other versions.

Also https://forth-standard.org/standard/search which is basically Vocabulary ANS-ified

ALSO / ONLY are words I've seen but don't have definitions for.

• Find and work through the Vocabulary words in ANSForth -
• Think through "FORTH " which uses tokenVocabulary, hard to do as a CREATE DOES> since its first word in dic

: VOCABULARY CREATE 0 , 0 , DOES> CONTEXT ! ; 

also words like ONLY and ALSO

• What is needed in package.json for NPM
• Make version number in Forth align
• Think thru how to require etc, should it export one word "forth" and can functions have properties ?
• Edit README.md

Recode an alternative using webassembly as the base, rather than Javascript

I'm not familiar with webassembly, other than knowing its stack based, so I'm not sure if/how you program in it directly.

Forth seems to still - after all these years - subscribe to the idea that if one standard is good, then many must be better :-(.

Forth79, Forth83, ANS and probably a few more defacto standards.

Currently the system is based on eForth and should match those definitions, but that might not be the best solution.

I propose adding some vocabularies that can be used to switch between standards so that a package could for example goANS at the top and then write in ANS forth. It might not be perfect, but I'm guessing the breaking differences are minor. (e.g. the presence or absence of a word doesn't break anything - just add the word - but the same word with two different definitions breaks things).

Please note any differences in comments and I'll edit this list to include them.

• None documented so far ...

See also for documentation and issues related to specific language variants:

• ANS issue #12
• forth-standard.org
• forth200x its unclear how this relates to forth-standard.org

eForth has provision for data structures for vocabularies but none of the required words.

See https://www.taygeta.com/forth/dpans16.htm for the ANS versions of them, there may be other versions.

ALSO / ONLY are words I've seen but don't have definitions for.

Code has had enough changes to be worth a cleanup pass - some items to check are ....

• any TODO's - copy to issues
• move logical stuff together - especially JS routines that can be deleted after replacement
• Check the COLD/WARM/QUIT thinking but see #24
• Move all constants likely to be edited together.
• Make sure using whatever can of new code - e.g. vectoring EXPECT etc
• Clean out forth_v1.js
• Push to Node - moved to #23

README.md

• Installation instructions (depends on Node).- moved to #23

DESIGN.md

• Code - moved to #25

There appear to be a number of tests commented out because they failed for reasons due to the test, not the code.

They need evaluating and either removing or uncommenting.

Attempt at doing input asynchronously as synchronous failed as did simple attempts to yield etc.

• Rewrite ?RX
• Rewrote !IO to set it up
• Work in from "run" as async thru threadtoken to RX?
• Work out from run
• Fix nesting of Run (maybe if (IP) await threadtoken else await run - but should be ok since have test for IP===0
• Figure out outside loop

THEN

• TODO-IO TODO-INPUT TODO-OUTPUT
• Check efficiency - at first working it was 95% of a CPU even when looping waiting
• Maybe event driven top end - "on('data') - see #26

interpret is fragile after it switches to use $INTERPRET
If it sees something it doesn't understand it does a THROW and has no HANDLER to catch that, this resets the stack to something bizarre

STR
interpret('XYZZY') will blow the stack.

Check if should leave !IO converting to RAW - test outside WebStorm

• See TODO-31-RAW

Better handling of memory in preparation for Multitasking or ROMming etc

• Variables should have data in separate place - see #15
• See TODO-27-MEMORY
• See TODO-VM

COLD should do a userInit - etc, and should leave all current definitions still live.
Maybe need to save some variables in userinit space after bootstrap e.g 'EXPECT

Basic idea -

• visit page,
• opens up page with choice of CELLL etc
• creates local instance
• optional UI elements for stack etc

Probably use webcomponents to avoid webpack overhead etc else React

Need ability to do things like
forth.load("https://foo.bar/baz.f") or "./foo.f"

I think there is a standard set of words in ANS Forth - see #12 and ANS 11 or forth-standard.org 11
See also Block Word set ANS 7 or forth-standard.org 7

User initialization is in a JS structure (UserInit) - best to move to COLDD and copy from there.

The console is echoing back to the screen -

STR

• load forth in terminal, through to forth.console()
• type 123 .S
• see both the 123 .S repeated back and th eresult.

• Define DOES
• Test it
• Implement VOCABULARY words as CREATE> DOES - won't do - see #35
• Look at "FORTH" the first word in dic, can it be CREATE> DOES - see #35

Look at making top end event driven - only run EVAL via on('data') event,

Hard part will be locking

One way ..... catch event, if buff is non-zero then append, if its zero , then append and start the Loop to evaluate.

Relevant to #22

eForth is missing some words that many users expect to be there.. These include the following (please add comments for anything else missing).

• CREATE ... DOES>
• VOCABULARY etc see #13 and #35
• DO...LOOP and related I, J. there may be other control structures missing
• ?\ not standard, but looks really useful - see https://arduino-forth.com/article/FORTH_FlashForth_development_conditionalCompilation and https://www.facebook.com/groups/FORTHarduino/permalink/620709772132935
• POSTPONE as alternative to COMPILE and [COMPILE] https://forth-standard.org/standard/core/POSTPONE

The HTML uses Web Components and index.js uses TextEncoders so they should work in any modern browsers
(Firefox, Chrome, Opera, Edge, Android, but not IE or Safari on IOS or OSX).

There are no plans to support older browsers, however I'll be happy to add PRs if someone else does, I'd particularly like to see it work in IOS.

I tried adding a TextEncoder polyfil - but that didnt work and I'm not familiar enough with debugging on Safari to figure out why ....
https://github.com/anonyco/FastestSmallestTextEncoderDecoder

Write a jsWORD equivalent of WORD
use it to recode ' at ~L583

Need a way to pass events to interpreter.

This will be relevant for #22

There are some places errors aren't handled well ... specifically

• In NUMBERQ from parseInt
• Anywhere it does a THROW but isn't inside a CATCH
• See TODO-32-ERRORS

See Discussion on ProgrammingForth FB group; proposal in forth-standard.org different proposal in forth-200x . I think (but am not sure) that the forth-standard.org one is more current, but its devoid of usable examples.

Their is example code for {: in forth200x and forth-standard and https://forth-standard.org/standard/locals which depend on system-dependent code.

Current thinking ... subject to any suggestions below .... is to try and implement the forth-standard.org version by implementing (LOCAL) (in JS) and then using the sample code from there for {:

Sticks a lot of stuff on stack, and doesnt display prompt .

Noted that hi is COMPILE-ONLY but not clear that interpret() checks that correctly yet, as presumed to be used only during source loading.

Idea is to define //TEST that skips ahead unless testing. and use on all lines ending TEST

Either different versions, or better, a single version with a CELLL size parameter.

• Catch all the places where console.assert presumes its CELLL=2
• Adjust the Mstore, SPstore etc to use CELL size
• Incrementally work through checking tests.
• Consider (e.g. via speed tests) using Array instead of Buffer with 32 bit cells. I've heard it is well optimized in JS.
• Catch anywhere access m. or m[ directly

@thomas goeppel reported an issue with console not wrapping

Figure out how to make it Multitask, especially light weight multi-tasks.

• See TODO-28-MULTI

Need a simple UI element to display the stack

• use for Return and Data stack
• Figure out method to trigger it
• FIgure out where to plug that method in