justinethier / cyclone Goto Github PK

EG:

(define (make-thread thunk . name)
...
ok
cyclone> make-thread
(procedure (thunk . name) ...)
cyclone> (make-thread list)
Error: length - invalid parameter, expected list

Should try checking for compiler warnings, at least when building C, EG:

-Wall -Werror

At the moment lists are used as the data structure for these elements. So lookups are O(n). It would improve performance to use a more appropriate data structure such as a hashtable or map.

These functions really should have type checking to prevent segfaults. There will be a performance impact, but it may be possible to mitigate that by adding an optimization level/option that suppresses runtime type checking.

In scheme/base.sld the make-thread function generates a compile error if it is placed above the macro definitions. It seems let is not expanded in this case.

I suspect this is because the same expand pass handles both reading define-syntax forms and expanding existing macros. Maybe we need a separate define-syntax pass? That is not ideal but might be able to avoid this issue.

It takes a long time to process a literal list containing hundreds or even thousands of elements. The compiler eventually crashes with a 'too many arguments' error.

Traces are pointing to wrong files.

For example write should have output "a" below:

cyclone> (write #("a" b (c)))
#(a b (c))

Instead it emits what display would have.

If icyc is passed a filename on the command line, it should evaluate the contents of that file on startup before (instead of?) opening a prompt.

Need to protect the names of identifiers/functions used internally by modules. Might be able to do this by predictably mangling the names of globals, by using the global as well as the name of the module that defines it.

Should make GC more efficient, only transport mutated vector index, not the whole vector

Peter Bex mentions the folllowing in an excellent article about Chicken's GC internals: http://www.more-magic.net/posts/internals-gc.html

The smart reader might have noticed a small problem here: what if the amount of garbage cleaned up is less than the data on the stack? Then, the stack data can't be copied to the new heap because it simply is too small. Well, this is when a third GC mode is triggered: a reallocating GC. This causes a new heap to be allocated, twice as big as the current heap. This is also split in from- and tospace. Then, Cheney's algorithm is performed on the old heap's fromspace, using one half of the new heap as tospace. When it's finished, the new tospace is called fromspace, and the other half of the new heap is called tospace. Then, the old heap is de-allocated.

Presently, cyclone is not smart enough to increase the size of the heap. That logic needs to be added to prevent segfaulting.

Should also come up with a test program or two that demonstrate this.

The following segfaults at the repl:

cyclone> (lambda)

See the api-testing.scm example on master. This code worked fine in the compiler, but not in icyc.

Longer-term will need to handle arbitrary object types:

void gc_mutator_thread_runnable(gc_thread_data *thd, object result)              
{                                                                                
   ...
    if (is_object_type(result)) {                                                
      // TODO: need to move object to heap                                       
      // TODO: also, then need to gc_mark_gray heap obj                          
      fprintf(stderr, "Unhandled object type result, TODO: implement\n");        
      exit(1);                                                                   
    }  

TBD: is this a problem with mutexes, as they return boolean objects?

Presently define-syntax can only be used by compiled code. It should be possible for eval to define a macro as well.

Need to add support for application of pure varargs functions, EG:

     ((lambda test (write test)) 1 2 3 4)

Now that there is a define-c API, there are several non-standard primitives that could transition to this API. This would help simplify the runtime.

Need some prep for this:

• Current master should be tagged as the 0.0.2 release
• Merge itself should be simple
• Performance is disappointing compared between macro-dev2 and master. Should try to profile a bit and see if there are any improvements that can be made. It may also be helpful to find modules that are unchanged between master and macro-dev2 and measure their compile times to see what effect the macro changes have had. I suspect the real issue with slower compilations is because all of the macros in the base module are interpreted during compilation of that module (not the rest of the modules!), which will be much slower than using compiled macros.

The fill argument is optional and should not be required in compiled code.

See: https://matt.sh/howto-c

We might want to update a few things, in particular use of stdint.h

(a 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75)
"producer thread done"
*** Error in `./producer-consumer': realloc(): invalid pointer: 0xb6fa0680 ***
Aborted

Need to be able to prevent collisions with special form symbols.
For example:

(import (scheme base) (scheme write))
(let ((if write))
  (if 1)
  (set! if (lambda (a b c) (write (list a b c))))
  (if 1 2 3))

Note this should also work without the let, in both the compiler and interpreter.

A bytevector-copy! function needs to be implemented.

Need some way of linking to other libraries, in order to compile programs that use them.

This should not be an error:

cyclone> (map (lambda (l) (list l)) `(1 2 3 4 a b c))
Error: Unbound variable c

Neither is implemented properly, per spec. Although it seems truncation division works per spec while floor does not?

This will be necessary to pull in 3rd party headers.

Should detect dependent libraries and build them if necessary. Basically, build them if the .o file does not exist or if it is older than the corresponding source file(s)

Add support for syntax-rules using an existing system such as chibi scheme's ER macros or the alexpander.

Call history is a significant percentage of CPU usage. One way to reduce it might be to only emit call history traces for the top-level function and not for "internal" function calls. Just an idea...

Consider there are no cycles in any of these lists:

cyclone> (list 'a 'a 'a)
(a a a ...)
cyclone> (list 'a 'a)
(a a)
cyclone> (list 'a 'a 'a 'a)
(a a a a ...)
cyclone> (list 'a 'a 'a 'b)
(a a a b ...)
cyclone> (list 'a 'a 'b 'b)
(a a b b ...)
cyclone> (list 'a 'b 'b 'b)
(a b b b)
cyclone> (list 'a 'b 'c 'd)
(a b c d)
cyclone> (list 'a 'b 'c 'd 'd)
(a b c d d)
cyclone> (list 'a 'b 'c 'd 'd 'd)
(a b c d d d)
cyclone> (list 'a 'b 'c 'd 'd 'd 'd)
(a b c d d d d)
cyclone> (list  'b 'c 'd 'd 'd 'd)
(b c d d d d)
cyclone> (list   'c 'd 'd 'd 'd)
(c d d d d ...)

Observed the following error during compilation:

cyclone cyclone.scm
gc_allocated_bytes: unexpected object 0xbf8a807c of type 2
make: *** [cyclone] Error 1

Is it possible to reduce compile/build/test times by using dynamic linking of modules?

Would need to dynamically link the runtime (should be easy) as well as compiler modules (may be difficult).

From notes:

 ;'a'c  ;; TODO: this is still an issue, try it

From Clinger's paper:

For the (include "sets.body.scm") convention to work, implementations must search for the included file within the directory of the including file. Chicken, Gauche, Kawa, Larceny, and Petit Larceny do so, and the development version of Foment 0.5 is said to do so as well.

I think there may be issues here with Cyclone, but need to verify. It may have just been inconvenient to have separate sld and scm files for the compiler.

• (equal?) loops forever when comparing two different circular lists
• printing *global-environment* in the repl still loops forever

It should be possible to port existing benchmarks. Ideally in a way that they can also be run in another scheme or two for comparison.

Should be able to specify as arguments to the compiled executable.

There should be a command line option to print the C compiler commands invoked by Cyclone.

Consider cleaning up the gc blocked/runnable functions to make it more explicit what is going on. In particular, that the runnable function also calls into the current continuation.

When this is done, need to update the user manual to mention these functions if the C code can block.

It should be an error to list a non-existent variable in a library's export list. But instead the compiler silently ignores it.

These variables are used by multiple threads but there is no synchronization:

cached_heap_free_size
cached_heap_total_size

There are many things that could be optimized (but not necessarily all the time) such as:

• gcc optimization level
• suppress runtime type checking (trades speed for safety)
• suppress call history, or reduce level of call history tracking
• etc

Received this error during long-running-process:

Error: Call of non-procedure: ((if sleep? (thread-sleep! 1000) #f) (loop))
Call history:
scheme/cyclone/transforms.sld:ref?
scheme/cyclone/transforms.sld:expand
scheme/cyclone/transforms.sld:const?
scheme/cyclone/transforms.sld:prim?
scheme/cyclone/transforms.sld:ref?
scheme/cyclone/transforms.sld:expand-body
scheme/base.sld:reverse
scheme/base.sld:foldl
scheme/write.sld:write

From the spec regarding - and /:

With one argument, however, they return the additive or multiplicative inverse of their argument

ER macros have rename/compare, but rename is not hygienic (at least not completely). In order for it to be correct we will probably have to add syntactic closure support in some form, or at least the equivalent.

EG:

cyclone> (call/cc (lambda (k) (k 1)))
Error: Expected 2 arguments but received 1.

The extra 'cont' param is being exposed, because this seems to work:

(call/cc (lambda (k) (k 1 1)) (lambda (k) (k 1 1)))