irmen / prog8 Goto Github PK
View Code? Open in Web Editor NEWhigh level programming language and compiler targeting 6502 machines such as the C-64 and CommanderX16
Home Page: https://prog8.readthedocs.io/
License: Other
high level programming language and compiler targeting 6502 machines such as the C-64 and CommanderX16
Home Page: https://prog8.readthedocs.io/
License: Other
The docs for prog8 state that the system targets V39 of the Commander X16 ROMS.
However the latest version available on the Commander X16 github is V38 (as of 11.06.2021)
So far what I have compiled has worked on V38.
The prog8 documentation as it stands seems a little confusing.
travis.ci stopped working. https://travis-ci.org/github/irmen/prog8
move it over to github actions
see pyminiaudio
right now the irq-handler will use the same eval stack as the regular code.
This will cause issues when the irq is triggered while stack manipulations occur in regular code.
Irq handler should get its own eval stack.
.... is it acceptable to take the lower say 100 bytes of the normal stack (the size of which will be reduced to 156 bytes then instead of 256). Saves one whole memory page...
ALSO: the interrupt handler should backup the contents of the handful of zeropage scratch addresses and restore them on exit (SCRATCH_ZP_REG etc etc)
investigate support for 8bitguy's Commander X16 platform https://www.commanderx16.com and https://github.com/commanderx16/x16-docs
For instance when compiling cx16assem:
Processing for target cx16...
src/assembler.p8:426:31: WARNING this asmsub also has one or more return 'values' in one of the status flags
src/assembler.p8:458:33: WARNING this asmsub also has one or more return 'values' in one of the status flags
/home/irmen/Projects/cx16assem/src/asmsymbols.p8:86:21: WARNING this asmsub also has one or more return 'values' in one of the status flags
/home/irmen/Projects/cx16assem/src/asmsymbols.p8:118:21: WARNING this asmsub also has one or more return 'values' in one of the status flags
/home/irmen/Projects/cx16assem/src/asmsymbols.p8:162:21: WARNING this asmsub also has one or more return 'values' in one of the status flags
Seems like the position object now sometimes contains full absolute paths
that's not bad in itself but
java -jar prog8compiler-6.4-all.jar prog8-issue34.p8
%import textio
%zeropage basicsafe
%option no_sysinit
main {
byte[] xs = "foo" ; <<<<<<<<<<<<
sub start() {
txt.print(xs)
}
}
* internal error *
Exception in thread "main" java.lang.NullPointerException
at prog8.compiler.astprocessing.AstChecker.visit(AstChecker.kt:631)
at prog8.ast.statements.VarDecl.accept(AstStatements.kt:242)
at prog8.ast.walk.IAstVisitor$DefaultImpls.visit(IAstVisitor.kt:30)
at prog8.compiler.astprocessing.AstChecker.visit(AstChecker.kt:182)
at prog8.ast.statements.Block.accept(AstStatements.kt:66)
at prog8.ast.walk.IAstVisitor$DefaultImpls.visit(IAstVisitor.kt:14)
at prog8.compiler.astprocessing.AstChecker.visit(AstChecker.kt:48)
at prog8.ast.Module.accept(AstToplevel.kt:365)
at prog8.ast.walk.IAstVisitor$DefaultImpls.visit(IAstVisitor.kt:10)
at prog8.compiler.astprocessing.AstChecker.visit(AstChecker.kt:44)
at prog8.compiler.astprocessing.AstExtensionsKt.checkValid(AstExtensions.kt:14)
at prog8.compiler.CompilerKt.processAst(Compiler.kt:257)
at prog8.compiler.CompilerKt.compileProgram(Compiler.kt:97)
at prog8.CompilerMainKt.compileMain(CompilerMain.kt:106)
at prog8.CompilerMainKt.main(CompilerMain.kt:23)
AstChecker.kt
, there's the private fun checkValueTypeAndRangeString(...) : Boolean
called from inside visit(...)
. It seems the return value is not used.[
Sorry for pressing on you, @irmen. But I only got the weekend and I need to at least write it down now.
And again: PLEASE don't be put off by the style of writing. I'm just trying to be concise. No offense of any sort intended; on the contrary!
What I'm laying out looks like a total mess (= what's in my head) - which does not at all mean that there isn't a simple solution.
Likewise: it's totally possible that I'm just dumb and not seeing things.
]
Looking at the constructor of AST node Module
,
class Module(override val name: String,
override var statements: MutableList<Statement>,
override val position: Position,
val isLibraryModule: Boolean,
val source: Path): Node, INameScope
together with that of Position
,
data class Position(val file: String, val line: Int, val startCol: Int, val endCol: Int)
we can see that name
, position
(via Position
's file
) and source
- plus, maybe?, isLibraryModule
- are somewhat interdependent/redundant, and not entirely clear wrt. their exact meaning.
source: Path
prog8lib/
in their file system, and stated it via CLI option -libdirs as a location to search in?Module
instance does not come from any source code file at all? Think of "prog8_interned_strings", or just plain Strings as in testing. Maybe also, eventually, precompiled libraries.position: Position
file
: same as above, only that it's a String instead of a Pathline
, startCol
, endCol
:
Node
's position and mark the chunk of text from which that Module
was parsed. But then that would technically be from 1st character up to last character of that text, which isn't even expressible with just line
, startCol
and endCol
.%import
directive elsewhere. But then it's impossible to properly define them:
%import
for the Module
at all ("prog8_interned_strings", and also the "root" module, the user "program")%import
for it, which is the general case because a module can be %import
ed from more than one other module, hence not uniquely definable.name: String
source
s / position
's file name without .p8
extension, thus same problems as theseModule
was already %import
ed by some other module. Since it's only the file name, this means that you can't have different modules of the same name
from different locations (and of course no user module in a file that happens to be named "prog8_interned_strings.p8")isLibrary: Boolean
isLibrary == true
)source.toString().contains("prog8lib")
? But then again what about a folder of that name in -libdirs?java -jar prog8compiler-6.4-all.jar test.p8
or with -noopt
pokew(non_zp_ptr_var + CONST_OFFSET, some_value)
non_zp_ptr_var + CONST_OFFSET
, stores the sum somewhere in zeropage, eg P8ZP_SCRATCH_W1
and then
lda some_value
ldy #0
sta (P8ZP_SCRATCH_W1),y
lda some_value+1
iny
sta (P8ZP_SCRATCH_W1),y
CONST_OFFSET
is small enough (ie. a byte) - stores just non_zp_ptr_var
in zeropage (say, P8ZP_SCRATCH_W1
), loads the CONST_OFFSET
into Y
and then
lda some_value
ldy #CONST_OFFSET
sta (P8ZP_SCRATCH_W1),y
lda some_value+1
iny
sta (P8ZP_SCRATCH_W1),y
non_zp_ptr_var
is used directly:
lda some_value
ldy #CONST_OFFSET
sta (non_zp_ptr_var),y ; <<<<<<<<<
lda some_value+1
iny
sta (non_zp_ptr_var),y ; <<<<<<<<<
.\test.asm:4807:8: error: not a direct page address address '($1439),y'
sta (non_zp_ptr_var),y
^
pokew
, but that may well be just one instance.non_zp_ptr_var
actually were a zeropage var, then the actually generated code would be fine. So: one necessary condition is that the compiler runs out of zeropage locations to put local variables.P8ZP_SCRATCH_W1
(or P8ZP_SCRATCH_W2
which exists as well)P8ZP_SCRATCH_W2
, as shown in "Expected". So my guess is that the compiler "thought" it ran out of scratches, so to speak.pokew
- maybe not the only instance, as said - in Prog8, no asm:
@(non_zp_ptr_var + CONST_OFFSET) = lsb(some_value)
@(non_zp_ptr_var + CONST_OFFSET + 1) = msb(some_value)
Referencing labels is problematic because of the preceding underscore in the generated code. Should we just remove that prefix?
This fails to assemble, because 64tass can't find the symbol.
%import textio
main {
str myBar = "main.bar"
foo_bar:
%asminclude "compiler/test/fixtures/foo_bar.asm" ; FIXME: should be accessible from inside start() but give assembler error
sub start() {
txt.print(myBar)
txt.print(&foo_bar)
return
}
}
See test fixture file asmIncludeFromSameFolder.p8
This is just a small annoyance and possibly just cause by my choice of editor (Emacs). The editor does not automatically add a newline at the last line when editing normally.
test.p8:9:1: missing EOL at ''
Attaching a minimal file that reproduce the error.
cbmcodecs2 got a few changes for the box drawing characters
irmen/cbmcodecs2#2
petscii table in prog8 should get these as well?
to replace kotlin.test / junit / hamcrest hodgepodge
or
and and
currently don't short-circuit.
maybe we should make them short-circuit?
how can this be done in the expression code generation? It needs to be aware of the preceding logical operator's result and branch to the end if it detects short circuit possibility? Is this even worth the hassel. sounds like it will make compount logical expressions very slow on 6502 (unless ofcourse the operands are hard to calculate and a short circuit avoids a whole calculation...)
See comment here https://www.reddit.com/r/programming/comments/alhj59/creating_a_programming_language_and_cross/eg898b9
wordarray[[byteindex]] could be easier to generate efficient loop code for than wordarray[index]
or it might not... we could also try to generate more optimized code for the iteration form of the for loop?
-emu doesn't seem to take the output directory into account.
Hello :)
I got a short question today: can I use my own constants in an assembler code? I can see you use it in your libraries, for example in syslib.p8:
asmsub init_system() {
%asm {{
lda #$80
sta VERA_CTRL
However, when I try to use my constant, i.e:
const ubyte SCREEN_BACKGROUND_COLOR = 1
asmsub set_pallete_color(ubyte color @ A) {
%asm {{
lda #SCREEN_BACKGROUND_COLOR
it fails on assembling phase since the constant's name isn't replaced by value nor there is SCREEN_BACKGROUND_COLOR defined in *.asm file.
java -jar prog8compiler-7.0-all.jar floatArrayInitializerWithRange.p8
floatArrayInitializerWithRange.p8
%option enable_floats
main {
sub start() {
float[] cs = 1 to 42 ; values are computed at compile time
cs[0] = 23 ; keep optimizer from removing it
}
}
Prog8 compiler v7.0 by Irmen de Jong ([email protected])
This software is licensed under the GNU GPL 3.0, see https://www.gnu.org/licenses/gpl.html
Compiler target: c64. Parsing...
importing 'floatArrayInitializerWithRange'
importing 'syslib' (library)
importing 'math' (library)
importing 'prog8_lib' (library)
Processing for target c64...
* internal error *
Exception in thread "main" prog8.ast.base.FatalAstException: range expressions in vardecls should have been converted into array values during constFolding VarDecl(name=cs, vartype=VAR, datatype=ARRAY_F, value=RangeExpr(from NumericLiteral(UBYTE:1), to NumericLiteral(UBYTE:42), step NumericLiteral(UBYTE:1), pos=[floatArrayInitializerWithRange.p8: line 4 col 22-29]), pos=[floatArrayInitializerWithRange.p8: line 4 col 9-29])
at prog8.compiler.astprocessing.AstChecker.visit(AstChecker.kt:537)
at prog8.ast.statements.VarDecl.accept(AstStatements.kt:228)
at prog8.ast.walk.IAstVisitor$DefaultImpls.visit(IAstVisitor.kt:39)
at prog8.compiler.astprocessing.AstChecker.visit(AstChecker.kt:253)
at prog8.ast.statements.Subroutine.accept(AstStatements.kt:643)
at prog8.ast.walk.IAstVisitor$DefaultImpls.visit(IAstVisitor.kt:30)
at prog8.compiler.astprocessing.AstChecker.visit(AstChecker.kt:207)
at prog8.ast.statements.Block.accept(AstStatements.kt:69)
at prog8.ast.walk.IAstVisitor$DefaultImpls.visit(IAstVisitor.kt:14)
at prog8.compiler.astprocessing.AstChecker.visit(AstChecker.kt:56)
at prog8.ast.Module.accept(AstToplevel.kt:369)
at prog8.ast.walk.IAstVisitor$DefaultImpls.visit(IAstVisitor.kt:10)
at prog8.compiler.astprocessing.AstChecker.visit(AstChecker.kt:52)
at prog8.compiler.astprocessing.AstExtensionsKt.checkValid(AstExtensions.kt:14)
at prog8.compiler.CompilerKt.processAst(Compiler.kt:264)
at prog8.compiler.CompilerKt.compileProgram(Compiler.kt:98)
at prog8.CompilerMainKt.compileMain(CompilerMain.kt:115)
at prog8.CompilerMainKt.main(CompilerMain.kt:21)
Discussion:
ConstantIdentifierReplacer.before(VarDecl, Node)
if decl.dataType == DataType.FLOAT
, around line 117 of compiler/src/prog8/optimizer/ConstantIdentifierReplacer.kt
where
range initializers are not covered, whereas they are for the other array types. However: why is it then not also showing if array size is given explicitly?Remarks:
RangeExpr
now, but not in order to solve this bug (I don't know how) but rather because it depends on IStringEncoding
, in particular its methods .toConstantIntegerRange
and .size
add a 'continuous mode' where it watches the files it used to compile the program for changes, and automatically recompiles.
To avoid having to warm up the jvm/jit everytime
Most metadata from the AST is lost when generating the IL.
It could be beneficial to keep it around, which allows the code generator to be more advanced.
Discussion here https://www.reddit.com/r/programming/comments/alhj59/creating_a_programming_language_and_cross/eg65te1
Hi,
there seems to be a bug in the optimizer when it comes to constant folding/propagation (wrt. strings/characters in this case).
java -jar ../prog8compiler-6.4-all.jar -target cx16 p8-bug_str-opt.p8
%target cx16
%import textio
%zeropage basicsafe
%option no_sysinit
main {
str input = "1234"
ubyte index = 0
str temp = "?"
sub start() {
ubyte @zp ch
repeat {
ch = input[index]
when ch {
0 -> {
break
}
else -> {
temp[0] = ch ; <<<<<<<<<<<<<
txt.print(temp) ; wrongly optimized to
; lda #$3f
; jsr txt.chrout
; with -noopt it's correctly turned into
; ldy #>temp
; ldy #<temp
; jsr txt.print
}
}
index++
}
return
}
}
; src line: p8-bug_str-opt.p8:21
lda ch
sta temp+0
; src line: p8-bug_str-opt.p8:22
jsr txt.chrout
; src line: p8-bug_str-opt.p8:21
lda ch
sta temp+0
; src line: p8-bug_str-opt.p8:22
lda #$3f
jsr txt.chrout
-noopt
of course correct code is generatedtemp
to be only one character long (so we could use txt.chrout
rather than txt.print
. This is by no means a simple "peephole" optimization, so better not try such "stunts" at all - at least at this stage..asm
files for convenience if desired, but rather not here in the OP, as they are of course rather long.the pascal-string type (with the length prefix byte) should be removed, so that the string type (where string ends with a 0-byte) remains.
This reduces the number of string types from 4 to 2, and avoids having to code two types of loops every time when we want to deal with a string.
While pascal strings could be faster or more flexible, the c-string type is actually used by several kernel routines in the C64 and having just a pascal string type will make them unusable
In Millfork you can extract the data from variables like this:
**code_start
word mydata = 258 ; that would be $0102 in hex.
byte look_lo = mydata.b0
byte look_hi = mydata.b1
**code_end
look_lo whould have the value 2
look_hi whould have 1
Is there a way to do that in Prog8?
I did write about my problems with %asmbinary in the X16 forum, but I actually did find a workaround for that part after thinking about the problem a bit more.
I added an small array in from of the import and can use that to figure out the start address of the actual data. Very hacky though.
However, the actual import by 64tass fails.
%asmbinary "data.bin"
then fails with
.\screen_test.asm:40:10: warning: the file's real name is not './data.bin' [-Wportable]
.binary "./data.bin"
^
Seem like 64tass doesn't like the ./ prefix for path. I can assemble this fine if I just remove this and run 64tass myself.
I'd like prog8 to better support other 6502 machines as well. But I know very little of the others, and any help here would be very much appreciated
Here is a guide to help fill out the blanks required to add a new target to the compiler: https://prog8.readthedocs.io/en/latest/portingguide.html
%import textio ; txt.*
main {
sub start() {
; ATTENTION: uncomment only one problematic line at a time!
; Normal string literals, i.e. PETSCII encoding
; ---------------------------------------------
txt.print("\"") ; fine
txt.print("\n") ; fine
txt.print("\r") ; fine
; txt.print("\\") ; yields CharConversionException
; txt.print("xyz\\") ; yields prog8.compiler.AssemblyError
; @-strings, i.e. translated into
; the alternate character encoding (Screencodes/pokes)
; ----------------------------------------------------
txt.print(@"\"") ; fine
txt.print(@"\n") ; yields CharConversionException
; txt.print(@"xyz\n") ; yields prog8.compiler.AssemblyError
; txt.print(@"\r") ; yields CharConversionException
; txt.print(@"xyz\r") ; yields prog8.compiler.AssemblyError
; txt.print(@"\\") ; yields CharConversionException
; txt.print(@"\\") ; yields prog8.compiler.AssemblyError
; there may be more...
}
}
CharConversionException
in the case of one-letter only strings (eg @"\\"
, "\\"
)prog8.compiler.AssemblyError
in the case of longer strings (eg @"xyz\\"
, "xyz\\"
)@"\n"
and @"xyz\n"
cases; the others are similar
* internal error *
Exception in thread "main" java.io.CharConversionException: no lowerScreencode character for '
' (10)
at prog8.compiler.target.c64.Petscii.encodeScreencode(Petscii.kt:1088)
at prog8.compiler.target.Cx16Target.encodeString(ICompilationTarget.kt:92)
at prog8.optimizer.StatementOptimizer.after(StatementOptimizer.kt:103)
at prog8.ast.walk.AstWalker.visit(AstWalker.kt:247)
at prog8.ast.statements.FunctionCallStatement.accept(AstStatements.kt:547)
at prog8.ast.walk.AstWalker.visit(AstWalker.kt:232)
at prog8.ast.statements.Subroutine.accept(AstStatements.kt:670)
at prog8.ast.walk.AstWalker.visit(AstWalker.kt:218)
at prog8.ast.statements.Block.accept(AstStatements.kt:67)
at prog8.ast.walk.AstWalker.visit(AstWalker.kt:194)
at prog8.ast.Module.accept(AstToplevel.kt:366)
at prog8.ast.walk.AstWalker.visit(AstWalker.kt:188)
at prog8.optimizer.ExtensionsKt.optimizeStatements(Extensions.kt:49)
at prog8.compiler.CompilerKt.optimizeAst(Compiler.kt:270)
at prog8.compiler.CompilerKt.compileProgram(Compiler.kt:99)
at prog8.CompilerMainKt.compileMain(CompilerMain.kt:106)
at prog8.CompilerMainKt.main(CompilerMain.kt:23)
* internal error *
Exception in thread "main" prog8.compiler.AssemblyError: There was a problem converting a string to the target machine's char encoding: no lowerScreencode character for '
' (10)
at prog8.compiler.target.cpu6502.codegen.AsmGen.encode(AsmGen.kt:254)
at prog8.compiler.target.cpu6502.codegen.AsmGen.vardecls2asm(AsmGen.kt:393)
at prog8.compiler.target.cpu6502.codegen.AsmGen.block2asm(AsmGen.kt:198)
at prog8.compiler.target.cpu6502.codegen.AsmGen.compileToAssembly(AsmGen.kt:62)
at prog8.compiler.CompilerKt.writeAssembly(Compiler.kt:312)
at prog8.compiler.CompilerKt.compileProgram(Compiler.kt:105)
at prog8.CompilerMainKt.compileMain(CompilerMain.kt:106)
at prog8.CompilerMainKt.main(CompilerMain.kt:23)
-noopt
irrelevant for the prog8.compiler.AssemblyError
txt.print(@'\n')
, txt.print(@'\r')
, txt.print(@'\\')
- does yield a proper syntax error, even though characters aren't properly escaped there eitherCurrently you can use str
only in asmsub
signature, for regular subroutines the compiler gives an error about that reference types should be an uword
.
examples/test.p8:10:0: ERROR Pass-by-reference types (str, array) cannot occur as a parameter type directly. Instead, use an uword to receive their address, or access the variable from the outer scope directly.
This should be fixed so that str
can be used in signature .
(The issue is related to the way subroutine params are treated as local variables.)
Ideally we do the same for array types but that's a different issue, one that is difficult to solve because the compiler now treats array vars as a direct (name) reference to the array in memory. Passing different arrays in the place of an array var can't be done right now.
the following code crashes the compiler:
ubyte[] ubs = [1]
txt.print_uw(ubs[0])
expected result: compiles fine, prints 1
(it should successfully typecast the argument from ubyte to uword)
observed result:
compiler crashes with require() not met (the requirement is that it expects a byte/ubyte datatype):
* internal error *
Exception in thread "main" java.lang.IllegalArgumentException: Failed requirement.
at prog8.compiler.target.cpu6502.codegen.assignment.AssignmentAsmGen.assignRegisterByte$compiler(AssignmentAsmGen.kt:1369)
at prog8.compiler.target.cpu6502.codegen.assignment.AssignmentAsmGen.translateNormalAssignment(AssignmentAsmGen.kt:75)
at prog8.compiler.target.cpu6502.codegen.assignment.AssignmentAsmGen.assignExpressionToRegister$compiler(AssignmentAsmGen.kt:2162)
at prog8.compiler.target.cpu6502.codegen.assignment.AssignmentAsmGen.assignTypeCastedValue(AssignmentAsmGen.kt:409)
at prog8.compiler.target.cpu6502.codegen.assignment.AssignmentAsmGen.translateNormalAssignment(AssignmentAsmGen.kt:155)
at prog8.compiler.target.cpu6502.codegen.AsmGen.translateNormalAssignment$compiler(AsmGen.kt:850)
at prog8.compiler.target.cpu6502.codegen.FunctionCallAsmGen.argumentViaRegister(FunctionCallAsmGen.kt:341)
at prog8.compiler.target.cpu6502.codegen.FunctionCallAsmGen.translateFunctionCall$compiler(FunctionCallAsmGen.kt:70)
....
The AST passed to the codegenerator is fine btw, it contains the cast:
txt.print_uw((ubs[0] as uword) )
in prog8lib.asm
Just as the evaluation stack, store the bytes in 'separate' arrays in memory.
This will break the few places where we now depend on word arrays to be sequential in memory so that needs to go (most notably the SPXYW array for the sprite x/y coordinate pairs) or we need to support both array layout types.
Changing this also requires rewriting all the code that now deals with arrays and assumes the values are sequential in memory....
Should this fail if you do NOT use -target cx16 ?
PS D:\CommanderX16\Prog8> & 'C:\Program Files\AdoptOpenJDK\jre-11.0.11.9-hotspot\bin\java.exe' -jar D:\CommanderX16\Prog8\prog8compiler-7.0-all.jar .\cx16\amiga.p8
Prog8 compiler v7.0 by Irmen de Jong ([email protected])
This software is licensed under the GNU GPL 3.0, see https://www.gnu.org/licenses/gpl.html
Compiler target: c64. Parsing...
importing 'amiga' (from 'cx16\amiga.p8')
importing 'textio' (library)
importing 'syslib' (library)
importing 'conv' (library)
โ[91m[cx16\amiga.p8: line 3 col 1-16] Import: no module source file 'palette.p8' found (I've looked in: embedded libs and [cx16, D:\CommanderX16\Prog8\prog8lib])
Works fine if I add -target cx16 to the command-line, but the source-code already have %target cx16 ?
PS it does this on the examples files for x16
[
Here's a little write-up of how I currently understand things to work in compilerAst
, parts of which may be of use for a future guide on how to get going/engaged with development of the compiler.
In addition I'm trying to evaluate the pros and cons of the current approach and offer some ideas re possible other approaches.
@irmen: this is NOT URGENT AT ALL, rather a recap for myself and some outlook. Up (maybe) only after "the plan" (CompilerDevelopment.md).
]
parser/
.compilerAst/
, which is to provide an interface to this generated thing - purely in Kotlin. Here's also where we put tests of the grammar (via the Kotlin parser), and of the AST nodes.compilerAst
is to turn this parse tree into an AST (Abstract Syntax Tree), which
prog8.parser.ast.Node
s, no more (Java) prog8.parser.Prog8ANTLRParser.ParserRuleContext
s.%import
resolution, it simply puts such directives (similar: %asminclude
, %asmbinary
) in the AST as is.Aside from error handling and possibly recovery, and looking only at the parse tree -> AST conversion:
Antlr2Kotlin.kt
, via extension methods .toAst(...)
on ParserRuleContext
subclasses..toPosition()
(also in Antlr2Kotlin.kt
).ModuleContext.toPosition(..)
, which is the only such ext. method visible from the outside.Node
constructors, child nodes have to be instantiated before the parent node can be.this
in ext. methods (here: the ParserRuleContext
instance), hence no need to write it (or "it"...)IStringEncoding
(atm, but hopefully soon to go, see #50)Node
with a proper Position
(-> ext. method .toPosition()
)Module
instance is built at the very last. Likewise for any other non-leaf Node
...Node
s .parent
has to be lateinit
; and there's Node.linkParents(...)
(although this linking might be possible without it, even with the current approach)Node
s .position
is (atm) NOT lateinit
, leading to a variety of weird and intricate problems (the Position
s .file
being the cause).ParserRuleContext
instances from the ANTLR-generated parser for anything other than just turning them into our own Kotlin objects.XyzContext
..!EOL
tokens, commas, etc. - in a children
list on each ParserRuleContext
, but that can be switched off. Atm, it is switched on, but we don't use any of that (I think). It's probably not so big an impact, but still work to do for the garbage collector.enterXyzContext
/exitXyzContext
"events") and/or b) a visitor interface (= visitXyzContext(...)
callbacks). However, as alluded to in parentheses, neither of them uses double-dispatch, meaning you're stuck with those all-to-chatty names. Currently, we don't use them, and neither of these options are enabled.[
Remark: from here on it's pretty much "TODO"/rough sketch. Sorry for that, but it's late and a lot already. I'll edit and amend this OP later, possibly incorporating comments, if any.
]
During conversion, it would be nice to have the sharing of state...
IAstVisitor
, AstWalker
(even though somewhat "unusual").TODO: example client code
Node
subtypesTODO: example client code
we have &
to make a pointer to a value,
we need @ (pointer) to be able to dereference it again to get the actual value once more.
Problem: we don't have a true pointer type so we can't dereference pointers (UWORDS) without explicitly telling the compiler what type it is we expect.... something like:
@ (pointer) as str
@ (pointer) as ubyte[3]
should probably be used (where pointer is a UWORD)
note: @ ( &variable ) is ok already because it is reduced in the AST to only the variable
Using somthing like this
ubyte[] ar = [-2]
gives
error.p8:9:18: ERROR cannot type cast to string or array type
which is a bit confusing since I assume it's assigning a byte to a ubyte arrray that fails.
Note that I did have this somewhere like
ubyte[] ar = [ ..., CONST-2, ... ]
which made a bit hard to figure out what was wrong.
min(floatarray) and max(floatarray) were broken
a correct implementation should be added in c64flt.p8 for func_min_f and func_max_f
Hello!
I have developed an extension for Visual Studio Code to proper Prog8 syntax highlighting. It's far from being perfect but it helps a lot: https://github.com/akubiczek/Prog8-TmLanguage-VsCode
There is also a regular Textmate XML that can be used in other tools.
best regards
Adam
Hi Irmen!
When trying to use the option -watch on MacOS (Big Sur v 11.0.1), i.e:
java -jar prog8compiler-5.0-all.jar -target cx16 -watch test.p8
it gives me a result:
Imported files (now watching:)
test.p8
Exception in thread "main" java.lang.NullPointerException: Cannot invoke "java.nio.file.Path.register(java.nio.file.WatchService, java.nio.file.WatchEvent$Kind[])" because the return value of "java.nio.file.Path.getParent()" is null
at prog8.CompilerMainKt.compileMain(CompilerMain.kt:71)
at prog8.CompilerMainKt.main(CompilerMain.kt:21)
Java version:
java 15 2020-09-15
Java(TM) SE Runtime Environment (build 15+36-1562)
Java HotSpot(TM) 64-Bit Server VM (build 15+36-1562, mixed mode, sharing)
This may be fixed in the latest source but I thought I should report this just in case. The following doesn't work
main {
ubyte[] bytearr = [0,1,2,3,4]
sub start() {
uword ref = &bytearr
ubyte get = ref[0]
}
}
I get
* internal error *
Exception in thread "main" java.lang.IllegalArgumentException: Failed requirement.
at prog8.ast.statements.VarDecl.replaceChildNode(AstStatements.kt:235)
at prog8.ast.walk.IAstModification$ReplaceNode.perform(AstWalker.kt:62)
at prog8.ast.walk.AstWalker.applyModifications(AstWalker.kt:178)
at prog8.compiler.astprocessing.AstExtensionsKt.reorderStatements(AstExtensions.kt:26)
at prog8.compiler.CompilerKt.processAst(Compiler.kt:252)
at prog8.compiler.CompilerKt.compileProgram(Compiler.kt:97)
at prog8.CompilerMainKt.compileMain(CompilerMain.kt:106)
at prog8.CompilerMainKt.main(CompilerMain.kt:23)
I've attached a small test program for testing
Hello again :)
Is there any limit of variables I can declare?
Look at this code snippet:
-target cx16
ubyte i
for i in 0 to BOARD_FIELDS - 1 {
if (board_fields[i-5] == 4) {
txt.plot(30, 12)
}
It compiles to:
;if (board_fields[i-5] == 4) starts here:
lda i
sta prog8_autovar_index_1835841364
sec
sbc #5
sta prog8_autovar_index_1835841364
ldy prog8_autovar_index_1835841364
lda board_fields,y
;this is important:
sta P8ESTACK_LO,x
dex
lda #4
sta P8ESTACK_LO,x
dex
jsr prog8_lib.equal_b ;this comparison sometimes fails
inx
lda P8ESTACK_LO,x
beq _prog8_label_51_if_end
ldy #$1e
lda #12
jsr txt.plot
It works for a while until it stops working :) The problem is with prog8_lib.equal_b procedure and X-register. I have debugged code and found that X-register is equal to $00 what causes problem:
;X = $00
;this is important:
sta $0400,x
dex ;X = $ff
lda #04
sta $0400,x
dex ;X = $fe
jsr prog8_lib.equal_b
inx
lda $0400,x
beq _prog8_label_51_if_end
prog8_lib.equal_b:
lda $0402,x ;X = $fe, $402+$FE=$500 WRONG!
cmp $0401,x
So it should compare $0400 to $04ff - instead it compares $0500 ($402+$FE) to $04ff ($401+$fe).
I found a solution, I can change:
if (board_fields[i-5] == 4) {
into:
ubyte val = board_fields[i-5]
if (val == 4) {
And it's even faster because it doesn't use prog8_lib.equal_b anymore but I would like to know why the problem exists in my original case.
for instance
inline sub color_word(ubyte ix) -> uword {
uword color = mkword(reds[ix], (greens[ix] << 4) | blues[ix] )
return color
}
this behaves wrong. When it is not inlined it's ok
Also when you omit the return, no error or warning is issued
Prog8 currently (v6.4) supports two character encodings: PETSCII (eg "foo"
) and screencodes/pokes (eg @"foo"
).
Now there's also Commander X16's "ISO mode".
However, there's no easy way in Prog8 to use strings or characters (literals) in this encoding.
Add some kind of support for switching to/from this and other encodings to the language.
The exact syntax, semantics and mechanism remain to be discussed.
\xHH
escapes inside strings, these are copied verbatim without translation.
.enc
, .cdef
and .edef
which let you define a custom encoding.asm
file - as is done atm - the literals from the p8 source could be pasted in there pretty much as-is\"
which would have to be replaced by eg \{qq}
(see demo code)@"string"
notation for screencodes/pokes - which is a bit unfortunate, as "@" probably rather hints to ASCII...%target cx16
. Or maybe better not as restrictive, and just warn about it.@
is also used for indicating where parameters go in asmsubs - something completely different.%option encoding_xyz
, setting the default for the whole src file (of course not imported ones)@
syntax, IMHOOf course these are just a few thoughts of mine on the subject - looking forward to anybody else's :)
I've written a little demo, with explanations in comments. But first, here's how it looks like:
; x16emu -prg prog8_issue38.prg -run -gif prog8_issue38.gif,wait
%target cx16
%import textio ; txt.*
%import syslib ; cx16.*, sys.*
%zeropage basicsafe
%option no_sysinit
main {
; Define the encoding at top of block
; NOT inside a sub!
%asm {{
.enc "ascii" ; define an encoding named ascii
.cdef " ~", 32 ; identity for printable characters (from " " to "~")
.edef "\r", 13 ; define escape sequence for CR carriage return
.edef "\n", 13 ; define escape sequence for LF line feed, map it to CR (original: $0a)
.edef "\\", $5c ; define escape sequence for backslash (cannot be used in P8's PETSCII enc)
.edef "\t", [32, 32, 32, 32] ; TAB (original $09) -> four spaces
.edef "\{clr}", 147 ; we can even define nice custom control sequences
.edef "\{qq}", $22 ; or this: double quote (cannot define \")
}}
; String literals must also be defined via %asm; also in block
%asm {{
ascii_strings .proc ; create a 64tass scope s.t. we can give them the same names as in p8
.enc "ascii" ; not actually necessary since directly following the definition
mytext
.text "This is ASCII: lower & UPPER\nsecond\tline\twith\ttabs\r"
.text "backslash in double quotes: \{qq}\\\{qq}"
.byte 0 ; null-terminated for txt.print
.pend
}}
; We can still use normal PETSCII by switching back via .enc "none":
%asm {{
petscii_strings .proc
.enc "none"
myothertext
.text "we are petscii (all upper case)"
.byte 0 ; null-terminated for txt.print
.pend
}}
; inline subs for convenient switching between modes
inline sub iso_on() {
txt.chrout($0f)
}
inline sub iso_off() {
txt.chrout($8f)
}
sub start() {
void cx16.screen_set_mode(0) ; switch to 40x30 text mode
iso_on(); switch to ISO-8859-15 encoding ("ISO mode")
; We cannot (yet?) use iso-/ascii-encoded literals in Prog8
; because they're always a) translated and b) put "some place we don't know" (see issue #31)
str garbled = "this is garbled: lower & UPPER\nsecond line\rdouble quote: \""
txt.print(garbled)
txt.nl()
txt.nl()
txt.nl()
; Neither can we use escapes \\ (because there is no petscii equiv) nor \t (missing?)
; ...not to speak of our custom ones like \{clr} (of course).
; So each of the following would yield Prog8 syntax errors:
;str backslash = "\\"
;str tab = "\t"
;str clr = "\{clr}"
; Workaround:
; 1. put actual ASCII-encoded text in block (above) via %asm
; 2. declare Prog8 uword to point to actual text
; 3. with %asm: initialize the pointer accordingly
; That's basically what the compiler would do as well.
uword mytext
%asm {{
lda #<ascii_strings.mytext
sta mytext
lda #>ascii_strings.mytext
sta mytext+1
}}
txt.print(mytext)
txt.nl()
txt.nl()
txt.nl()
uword myothertext
%asm {{
lda #<petscii_strings.myothertext
sta myothertext
lda #>petscii_strings.myothertext
sta myothertext+1
}}
txt.print(myothertext)
txt.nl()
txt.nl()
txt.nl()
sys.wait(60) ; wait a second
; make single screenshot
%asm {{
lda #1
sta $9FB5
}}
}
}
The various phases in the compiler could use more optimization steps.
I.e I'm working on some code that is almost platform independent (so far). I just have 1 line of CX16 specific code I need to remove to get my code to compile on both platforms. Basically I'm asking for something like this
#ifdef TARGET_C64
void cx16.screen_set_mode(0)
#endif
If I ever consider adding sound this might possible be done with a similar solution just around the %import
#ifdef TARGET_C64
%import c64_sound
#endif
#ifdef TARGET_CX16
%import cx16_sound
#endif
or perhaps some target specific flag like this
%import %target%_sound
might be possible instead
issue46_main.p8
:
%import issue46_imported
main {
sub start() {
}
}
issue46_imported.p8
:
foo {
str str ; parse error (non-lexical) -> BUG
str bar = "missing closing quotes (lexical error) -> NO bug
} ; incidentally, the previous lexical error yields another non-lexical one -> BUG
Prog8 compiler v6.4 by Irmen de Jong ([email protected])
This software is licensed under the GNU GPL 3.0, see https://www.gnu.org/licenses/gpl.html
Compiler target: c64. Parsing...
importing 'issue46_main'
importing 'issue46_imported' (from '.\issue46_imported.p8')
.\issue46_imported.p8:3:11: token recognition error at: '"missing closing quotes (lexical error) -> no bug\r'
.\issue46_imported.p8:2:5: no viable alternative at input 'strstr'
.\issue46_imported.p8:3:61: mismatched input '\n' expecting {'[', '+', '-', '~', 'not', '(', 'true', 'false', NAME, DEC_INTEGER, HEX_INTEGER, BIN_INTEGER, '&', '@', FLOAT_NUMBER, STRING, SINGLECHAR}
There are 3 errors in 'issue46_imported'.
Prog8 compiler v7.0-BETA2 by Irmen de Jong ([email protected])
This software is licensed under the GNU GPL 3.0, see https://www.gnu.org/licenses/gpl.html
Compiler target: c64. Parsing...
importing 'issue46_main'
importing 'issue46_imported' (from '.\issue46_imported.p8')
.\issue46_imported.p8:3:11: token recognition error at: '"missing closing quotes (lexical error) -> no bug\n'
<unknown>:2:5: no viable alternative at input 'strstr'
<unknown>:4:0: mismatched input '}' expecting {'[', '+', '-', '~', 'not', '(', 'true', 'false', NAME, DEC_INTEGER, HEX_INTEGER, BIN_INTEGER, '&', '@', FLOAT_NUMBER, STRING, SINGLECHAR}
There are 3 errors in 'issue46_imported'.
compilerAst/src/prog8/parser/Moduleparsing.kt
) wrt to the bug is the first argument to importModule
; before: CharStreams.fromPath(filePath)
(effectively) and after: CharStreams.fromString(content)
. content
is read from filePath
and modified. The problem is that CharStreams
(from antlr) remembers the file name info when called with .fromPath
, but cannot do so when called with .fromString
.filePath
info was still passed to importModule
as additional argument, this then goes on and registers a custom ErrorListener
, which in turn relies on recognizer.inputStream.sourceName
for reporting errors - but only so for non-lexical errors. For lexical errors it uses recognizer.modulePath
, which continued to work.CharStreams.fromPath(filePath)
, but obviously that's only removing the symptom, not the real cause.issue46_main.p8
is reported as being "imported" - minor but awkwardHi Irmen,
Is it possible to allocate a memory block bigger than 256 bytes?
Sure, I can always make a bunch of arrays:
ubyte[256] memory_block0
ubyte[256] memory_block1
ubyte[256] memory_block2
ubyte[256] memory_block3
However, it's not an elegant solution and I'm not sure if the compiler puts them one after another in memory. Does it?
Since merging #45 a new parse error popped up when using comment lines alternated with empty lines:
; what
; what
main {
sub start() {
}
}
examples/test.p8:5:0: mismatched input 'main' expecting <EOF>
see examples/looplabelproblem.p8
due to lack of proper label generation / label naming standardization, the above code generates invalid assembly due to invalid label naming.
cx16.joystick_get2() currently contains a work around for a irq race condition in the cx16 kernal.
This workaround should be removed once the kernal issue is resolved: commanderx16/x16-rom#203
The following code crashes the compiler with null pointer exception:
main {
sub start() {
cx16.set_irq(&irq, false)
}
sub irq() {
}
}
prog8_label* are autogenerated and are useless in the vice mon list.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.