smontanaro / cpython Goto Github PK

Instruction implementation in both the translator and ceval.c is incomplete. The PyVM instruction set currently contains 119 instructions 3.9.0a5+). It might make sense for this to be split into smaller chunks and implemented by multiple people in parallel.

The current implementation is a translator of existing wordcode, essentially a complex peephole optimizer. At some point, the AST-based compiler will need to be retargeted.

This function

def while0():
    while True:
        break

compiles to

 43           0 JUMP_ABSOLUTE               4
              2 JUMP_ABSOLUTE               4
        >>    4 LOAD_CONST                  0 (None)
              6 RETURN_VALUE

It's not clear where the extra JUMP_ABSOLUTE instruction came from. This elicits a warning from my translator that the stack level was set to 0 twice for the same block.

Consider this function:

def load_store():
    a = 7
    b = a
    return a + b

It naively translates to:

 0 EXTENDED_ARG                    2
 2 LOAD_CONST_REG                513 (%r2 <- 7)
 4 EXTENDED_ARG                    0
 6 STORE_FAST_REG                  2 (%r0 <- %r2)
 8 EXTENDED_ARG                    2
10 LOAD_FAST_REG                 512 (%r2 <- %r0)
12 EXTENDED_ARG                    1
14 STORE_FAST_REG                258 (%r1 <- %r2)
16 EXTENDED_ARG                    2
18 LOAD_FAST_REG                 512 (%r2 <- %r0)
20 EXTENDED_ARG                    3
22 LOAD_FAST_REG                 769 (%r3 <- %r1)
24 EXTENDED_ARG                    2
26 EXTENDED_ARG                  514
28 BINARY_ADD_REG             131587 (%r2 <- %r2 + %r3)
30 RETURN_VALUE_REG                2 (%r2)

but after backward propagating fast store results, the result is:

 0 EXTENDED_ARG                    0
 2 LOAD_CONST_REG                  1 (%r0 <- 7)
 4 EXTENDED_ARG                    2
 6 EXTENDED_ARG                  512
 8 BINARY_ADD_REG             131073 (%r2 <- %r0 + %r1)
10 RETURN_VALUE_REG                2 (%r2)

Note that propagation of loads and stores has elided %r1 completely. Consequently, the add operation segfaults. Somewhere along the way, either the %r1 reference needs to be changed to %r0 (correct at least in this simple example), or the STORE_FAST_REG instruction at 14 needs to be preserved.

This seems like it should be easy, but I've yet to figure out how to trigger use of LOAD_GLOBAL's opcache code. Accordingly, that remains untested in LOAD_GLOBAL_REG.

In Objects/genobject.c, it distinguishes between a simple yield and exhaustion of the generator by testing f->f_stacktop:

if (result && f->f_stacktop == NULL) {
... exhausted ...
}
else ... {
... normal yield ...
}

Need a way to signal the distinction without relying on f->f_stacktop.

The current translator works with functions (and presumably unbound methods), but doesn't translate code at the module or class level.

Consider this function-in-a-function:

def outer(a):
    loc = a
    def inner():
        return loc
    return inner()

The function inner shows up as a constant in outer:

>>> dis.dis(outer.__code__.co_consts[1])
  4           0 LOAD_DEREF                      0 (loc)
              2 RETURN_VALUE

When testing LOAD_DEREF we won't generally have direct access to inner as a function. Instead, we need to translate the constant which represents it and replace the consts array in outer.

A long lost observation by Tim Peters that the existing stack space could be used for registers is employed to avoid increasing space taken by the frame object. The downside is that there are a limited number of registers available to the translator. The current translator does no proper register allocation. Victor Stinner's implementation did, however. That might be a good place to look for inspiration.

In the block level representation of the code, Instructions don't require EXTENDED_ARG opcodes. Calculating block and instruction length at that level doesn't take them into account. Inserting them into the instruction stream in the bytes method tosses a wrench into things. There are probably a couple ways to fix this. Just need to figure out which is best and implement.

I noticed that I forgot to use the inplace_convert and InplaceOpInstruction when converting in-place instructions, instead using binary_convert to convert them. That worked, but seemed wrong. Alas, I couldn't easily get the "right" code to work, so left things as-is.

I'm not entirely sure, but I think it has something to do with recent changes to the line number table.

Not sure if this is an issue with the current CPython implementation or with my converter. Consider this simple function:

def while1():
    while True:
        pass

It's clearly an infloop, but the code object still seems weird. It disassembles to this:

  3     >>    0 JUMP_ABSOLUTE               0
              2 LOAD_CONST                  0 (None)
              4 RETURN_VALUE

Note that it does have a LOAD_CONST instruction, so it seems it should have a stacksize > 0, even if it's unreachable.

>>> co = while1.__code__
>>> co.co_nlocals
0
>>> co.co_stacksize
0

The RVM translator crashes because it blindly attempts to generate code for LOAD_CONST_REG, which causes it to think it will run off the end of the allocated stack.