pyston / pyston_v1 Goto Github PK

Hello!
Thanks for opening this promising python JIT based on LLVM.
Have you plan to make a roadmap for the project?

Like what need to be done? Goals etc...

This is primarily an issue to talk about contributions to this project. I am looking forward to work as a contributor, and I'd like to pick something easy to start with.

I think that implementing default arguments, keywords, _args and *_kwargs may be a good start (Kevin, what do you say? Is this easy enough for someone who is not quite inside the codebase yet?).

So, just to let others know, so that there is no duplicate work: I'm planning on working on this feature. If anyone is already working on this, please let me know!

P.S.: We should really think about setting up an alternative dev discussion channel - perhaps a mailing list? IRC? (#pyston in freenode is always empty) A bug tracker? Kevin, with you being the project owner, what do you say? I understand that the project is still in an early development stage, and that the low number of contributors won't justify doing this. But the thing is, at the moment it is a little hard to tell who is working on what and which features one can pick.

I got this weird bug while developing the __future__ imports.

A commit demonstrating the issue is at https://github.com/tjhance/pyston/tree/weird_segfault

When I run make dbg_map ARGS=-csrq, I get the following backtrace from gdb:

Program received signal SIGSEGV, Segmentation fault.
0x00002aaaac3b5e08 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#0  0x00002aaaac3b5e08 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00002aaaac3b6b89 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#2  0x00000000007b4a60 in __gnu_cxx::new_allocator<long>::deallocate (this=0x1c5aba0 <pyston::Stats::getStatId(std::string const&)::counts>, __p=0x1d83de0) at /home/tjhance/pyston_deps/gcc-4.8.2-install/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../include/c++/4.8.2/ext/new_allocator.h:110
#3  0x00000000007b4a2e in std::_Vector_base<long, std::allocator<long> >::_M_deallocate (this=0x1c5aba0 <pyston::Stats::getStatId(std::string const&)::counts>, __p=0x1d83de0, __n=64) at /home/tjhance/pyston_deps/gcc-4.8.2-install/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../include/c++/4.8.2/bits/stl_vector.h:174
#4  0x00000000007b6259 in std::_Vector_base<long, std::allocator<long> >::~_Vector_base (this=0x1c5aba0 <pyston::Stats::getStatId(std::string const&)::counts>) at /home/tjhance/pyston_deps/gcc-4.8.2-install/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../include/c++/4.8.2/bits/stl_vector.h:160
#5  0x00000000007b15c8 in std::vector<long, std::allocator<long> >::~vector (this=0x1c5aba0 <pyston::Stats::getStatId(std::string const&)::counts>) at /home/tjhance/pyston_deps/gcc-4.8.2-install/lib/gcc/x86_64-unknown-linux-gnu/4.8.2/../../../../include/c++/4.8.2/bits/stl_vector.h:416
#6  0x00002aaaac372901 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#7  0x00002aaaac372985 in exit () from /lib/x86_64-linux-gnu/libc.so.6
#8  0x00002aaaac358774 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#9  0x00000000006535e9 in _start ()

Looks like it's segfaulting when it tries to destruct the local static variable

static std::vector<long> counts;

from core/stats.cpp

I valgrind'ed and gdb'ed but couldn't figure this out - it doesn't seem like the destructor is being called more than once. My best guess is that we're running into some undefined behavior here, but I can't figure out where it is coming from or how it's manifesting.

For my future diff, I resolved the issue by moving around some totally unrelated stuff (namely I moved the definitions of FutureOption and future_options from codegen/irgen/future.h to codegen/irgen/future.cpp. Notably, core/stats.cpp does not include (not even indirectly) future.h!) So this is more evidence that there is some undefined behavior going on.

But I have no idea where it's coming from. The use of of the variable counts, while hacky, looks fine as far as I can tell. I'm just throwing this issue up here in case anybody has any idea what's going on, because I'd like to know, and it might be an issue that shows up again later.

Currently machine functions (ie the product of the LLVM compiler) are named roughly $(PYTHON-FUNCTION-NAME)_$(UNIQUE-ID), and PYTHON-FUNCTION-NAME is simply "module" for the main part of a module.

This works, though there are a couple things that could be done to improve readability:

• Functions in classes maybe should have the class name in them.
• Modules should somehow incorporate the module name in them.

Hello,

Okay so I prepared a lot of string manipulation fuctions and I started preparing str.format.
Generaly I dont know how to prepare:
str.format(_args, *_kwargs)
So tell pyston it could has unlimited args/kwargs. Any idea how to handle it?

Besides that encode/decode is still missing.

Assembly re-writes have pointers to objects sometimes (usually, if not always, to type objects or function objects). These should be tracked by the GC.

Have you plan to support Python3.x versions in the future?
This will be in your eventual roadmap: #11?

From what I remember, Dropbox was famous for reusing open source and not reinventing the wheel. Now people are asking - how is it different from PyPy and why it can not reuse existing PyPy codebase adding more manpower there?

Hi all,

After installing all the mandatory dependencies, I tried to do a "make check" at pyston/src, and got the error when clang++ was trying to compile runtime/stacktrace.cpp and hit some empty struct declaration in libunwind header files. It seems that the front-end clang++ is too strict on this one.

$ make check
...
pyston: Compiling runtime/dict.o
pyston: Compiling runtime/file.o
pyston: Compiling runtime/float.o
pyston: Compiling runtime/importing.o
pyston: Compiling runtime/int.o
pyston: Compiling runtime/list.o
pyston: Compiling runtime/objmodel.o
pyston: Compiling runtime/stacktrace.o
In file included from runtime/stacktrace.cpp:16:
In file included from /home/toanxm/pyston_deps/libunwind-1.1-install/include/libunwind.h:23:
/home/toanxm/pyston_deps/libunwind-1.1-install/include/libunwind-x86_64.h:111:9: error: empty struct has size 0 in C, size 1 in C++ [-Werror,-Wextern-c-compat]
typedef struct unw_tdep_save_loc
^
/home/toanxm/pyston_deps/libunwind-1.1-install/include/libunwind-x86_64.h:120:9: error: empty struct has size 0 in C, size 1 in C++ [-Werror,-Wextern-c-compat]
typedef struct
^
2 errors generated.
make: *** [runtime/stacktrace.o] Error 1


I did a 'make check -n' and found that the command that got the error is exactly this one.

ccache /home/toanxm/pyston_deps/llvm-trunk-build/Release/bin/clang++ -I/home/toanxm/pyston_deps/llvm-trunk/include -I/home/toanxm/pyston_deps/llvm-trunk-build/include -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -O3 -fomit-frame-pointer -fvisibility-inlines-hidden -fno-exceptions -fno-rtti -fPIC -Woverloaded-virtual -ffunction-sections -fdata-sections -Wcast-qual -g -Werror -Wreturn-type -Woverloaded-virtual -Wall -Wno-sign-compare -Wno-unused -I. -I../include -fno-omit-frame-pointer -std=c++11 -I/home/toanxm/pyston_deps/valgrind-3.9.0/include -Wextra -Wno-sign-compare -Wno-unused-parameter -DGITREV=e715b9545772 -DLLVMREV=202092 -DDEFAULT_PYTHON_MAJOR_VERSION=2 -DDEFAULT_PYTHON_MINOR_VERSION=7 -DDEFAULT_PYTHON_MICRO_VERSION=3 -O0 -DBINARY_SUFFIX= -DBINARY_STRIPPED_SUFFIX=_stripped -Wno-mismatched-tags -ferror-limit=10 -Qunused-arguments -fcolor-diagnostics -MMD -MP -MF runtime/stacktrace.d runtime/stacktrace.cpp -c -o runtime/stacktrace.o
In file included from runtime/stacktrace.cpp:16:
In file included from /home/toanxm/pyston_deps/libunwind-1.1-install/include/libunwind.h:23:
/home/toanxm/pyston_deps/libunwind-1.1-install/include/libunwind-x86_64.h:111:9: error: empty struct has size 0 in C, size 1 in C++ [-Werror,-Wextern-c-compat]
typedef struct unw_tdep_save_loc
^
/home/toanxm/pyston_deps/libunwind-1.1-install/include/libunwind-x86_64.h:120:9: error: empty struct has size 0 in C, size 1 in C++ [-Werror,-Wextern-c-compat]
typedef struct
^
2 errors generated.


Some more details about my setup is that I installed libunwind locally at ${HOME}/pyston_deps/libunwind-1.1-install, and exposed the necessary header files and libraries to the compiler and linker by setting the flags as below.

Did anyone encountered a similar issue, and how did you get over that?

def bignum():
  return 2 ** 67

returns 0, since nothing (to my knowledge) in Pyston takes care of overflows. Are there any plans to fix this, if ever? I'm well aware it'd require a non-trivial amount of effort to fix it without major performance implications for the integers which fit into 64 bits, although it can be done.

Have you plan to move to an other build system like Cmake or autotools?
This will be in your roadmap #11 ?

Thanks! And sorry to disturb you but this questions seems important.

After going through docs/INSTALLING, I've notice that the step for libunwind installation is not very clear. It would be nice to improve the document and make that step a little less vague by explicitly describing that the process for libunwind is just the regular configure -> make -> make-install prodecure.

Furthermore, on my computer (running Kubuntu 13.10), I had to manually copy libunwind.so.8 from /usr/local/lib to /lib, otherwise, pyston wouldn't find it.

Hi, I am trying to write a compiler for python using LLVM. However, It make no sense to me how to compile embed function, I mean, How can a embed function visit the variable in a function, we know the local variable is kept in stack. Can you offer me some idea? Thanks a lot.

This is a big project and I don't know exactly what the strategy should be, but I am guessing that the main part of the effort will be to upgrade our simple conservative collector to a more-sophisticated Bartlett collector. I think JavaScriptCore has a production-quality implementation that we may be able to port.

There are probably also ways we can improve/tune our current implementation, but if we're going to replace it, those probably don't make sense to do.

So that contributors aren't discouraged and less headaches for the maintainer?

https://github.com/blog/1184-contributing-guidelines

Specifically it should inform people about the Dropbox CLA.

The following snippet

for x in [-5, 3, 5]:
    for y in [-5, 3, 5]:
        for z in [1, -1, 2, -2]:
            print x, y, z

outputs

3 -5 -2
3 3 1
3 3 -1
3 3 2
3 3 -2
3 5 1
3 5 -1
3 5 2
3 5 -2
5 -5 1
5 -5 -1
5 -5 2
5 -5 -2
5 3 1
5 3 -1
5 3 2
5 3 -2
5 5 1
5 5 -1
5 5 2
5 5 -2

I'm running Pyston off commit 32ccd8a. Sorry for the non-minimality of my example; I will try to improve on this.

Set of stuff we should get in before cutting a v0.2 release:

• set literals
• generator expressions
• descriptors
• I believe @tjhance is working on this
• optparse.py
  • globals()
  • _sre module
• int->long promotion
• decorators

(experimenting to see if we'd rather track this as individual issues within a milestone, or as a single issue)

When we OSR we generate multiple machine-level stack frames, which we interpret as multiple Python-level stack frames. We should filter out any frames that call into an OSR frame.

Example:

for i in xrange(100000):
    if i == 20000:
        1/0

Traceback (most recent call last):
  File "../test/tests/t.py", line 0, in __main___e0_0:
    for i in xrange(100000):
  File "../test/tests/t.py", line 0, in __main___e1_osr4_from___main___e0_0_1:
    for i in xrange(100000):
  File "../test/tests/t.py", line 3, in __main___e3_osr4_from___main___e1_osr4_from___main___e0_0_1_2:
    1/0
ZeroDivisionError: integer division or modulo by zero

Ok, I discovered a bug in Pyston today:

output from cpython:
a = 2.6755888888888888888888888888888888
2.675588888888889

output from pyston:
a = 2.6755888888888888888888888888888888
2.67558888889

Some tests (barely) time out on one of my machines - I guess it's just a slower machine.

Are the timeouts for the purpose of "make sure perf doesn't get bad enough that this test fails" or just "make it so that infinite loops don't prevent all tests from running"?

If the latter, we can just bump the time limits for problematic tests, but if the former, I'm not sure how we can make quality tests that work well on all machines.

https://docs.python.org/2/library/__future__.html
I could send a PR with that if agreed.

These are quite difficult in Python due to the possibility of object resurrection, ie a del method is allowed to store a reference to 'self' somewhere, stopping the deallocation of the object.

I think PyPy has an smart and/or efficient implementation for this: http://doc.pypy.org/en/release-2.0-beta2/discussion/finalizer-order.html

static_assert(sizeof(GCObjectHeader) <= sizeof(void*), "");

This line seems to fail. Looking at GCObjectHeader, the structure seems to have an kindid_t(int), uint16_t & uint8_t. So with structure padding that should be (4 + 4 = 8 bytes). However, sizeof(void*) would be 4.

Anything wrong with my calculation ?

when I execute the command : make llvm -j4 after the make llvm_configure and came across the errors. Is there some clue to the linker error? Thanks

Assertion failed: (cfiStartsArray[i] != cfiStartsArray[i-1]), function parse, file /SourceCache/ld64/ld64-236.4/src/ld/parsers/macho_relocatable_file.cpp, line 1685.
0 0x1088890e7 assert_rtn + 144
1 0x1088ae759 mach_o::relocatable::Parser<x86_64>::parse(mach_o::relocatable::ParserOptions const&) + 3213
2 0x1088939e2 mach_o::relocatable::Parser<x86_64>::parse(unsigned char const, unsigned long long, char const, long, ld::File::Ordinal, mach_o::relocatable::ParserOptions const&) + 374
3 0x1088c0afc archive::File<x86_64>::makeObjectFileForMember(archive::File<x86_64>::Entry const_) const + 752
4 0x1088c064e archive::File<x86_64>::justInTimeforEachAtom(char const_, ld::File::AtomHandler&) const + 122
5 0x1088d5a0b ld::tool::InputFiles::searchLibraries(char const*, bool, bool, bool, ld::File::AtomHandler&) const + 219
6 0x1088dd7c4 ld::tool::Resolver::resolveUndefines() + 160
7 0x1088df92d ld::tool::Resolver::resolve() + 79
8 0x108889c47 main + 679

I am attempting to implement class attribute, that is supporting something like

class AttrClass(object):
     clsAttr=1

I change the function irgenerator.cpp:evalName to first look up the node->id in the currently defining class.

CompilerVariable* attr = curClass->getattr(emitter, getOpInfoForNode(node, exc_info), &node->id, false);

However, this code actually raises the AttributeError if looking up some not existing attribute. I find i need to implement something similar to the following python code, which makes the approach quite complicated. Do you have any recommendations?

try:
    return curClass.attr
except AttributeError:
    return attr

You're using fflush in jit.cpp while commenting that you don't know why.
This leads me to ask two questions:

• How did you end up using it?
• Could it be, that you need it because the prompt (i.e. ">> ") you printf doesn't end with a newline, and you want to make sure it's printed anyway?

Hi, how are you? Nice to meet you! I'm here in the cybercafe at Gold's and I'm reading this blog you have, and then I cloned this project and realized immediately that there is a serious design flaw. You are damaged goods, lady! This project misallocates valuable resources of the Python community. Did you think you could get away with this? Did you poach the Guido from Google just to stick him in a cage and force him to work on a Python 2.7 project for the past year? ARGH! I have to hand it to you, best mind fuck yet.

Sure llvm is what the damn hippies like, but your are either late to the party or ignoring the empty bottles of peppermint schnapps. Pypy, Numba, Blaze. You must learn why they fear the night. You must learn their riddle, you must learn their discipline. Let the Guido go work on something like salt, or shipyard, or sympy.

Also, why don't your prices drop relative to S3? I'm terribly sorry to disturb your party. HA!

Hello.
We're iterating over dict in reverse order.

Lets see an example. This is how CPython does that:

{i: i for i in range(10)}

Output is:

{0: 0, 1: 1, 2: 2, 3: 3, 4: 4, 5: 5, 6: 6, 7: 7, 8: 8, 9: 9}

In Pyston output is:

{9: 9, 8: 8, 7: 7, 6: 6, 5: 5, 4: 4, 3: 3, 2: 2, 1: 1, 0: 0}

:D

ilfiron@gentoo ~/pyston/src $ make check -j4
make lint
make[1]: Вход в каталог /home/ilfiron/pyston/src' pyston: linting... Lint checks passed make[1]: Выход из каталога/home/ilfiron/pyston/src'
make ext pyston_dbg
make[1]: Вход в каталог /home/ilfiron/pyston/src' cc -O2 -fPIC -Wimplicit -I../include -c ../test/test_extension/test.c -o ../test/test_extension/test.o -g pyston: Compiling runtime/inline/xrange.o.bc pyston: Compiling ../tools/publicize.o pyston: Compiling runtime/inline/boxing.o.bc pyston: Compiling runtime/inline/link_forcer.o.bc pyston: Compiling runtime/inline/list.o.bc pyston: Compiling runtime/inline/dict.o.bc pyston: Compiling runtime/inline/tuple.o.bc pyston: Compiling runtime/inline/gc_runtime.o.bc pyston: Compiling codegen/irgen.h.pch pyston: Compiling asm_writing/assembler.o pyston: Compiling asm_writing/rewriter.o pyston: Compiling asm_writing/icinfo.o pyston: Compiling asm_writing/rewriter2.o pyston: Compiling asm_writing/mc_writer.o pyston: Compiling analysis/scoping_analysis.o pyston: Compiling analysis/type_analysis.o pyston: Compiling analysis/function_analysis.o pyston: Compiling core/stats.o pyston: Compiling core/cfg.o pyston: Compiling core/threading.o pyston: Compiling core/ast.o pyston: Compiling core/options.o pyston: Compiling core/util.o pyston: Compiling jit.o pyston: Compiling codegen/profiling/profiling.o pyston: Compiling codegen/profiling/dumprof.o pyston: Compiling runtime/set.o pyston: Compiling runtime/objmodel.o pyston: Compiling runtime/capi.o pyston: Compiling runtime/bool.o pyston: Compiling runtime/types.o pyston: Compiling runtime/str.o pyston: Compiling runtime/list.o pyston: Compiling runtime/dict.o pyston: Compiling runtime/float.o pyston: Compiling runtime/stacktrace.o pyston: Compiling runtime/int.o pyston: Compiling runtime/tuple.o pyston: Compiling runtime/util.o pyston: Compiling runtime/file.o pyston: Compiling runtime/builtin_modules/sys.o pyston: Compiling runtime/builtin_modules/thread.o pyston: Compiling runtime/builtin_modules/time.o pyston: Compiling runtime/builtin_modules/builtins.o pyston: Compiling runtime/builtin_modules/math.o pyston: Compiling gc/heap.o pyston: Compiling gc/root_finder.o pyston: Compiling gc/collector.o cc -shared ../test/test_extension/test.o -o ../test/test_extension/test.so -g pyston: Linking ../tools/publicize which: no gold in (/home/ilfiron/google-cloud-sdk/bin:/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/x86_64-pc-linux-gnu/gcc-bin/4.8.2:/opt/android-ndk:/opt/android-ndk/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-x86_64/bin:/opt/android-ndk/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86_64/bin:/opt/android-ndk/toolchains/llvm-3.3/prebuilt/linux-x86_64/bin:/opt/android-ndk/toolchains/llvm-3.4/prebuilt/linux-x86_64/bin:/opt/android-ndk/toolchains/mipsel-linux-android-4.6/prebuilt/linux-x86_64/bin:/opt/android-ndk/toolchains/mipsel-linux-android-4.8/prebuilt/linux-x86_64/bin:/opt/android-ndk/toolchains/renderscript/prebuilt/linux-x86_64/bin:/opt/android-ndk/toolchains/x86-4.6/prebuilt/linux-x86_64/bin:/opt/android-ndk/toolchains/x86-4.8/prebuilt/linux-x86_64/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/usr/games/bin) gold not available ld: /home/ilfiron/pyston_deps/llvm-trunk-build/Release+Asserts/lib/libLLVMSupport.a(Process.o): undefined reference to symbol 'del_curterm' ld: note: 'del_curterm' is defined in DSO /lib64/libtinfo.so.5 so try adding it to the linker command line /lib64/libtinfo.so.5: could not read symbols: Invalid operation clang: error: linker command failed with exit code 1 (use -v to see invocation) make[1]: *** [../tools/publicize] Ошибка 1 make[1]: *** Ожидание завершения заданий... make[1]: Выход из каталога/home/ilfiron/pyston/src'
make: *** [check] Ошибка 2

What is a "gold" ?

I am not sure whether it happens only on my machine. It seems CPython doesn't invoke the repr on the message of the AttributeError, which is a bit weild. The message of CPython is surrouned by double quote.

CPython:
AttributeError("'int' object has no attribute 'b'",)
Pysthon
AttributeError(''int' object has no attribute 'b'',)

Since we use a custom memory allocator (our GC), Valgrind is unable to track our memory, and also raises a number of false-positive errors. Valgrind offers hooks for annotating your memory allocator, which we should use so that we can use Valgrind again to debug memory issues.

Hello, after commit: 0a7dc33

try:
    var = 'hello'
    var.lower(42)
    print 'TypeError not raised'
except TypeError:
    print 'TypeError raised'

Returns error:

try:
    var = "hello"
    var.lower(42)
    print "TypeError not raised"
except TypeError:
    print "TypeError raised"

==============

Error: doing a non-trivial assignment in an invoke is not allowed:
var = "hello"

Is this desired behaviour?

Hey.

I think you should look at mypy (http://www.mypy-lang.org/) and add support to optional static type annotations. Those can be expressed directly on python3 and with decorators / docstrings on python2.7.

This could allow the compiler to offer aggressive optimizations where a binding can only bind to particular subset of types (various numeric types) and the compiler can report an error (or discard the annotation and issue a warning) when declarations are conflicting actual usage.

Lambda expressions are part of the Python language and are still missing on Pyston. I will be working on this in the next couple of days. This issue is just so that other developers can know that someone is working on this. If you'd like to help, let me know.

Note: I advice interested contributors to sign up on the development mailing list: http://lists.pyston.org/cgi-bin/mailman/listinfo/pyston-dev

I'm not sure how difficult this is; I think we have most of the framework in place, but and we "just" need to 1) look at CPython to see the rules for when it's allowable, 2) calculate the MRO appropriately and use that in ex typeLookup(), and 3) update miscellaneous places that deal with the type hierarchy (PyType_Ready(), isSubclass(), etc).

I tried to build pyston in Ubuntu 12.04 and found that ccache is not optional but necessary to build LLVM.

I think content of docs/INSTALLING should be updated.

It is great to see this project. One are perhaps you can make a greater impact is perhaps having static whole program analysis, effects inference and type inference. This might produce faster code than doing this all dynamically but certain incremental optimisations can be done dynamically. This infrastructure can be a basis for the dynamic analysis and optimisation also, in case you want a dynamic solution.

I'm currently implementing count, full count for strings.

So that's why I prepared method:

Box* strCount(BoxedString* self, Box* elt, Box* startbox, Box* endbox) {
    assert(self->cls == str_cls);

    printf("startbox: %li\n", static_cast<BoxedInt*>(startbox)->n);
    printf("endbox: %li\n", static_cast<BoxedInt*>(endbox)->n);
}

And defined it this way:

    str_cls->giveAttr("count", new BoxedFunction(boxRTFunction((void*)strCount, BOXED_INT, 4, 2, true, false), { boxInt(0), boxInt(0) }));

Then im calling it from REPL:

var = 'aaa'
assert var.count('a') == 3

Sadly, output is always:

startbox: 0
endbox: 140734048053600

For:

var = 'aaa'
assert var.count('a', 10, 1) == 1

Output is:

startbox: 10
endbox: 140734557438864

Seems like for more than 3 arguments, theres a problem.
Any idea what could be wrong?

Tracebacks look at the filename for the corresponding module, but this isn't correct when using execfile(), since that will change the filename but not the module.

Setup a tool that checks how much of the code is covered by tests https://en.wikipedia.org/wiki/Code_coverage

I'm writing here as I am unable to find a mailing list for pyston.
I would like to know how pyston coverts python classes into LLVM IR ?
Does it generate one LLVM struct per python class ?

I tried to find it out myself by installing pyston on my machine,but was not successful .
Can you please answer my question?

codegen/stackmaps.cpp:187:242: error: format specifies type 'long' but the argument has type 'uint64_t' (aka 'unsigned long long') [-Werror,-Wformat]
do { if (!(ptr.i8 - start_ptr == stackmap_size)) { fprintf(stderr, "codegen/stackmaps.cpp" ":" "187" ": %s: Assertion `" "ptr.i8 - start_ptr == stackmap_size" "' failed: " "%ld %ld" "\n", PRETTY_FUNCTION, ptr.i8 - start_ptr, stackmap_size); abort(); } } while (false);

I keep on getting this error in pretty much every file which has a printf associated.

Is this related ??

https://mail.python.org/pipermail/patches/2000-June/000871.html

For "import re", the breakdown is roughly:

• 215ms definedness analysis
• 50ms liveness analysis
• 80ms phi analysis
• 200ms type analysis
• 5ms irgen
• 25ms other compilation stuff
• 40ms non-compilation time

Our new tiering framework will allow us to build an AST interpreter which will let us avoid doing the codegen analyses at all, but there's clearly also room for us to improve the performance of them as well.

One such way to improve them is to replace our usage of std::unordered_map<std::string, Foo> with efficient bitmasks/arrays, since the set of keys could be collected once at the beginning.

Another thing that could be improved is using more efficient data flow algorithms for these problems; right now they use a generic fixed-point data flow analysis algorithm that I'm assuming isn't the most efficient.

The stacks for generators are currently heap-allocated, which means that if there is any stack overflow (quite easy since we don't have any guards against that), we will start overwriting arbitrary heap data.

We should probably mmap separate memory for the stacks and have non-dereferenceable memory on either side.

Currently the installations steps are only available on Ubuntu/Debian. It would be great if the installation steps were also present for Mac.

Not a huge deal, but CPython timed out a few times on my laptop and the -SIGALRM was recorded as the expected exit code, which means a spurious Pyston fail until float_precision.expected_cache is removed and the test re-run. Not sure what would be the right fix for it (maybe not recording, at least, when CPython times out, which makes the problem non-persistent) or whether it's worth fixing at all.

Need to replace libstdc++.so.6 in /usr/lib/x86_64-linux-gnu/ with the one we got during the gcc-4.8.2 build process. Change the softlink to libstdc++.so.6.18