Compiled variants of the Python standard library.

Pure Python modules in the standard library can be a speed bottleneck sometimes, this package aims to provide "compiled" variants of the pure Python standard library modules, which are somewhere between 2-4x faster than the builtin ones.

pip install compiled

✨ This will install the pycompile CLI script as well.

Say your program asd.py looks like this:

import tomllib
from difflib import get_close_matches

# ... rest of the code

You can use the bundled pycompile script to turn those imports into the "compiled" variants:

$ pycompile asd.py
✨ Rewrote asd.py with compiled imports.

$ cat asd.py
import compiled.tomllib as tomllib
from compiled.difflib import get_close_matches

# ... rest of the code

With a real world program using tomllib and difflib, we get the following difference in speed:

It uses this awesome piece of software called mypyc which takes type-checked Python code and compiles it to C code, which is generally faster.

Simply making sure that the Python module is statically type checked is 95% of the work needed to be done to make the project work. The rest is just glue code to test the compiled code and package it.

PRs type checking other Python modules are welcome! Check the How to add a library section on the exact steps.

• Create and activate a virtual environment.

• Run pip install mypy, as mypy[c] is the only dependency.

• Scripts to test, build and package standard libraries are present in build.py:

  $ ./build.py test tomllib
  .............
  ----------------------------------------------------------------------
  Ran 13 tests in 0.006s
  OK
  
  $ ./build.py mypy tomllib
  Success: no issues found in 4 source files
  
  $ ./build.py package
  [...]
  ✨Built ./build/dist/compiled-0.2.1-cp311-cp311-macosx_13_0_arm64.whl

• Run pytest compiled_tests to run tests.

Say you wanted to add the heapq library to the project. The steps will be as follows:

• verify ./build.py test heapq passes. This will run the CPython test suite on the existing code.

• Tweak the code and add annotations until ./build.py mypy heapq passes.

  Verify ./build.py test heapq still passes.

• Tweak the code until ./build.py mypyc heapq passes.

  Verify ./build.py test heapq still passes.

• Run ./build.py test_compiled heapq to ensure the compiled version still passes all tests.