PythonMonkey is a Mozilla SpiderMonkey JavaScript engine embedded into the Python VM, using the Python engine to provide the JS host environment.
This product is in an early stage, approximately 80% to MVP as of July 2023. It is under active development by Distributive Corp., https://distributive.network/. External contributions and feedback are welcome and encouraged.
The goal is to make writing code in either JS or Python a developer preference, with libraries commonly used in either language
available eveywhere, with no significant data exchange or transformation penalties. For example, it should be possible to use NumPy
methods from a JS library, or to refactor a slow "hot loop" written in Python to execute in JS instead, taking advantage of
SpiderMonkey's JIT for near-native speed, rather than writing a C-language module for Python. At Distributive, we intend to use
this package to execute our complex dcp-client
library, which is written in JS and enables distributed computing on the web stack.
- Strings share immutable backing stores whenever possible (when allocating engine choses UCS-2 or Latin-1 internal string representation) to keep memory consumption under control, and to make it possible to move very large strings between JS and Python library code without memory-copy overhead.
- TypedArrays to share mutable backing stores; if this is not possible we will implement a copy-on-write (CoW) solution.
- JS objects are represented by Python dicts
- JS Date objects are represented by Python datetime.datetime objects
- Intrinsics (boolean, number, null, undefined) are passed by value
- JS Functions are automatically wrapped so that they behave like Python functions, and vice-versa
- [done] JS instrinsics coerce to Python intrinsics
- [done] JS strings coerce to Python strings
- [done] JS objects coerce to Python dicts [own-properties only]
- [done] JS functions coerce to Python function wrappers
- [done] JS exceptions propagate to Python
- [done] Implement
eval()
function in Python which accepts JS code and returns JS->Python coerced values - [done] NodeJS+NPM-compatible CommonJS module system
- [done] Python strings coerce to JS strings
- [done] Python intrinsics coerce to JS intrinsics
- [done] Python dicts coerce to JS objects
- [done] Python
require
function, returns a coerced dict of module exports - [done] Python functions coerce to JS function wrappers
- [done] CommonJS module system .py loader, loads Python modules for use by JS
- JS object->Python dict coercion supports inherited-property lookup (via getattribute?)
- [done] Python host environment supplies event loop, including EventEmitter, setTimeout, etc.
- Python host environment supplies XMLHttpRequest (other project?)
- Python host environment supplies basic subsets of NodeJS's fs, path, process, etc, modules; as-needed by dcp-client (other project?)
- [done] Python TypedArrays coerce to JS TypeArrays
- [done] JS TypedArrays coerce to Python TypeArrays
Read this if you want to build a local version.
-
You will need the following installed (which can be done automatically by running
./setup.sh
):- bash
- cmake
- doxygen
- graphviz
- llvm
- rust
- python3.8 or later with header files (python3-dev)
- spidermonkey 102.2.0 or later
- npm (nodejs)
- Poetry
- poetry-dynamic-versioning
-
Run
poetry install
. This command automatically compiles the project and installs the project as well as dependencies into the poetry virtualenv.
If you are using VSCode, you can just press Ctrl + Shift + B to run build task - We have the tasks.json
file configured for you.
- Compile the project
- Install development dependencies:
poetry install --no-root --only=dev
- From the root directory, run
poetry run pytest ./tests/python
For VSCode users, similar to the Build Task, we have a Test Task ready to use.
Install from PyPI
PythonMonkey is not release-ready yet. Our first public release is scheduled for mid-June 2023.
$ pip install pythonmonkey
Install the nightly build
$ pip install -i https://nightly.pythonmonkey.io/ --pre pythonmonkey
pythonmonkey
is available in the poetry virtualenv once you compiled the project using poetry.
$ poetry run python
Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pythonmonkey as pm
>>> hello = pm.eval("() => {return 'Hello from Spidermonkey!'}")
>>> hello()
'Hello from Spidermonkey!'
Alternatively, you can build a wheel
package by running poetry build --format=wheel
, and install it by pip install dist/*.whl
.
- build the project locally
- To use gdb, run
poetry run gdb python
.
See Python Wiki: DebuggingWithGdb
If you are using VSCode, it's more convenient to debug in VSCode's built-in debugger. Simply press F5 on an open Python to start debugging - We have the launch.json
file configured for you.
- examples/
- https://github.com/Distributive-Network/PythonMonkey-examples
- https://github.com/Distributive-Network/PythonMonkey-Crypto-JS-Fullstack-Example
These methods are exported from the pythonmonkey module.
Return the exports of a CommonJS module identified by moduleIdentifier
, using standard CommonJS
semantics
- modules are singletons and will never be loaded or evaluated more than once
- moduleIdentifier is relative to the Python file invoking
require
- moduleIdentifier should not include a file extension
- moduleIdentifiers which do not begin with ./, ../, or / are resolved by search require.path and module.paths.
- Modules are evaluated immediately after loading
- Modules are not loaded until they are required
- The following extensions are supported:
**
.js
- JavaScript module; source code decoratesexports
object **.py
- Python module; source code decoratesexports
dict **.json
-- JSON module; exports are the result of parsing the JSON text in the file
A Python Dict which is equivalent to the globalThis object in JavaScript.
Factory function which returns a new require function
- filename: the pathname of the module that this require function could be used for
- extraPaths: [optional] a list of extra paths to search to resolve non-relative and non-absolute module identifiers
- isMain: [optional] True if the require function is being created for a main module
Load and evaluate a program (main) module. Program modules must be written in JavaScript. Program modules are not necessary unless the main entry point of your program is written in JavaScript.
- filename: the location of the JavaScript source code
- argv: the program's argument vector
- extraPaths: [optional] a list of extra paths to search to resolve non-relative and non-absolute module identifiers
Care should be taken to ensure that only one program module is run per JS context.
console
setTimeout
setInterval
clearTimeout
clearInterval
The CommonJS subsystem is activated by invoking the require
or createRequire
exports of the (Python)
pythonmonkey module.
require
exports
module
python.print
- the Python print functionpython.getenv
- the Python getenv functionpython.stdout
- an object withread
andwrite
methods, which read and write to stdoutpython.stderr
- an object withread
andwrite
methods, which read and write to stderrpython.exec
- the Python exec functionpython.eval
- the Python eval functionpython.exit
- the Python exit function (wrapped to return BigInt in place of number)python.paths
- the Python sys.paths list (currently a copy; will become an Array-like reflection)
When sending variables from Python into JavaScript, PythonMonkey will intelligently coerce or wrap your variables based on their type. PythonMonkey will share backing stores (use the same memory) for ctypes, typed arrays, and strings; moving these types across the language barrier is extremely fast because there is no copying involved.
Note: There are plans in Python 3.12 (PEP 623) to change the internal string representation so that every character in the string uses four bytes of memory. This will break fast string transfers for PythonMonkey, as it relies on the memory layout being the same in Python and JavaScript. As of this writing (July 2023), "classic" Python strings still work in the 3.12 beta releases.
Where shared backing store is not possible, PythonMonkey will automatically emit wrappers that use the "real" data structure as its value authority. Only immutable intrinsics are copied. This means that if you update an object in JavaScript, the corresponding Dict in Python will be updated, etc.
Python Type | JavaScript Type |
---|---|
String | string |
Integer | number |
Bool | boolean |
Function | function |
Dict | object |
List | Array-like object |
datetime | Date object |
awaitable | Promise |
Error | Error object |
Buffer | ArrayBuffer |
JavaScript Type | Python Type |
---|---|
string | String |
number | Float |
bigint | Integer |
boolean | Bool |
function | Function |
object - most | JSObjectProxy which inherits from Dict |
object - Date | datetime |
object - Array | List |
object - Promise | awaitable |
object - ArrayBuffer | Buffer |
object - type arrays | Buffer |
object - Error | Error |
You can force a number in JavaScript to be coerced as an integer by casting it to BigInt.
function myFunction(a, b) {
const result = calculate(a, b);
return BigInt(Math.floor(result));
}
You can use a JavaScript IIFE to create a scope in which you can inject Python symbols:
globalThis.python.exit = pm.eval("""'use strict';
(exit) => function pythonExitWrapper(exitCode) {
if (typeof exitCode === 'number')
exitCode = BigInt(Math.floor(exitCode));
exit(exitCode);
}
""")(sys.exit);
A basic JavaScript shell, pmjs
, ships with PythonMonkey. This shell can also run JavaScript programs with
If you are having trouble with the CommonJS require function, set environment variable DEBUG='ctx-module*' and you can see the filenames it tries to laod.