denosaurs / deno_python Goto Github PK
View Code? Open in Web Editor NEW🐍 Python interpreter bindings for Deno and Bun.
Home Page: https://deno.land/x/python
License: MIT License
🐍 Python interpreter bindings for Deno and Bun.
Home Page: https://deno.land/x/python
License: MIT License
Let's add an entry to the python wiki about this!
https://wiki.python.org/moin/IntegratingPythonWithOtherLanguages
This requires this pr #65 to work, otherwise it will segfault
import { python } from "/home/mrcool/dev/deno/others/deno_python/mod.ts";
const inspect = python.import("inspect");
inspect.signature(python.callback(() => {}));
no signature found for builtin <built-in method JSCallback:anonymous of NoneType object at 0x7f218cf558a0>
error: Uncaught (in promise) PythonError: no signature found for builtin <built-in method JSCallback:anonymous of NoneType object at 0x7f218cf558a0> File "/usr/lib64/python3.12/inspect.py", line 3335, in signature
return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/inspect.py", line 3075, in from_callable
return _signature_from_callable(obj, sigcls=cls,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/inspect.py", line 2592, in _signature_from_callable
return _signature_from_builtin(sigcls, obj,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/inspect.py", line 2382, in _signature_from_builtin
raise ValueError("no signature found for builtin {!r}".format(func))
some warnings don't show up when using this library, for example:
a.py
import gi
gi.require_version('Gtk', '4.0')
from gi.repository import Gtk
from gi.repository import Gio
def on_activate(app):
win = Gtk.ApplicationWindow(application=app)
win.get_style_context()
win.present()
app = Gtk.Application()
app.connect('activate', on_activate)
app.run(None)
a.ts
import {
Gtk,
NamedArgument,
python,
} from "https://raw.githubusercontent.com/sigmaSd/deno-gtk-py/0.1.4/mod.ts";
const activ = python.callback((_, app) => {
const win = new Gtk.ApplicationWindow(new NamedArgument("application", app));
win.get_style_context();
win.present();
});
const app = new Gtk.Application();
app.connect("activate", activ);
app.run();
this warning only appear on the python version
DeprecationWarning: Gtk.Widget.get_style_context is deprecated
win.get_style_context()
I'm not really sure where does it come from , maybe the python binary itself is needed for such things ? or its lost in some return argument in ffi ?
I try this test but got 67010 segmentation fault
in MacOS Big Sur.
const { Test } = python.runModule(`
class Test:
def test(self, *args, **kwargs):
return all([len(args) == 3, "name" in kwargs])
`);
const t = new Test();
const d = python.dict({ a: 1, b: 2 });
const v = t.test(1, 2, new NamedArgument("name", "vampire"), d);
assertEquals(v.valueOf(), true);
After investigating the issue. I saw that we have a segfault when we try to new tuple but not fill all items
https://github.com/denosaurs/deno_python/blob/main/src/python.ts#L568
I seem to be getting a segfault when python code tries to callback into or return values to Javascript code.
Seems like it is related to deno_python missing these important calls:
c.PyEval_InitThreads()
or
c.PyGILState_Ensure()
Related discussion:
https://stackoverflow.com/questions/4866701/python-pygilstate-ensure-release-causes-segfault-while-returning-to-c-from-p
My environment is bunjs on Ubuntu Jammy WSL 2.
Also happens sporadically on another machine which is a MacBook Pro M1.
Hi.
I not sure about "support iterating over PyObject" PR but after the playground
PR: https://github.com/denosaurs/deno_python/pull/6/files
As you can see test
method return a list of int but still throw error that list is not iterable.
Environment:
Deno 1.18.2 (release, x86_64-apple-darwin)
v8 9.8.177.6
Typescript 4.5.2
Python 3.9
OS: MacOS Big Sur
Is possible to import deno module in python code ?
Currently this python.callback((_, _)
always gives a type error, becuase it seems the jscallback default to void
which is not a PythonConvertible.
Line 61 in 025e9da
Currently the workaround is to always annotate the type, mostly like this python.callback((_, button: Gtk_.ToggleButton): undefined
If void can be add as a PythonConvertible this would remove this unnecessary typing.
To reproduce
pip install ipykernel
a.ts
#!/usr/bin/env -S deno run --allow-all --allow-ffi --allow-env=DENO_PYTHON_PATH --unstable-ffi
import { NamedArgument, PyObject, python } from "jsr:@denosaurs/python";
const Kernel = python.import("ipykernel.kernelbase").Kernel;
class RKernel extends Kernel {
}
const p = PyObject.from(RKernel);
p.setAttr("implementation", "R");
p.setAttr("implementation_version", "1.0");
p.setAttr("language", "rust");
p.setAttr("language_version", "1");
p.setAttr("language_info", {
name: "R",
mimetype: "text/x-rust",
file_extension: ".rs",
});
p.setAttr("banner", "R");
p.setAttr("kernel_name", "irust");
// Commenting this out, fixes the segfault
p.setAttr(
"do_execute",`
python.callback(() => {
}),
);
if (import.meta.main) {
const IPKernelApp = python.import("ipykernel.kernelapp").IPKernelApp;
IPKernelApp.launch_instance(new NamedArgument("kernel_class", p));
}
chmod +x && ./a.ts
I followed with gdb
#0 0x00007ffff7e1dfb0 in __strncmp_sse42 () from /lib64/libc.so.6
#1 0x00007ffff4bf79f1 in find_signature () from /lib64/libpython3.12.so
#2 0x00007ffff4c79c70 in _PyType_GetTextSignatureFromInternalDoc ()
from /lib64/libpython3.12.so
#3 0x00007ffff4b8fedd in _PyObject_GenericGetAttrWithDict ()
from /lib64/libpython3.12.so
#4 0x00007ffff4b89be5 in _PyObject_LookupAttr ()
from /lib64/libpython3.12.so
#5 0x00007ffff4b9215f in builtin_getattr () from /lib64/libpython3.12.so
#6 0x00007ffff4b91ba2 in cfunction_vectorcall_FASTCALL ()
from /lib64/libpython3.12.so
#7 0x00007ffff4b88b7c in PyObject_Vectorcall ()
from /lib64/libpython3.12.so
#8 0x00007ffff4b71ea5 in _PyEval_EvalFrameDefault ()
from /lib64/libpython3.12.so
#9 0x00007ffff4b6bcbb in _PyObject_FastCallDictTstate ()
from /lib64/libpython3.12.so
#10 0x00007ffff4b9ad04 in slot_tp_init () from /lib64/libpython3.12.so
#11 0x00007ffff4b696cb in type_call () from /lib64/libpython3.12.so
#12 0x00007ffff4b9e049 in _PyObject_Call () from /lib64/libpython3.12.so
#13 0x00007ffff4b76e45 in _PyEval_EvalFrameDefault ()
from /lib64/libpython3.12.so
#14 0x00007ffff4bb0738 in method_vectorcall ()
from /lib64/libpython3.12.so
#15 0x00007ffff4b9e145 in _PyVectorcall_Call ()
from /lib64/libpython3.12.so
#16 0x00005555580bf052 in ffi_call_unix64 ()
#17 0x000055555adfbb27 in ffi_call_int.llvm ()
#18 0x000055555adfb89f in ffi_call ()
#19 0x000055555adce43f in <extern "C" fn(A0) .> R as v8::support::CFnFrom<F>>::mapping::c_fn ()
#20 0x000055555952d6df in Builtins_CallApiCallbackGeneric ()
If I check what its comparing, its comparing the function name to the function docs, but the function docs are an invalid pointer
(gdb) info registers
rax 0x15 21
rbx 0x55555c661e40 93825110777408
rcx 0x0 0
rdx 0x14 20
rsi 0x55555c661e40 93825110777408
rdi 0x5555 21845
rbp 0x7ffffffeabd0 0x7ffffffeabd0
rsp 0x7ffffffeaba8 0x7ffffffeaba8
r8 0xfcfe 64766
r9 0x7ffff72481e0 140737339752928
r10 0xd 13
r11 0x14 20
r12 0x5555 21845
r13 0x14 20
r14 0x0 0
r15 0x7ffff4f55f80 140737303109504
rip 0x7ffff7e1dfb0 0x7ffff7e1dfb0 <__strncmp_sse42+48>
eflags 0x10283 [ CF SF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
fs_base 0x7ffff7c8c480 140737350517888
gs_base 0x0 0
(gdb) x/s 0x55555c661e40
0x55555c661e40: "JSCallback:anonymous"
rdi register have an invalid pointer
The 2 (name and doc) should come from the same buffer which is this one https://github.com/denosaurs/deno_python/blob/main/src/python.ts#L479 docs: https://docs.python.org/3/c-api/structures.html#c.PyMethodDef
I also inspected that buffer when the segfault happened, and it seemed correct
reproduction examples
> python.builtins.eval("'中文'#foo", {})
中文
> python.builtins.eval("'中文'", {})
Uncaught PythonError: ('EOL while scanning string literal', ('<string>', 1, 3, "'中文"))
> python.builtins.str("'中文'#foo")
'中文'
python.builtins.str("'中文'")
'中
> python.builtins.str("中文")
Uncaught PythonError: <class 'str'> returned a result with an error set
Haven’t found this in the examples.
If I wanted to pass linewidth=2
to the plot
function of matplotlib, how can I do this from Deno code?
Thanks for creating deno_python. I tried to run NumPy example but faced this issue.
Could you help me to resolve this issue?
error: Uncaught (in promise) PythonError:
IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.
We have compiled some common reasons and troubleshooting tips at:
https://numpy.org/devdocs/user/troubleshooting-importerror.html
Please note and check the following:
* The Python version is: Python3.8 from "/usr/local/bin/deno"
* The NumPy version is: "1.22.1"
and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.
Original error was: dlopen(/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so, 2): Symbol not found: _PyBaseObject_Type
Referenced from: /usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so
Expected in: flat namespace
in /usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so
Traceback:
[' File "/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/__init__.py", line 144, in <module>\n from . import core\n', ' File "/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/__init__.py", line 49, in <module>\n raise ImportError(msg)\n']
throw new PythonError(errorMessage);
^
at maybeThrowError (https://deno.land/x/[email protected]/src/python.ts:641:9)
at Python.importObject (https://deno.land/x/[email protected]/src/python.ts:717:7)
at Python.import (https://deno.land/x/[email protected]/src/python.ts:727:17)
at file:///Users/hoangpq/WebstormProjects/deno-working/server.ts:3:25
But working well on Python REPL
Environment:
OS: macOS
Python version: 3.8 (Homebrew)
Deno version: 1.18.0
I tried to use deno_python with spacy (https://spacy.io/)
import { python } from "https://deno.land/x/[email protected]/mod.ts";
const spacy = python.import("spacy");
console.log(spacy);
But got a segmentation fault
48182 segmentation fault deno run -A --unstable spacy.ts
Working well with
python test.py
Environment:
deno 1.26.0 (release, x86_64-apple-darwin)
v8 10.7.193.3
typescript 4.8.3
Python 3.10.6
Do you have any idea about this issue @DjDeveloperr?
import { python } from "https://deno.land/x/[email protected]/mod.ts";
const gi = python.import("gi");
gi.require_version("Gtk", "4.0");
const Gtk = python.import("gi.repository.Gtk");
class MainWindow extends Gtk.ApplicationWindow {
constructor() {
super();
this.a = 4; // error here
}
}
console.log(new MainWindow());
error: Uncaught TypeError: 'set' on proxy: trap returned falsish for property 'a'
this.a = 4;
^
at new MainWindow (file:///home/mrcool/dev/deno/lab/b.ts:10:11)
at file:///home/mrcool/dev/deno/lab/b.ts:14:13
more info here #53 (comment)
It's not clear in doc (README.md) whether "DENO_PYTHON_PATH" (in a Windows 10 context) should point to the python DIRECTORY, or to the python.exe FILE, or to anything else.
In my case pointing to python.dll solved the problem. (More precisely "C:\Python310\python310.dll").
It would be also worth to mention in docs that the flag "--unstable" IS ABSOLUTELY NECESSARY.
Thank you for this amazing piece of code you are working on!
The bunpy
npm package is still on the v0.3.3 release; it would be good if it updates continued to be published to npm as releases are made (particularly as bunpy has stopped working for me since updating python).
A recent issue I found while attempting to use this module is that it lacks pyenv support instead resolving to the default python installation on mac. This makes it so that imports can't properly resolve among other issues.
import { python } from "jsr:@denosaurs/python";
const m = python.runModule(`
class A:
a = 4
`);
class B extends m.A {
a = 5;
}
const b = new B();
console.log(b.a); // results 4 instead of 5
There is a workaround by setting it after super,
import { python } from "jsr:@denosaurs/python";
const m = python.runModule(`
class A:
a = 4
`);
class B extends m.A {
constructor() {
super();
this.a = 5;
}
}
const b = new B();
console.log(b.a); // the expected 5
But this not workaround is not always available
For example I want to make a jupyter kernel using deno + deno_python, (for example the echo kernel here https://jupyter-client.readthedocs.io/en/stable/wrapperkernels.html)
but it seems like jupyter inspects the class without instantiating it, so it never reaches the super call, so in this case I want a way to override class members directly
There's currently no way to use slices (:
) or ellipsis (...
).
Would something similar to NamedArgument
be able to solve this?
import { python, Slice } from "https://deno.land/x/[email protected]/mod.ts";
const np = python.import("numpy");
const slice = np.array([1, 2, 3]).__getitem__(new Slice(1,3,null)); // equivalent to [1:3]
console.log(slice); // returns [2, 3]
import { NamedArgument, python } from "https://deno.land/x/[email protected]/mod.ts";
const gi = python.import("gi");
gi.require_version("Gtk", "4.0");
const Gtk = python.import("gi.repository.Gtk");
function onActivate(app: any) {
const win = Gtk.ApplicationWindow(new NamedArgument("application", app));
win.present();
}
const app = Gtk.Application();
app.connect("activate", onActivate); // error here
app.run();
error: Uncaught TypeError: Unsupported type: function
throw new TypeError(`Unsupported type: ${typeof v}`);
^
at Function.from (https://deno.land/x/[email protected]/src/python.ts:533:15)
at PyObject.call (https://deno.land/x/[email protected]/src/python.ts:760:57)
at Proxy.object (https://deno.land/x/[email protected]/src/python.ts:258:20)
at file:///home/mrcool/dev/deno/lab/a.ts:13:5
This is the reproduction I have, you need to keep moving the slider fast until it throws the above error or sefaults
import {
Adw,
Gtk,
Gtk_,
kw,
NamedArgument,
python,
} from "https://raw.githubusercontent.com/sigmaSd/deno-gtk-py/13f3da6c4890d62e09312747c905fa85263f5ca8/mod.ts";
class MainWindow extends Gtk.ApplicationWindow {
#slider: Gtk_.Scale;
constructor(kwArg: NamedArgument) {
super(kwArg);
this.set_default_size(600, 250);
this.#slider = Gtk.Scale();
this.#slider.set_range(0, 10);
this.#slider.connect("value-changed", this.slider_changed);
this.set_child(this.#slider);
}
slider_changed = python.callback(
(_kwargs, slider: Gtk_.Scale): undefined => {
slider.get_value(); // this line trigger the error/segfault
},
);
}
class App extends Adw.Application {
#win: MainWindow | undefined;
constructor(kwArg: NamedArgument) {
super(kwArg);
this.connect("activate", this.onActivate);
}
onActivate = python.callback((_kwarg, app: Gtk_.Application): undefined => {
this.#win = new MainWindow(new NamedArgument("application", app));
this.#win.present();
});
}
const app = new App(kw`application_id=${"com.example.com"}`);
app.run(Deno.args);
import { pip } from 'https://deno.land/x/[email protected]/ext/pip.ts';
await pip.import('numpy');
Second and consecutive runs of this code give me the following warnings:
WARNING: Target directory /Users/admin/Library/Caches/deno/plug/pip/numpy already exists. Specify --upgrade to force replacement.
WARNING: Target directory /Users/admin/Library/Caches/deno/plug/pip/numpy-1.26.0.dist-info already exists. Specify --upgrade to force replacement.
WARNING: Target directory /Users/admin/Library/Caches/deno/plug/pip/bin already exists. Specify --upgrade to force replacement.
My version of deno is 1.37.1
Not sure about other platforms, but findLib
is pretty slow on Windows. It takes up 2s at startup on my machine. I think we should not use a command (where
) to look for Python, instead try to load all supported Python versions in order and return the suitable library.
import { python } from "https://deno.land/x/[email protected]/mod.ts";
const m = python.runModule(`
b = True
`);
console.log(typeof m.b, m.b == true, /*works*/ m.b === true /*doesn' work*/);
function true false
I expected the type of b
to be bool
but any variables from python always have type function
In contrast to most JavaScript code, Python libraries often work synchronously and may even expect one's callback function to resolve synchronously, even if that doesn't make sense in JavaScript environment. One example might be the audible
Python package that performs all HTTP requests synchronously and expects a synchronous callback to show an image to the user and get their response.
I have no experience with Deno's FFI yet, but it seems to me that calling synchronous Python functions asynchronously should be possible, as the Python interpreter runs in a different thread. Similarly, I would expect "converting" asynchronous JS functions to synchronous Python callbacks should be possible, if one blocks the Python interpreter until the JS promise is settled.
Is this something that sounds viable to you? And if so, is this something you'd be interested in adding to this library?
Hi @eliassjogreen @DjDeveloperr
I'm trying to access TS/JS function by writing FFI in Rust and getting JS/TS function in v8 Context.
Unfortunately, rusty_v8 does not expose Isolate::Get_Current() method
denoland/rusty_v8#469.
Is there any way to call TS/JS function in Python land?
/Users/elias/.pyenv/versions/3.11.2/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
Every time my script shuts down i get a warning about a leaked semaphore. I am guessing we are forgetting to free some python memory somewhere somehow.
Trying to run this for the first time.
Code
import { pip } from "https://deno.land/x/python/ext/pip.ts";
const np = await pip.import("numpy");
const plt = await pip.import("matplotlib", "matplotlib.pyplot");
const xpoints = np.array([1, 8]);
const ypoints = np.array([3, 10]);
plt.plot(xpoints, ypoints);
plt.show();
Terminal
PS C:\Users\MyUser\Downloads> $env:DENO_PYTHON_PATH='C:\Users\MyUser\AppData\Local\Programs\Python\Python310\python310.dll'; deno run -A --unstable --allow-net=deno.com .\deno1.js
Installing numpy
WARNING: Target directory C:\Users\MyUser\AppData\Local\deno\plug\pip\numpy already exists. Specify --upgrade to force replacement.
WARNING: Target directory C:\Users\MyUser\AppData\Local\deno\plug\pip\numpy-1.25.2.dist-info already exists. Specify --upgrade to force replacement.
WARNING: Target directory C:\Users\MyUser\AppData\Local\deno\plug\pip\bin already exists. Specify --upgrade to force replacement.
[notice] A new release of pip available: 22.3.1 -> 23.2.1
[notice] To update, run: deno.exe -m pip install --upgrade pip
error: Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'packages_distributions')
const packages = importlib.metadata.packages_distributions();
^
at Pip.import (https://deno.land/x/[email protected]/ext/pip.ts:130:41)
at eventLoopTick (ext:core/01_core.js:183:11)
at async file:///C:/Users/MyUser/Downloads/deno1.js:3:12
PS C:\Users\MyUser\Downloads> deno --version
deno 1.36.1 (release, x86_64-pc-windows-msvc)
v8 11.6.189.12
typescript 5.1.6
PS C:\Users\MyUser\Downloads>
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.