denosaurs / deno_python Goto Github PK

View Code? Open in Web Editor NEW

542.0 7.0 17.0 89 KB

🐍 Python interpreter bindings for Deno and Bun.

Home Page: https://deno.land/x/python

License: MIT License

TypeScript 94.77% JavaScript 5.23%

deno python ffi hacktoberfest deno-ffi cpython python-c-api c-api javascript typescript

deno_python's People

Contributors

Stargazers

Watchers

Forkers

hoangpq ahgilak sno2 tjosepo cccdeno wuchirat hansonkd load1n9 lino-levan veryvanya perongh gmarko yuelight 3c1u sigmasd weweweok nberlette

deno_python's Issues

tkinter can be used with deno_python ?

Be explicit in doc about what exactly DENO_PYTHON_PATH should point to, and the need of --unstable flag

It's not clear in doc (README.md) whether "DENO_PYTHON_PATH" (in a Windows 10 context) should point to the python DIRECTORY, or to the python.exe FILE, or to anything else.

In my case pointing to python.dll solved the problem. (More precisely "C:\Python310\python310.dll").

It would be also worth to mention in docs that the flag "--unstable" IS ABSOLUTELY NECESSARY.

Thank you for this amazing piece of code you are working on!

pyenv support

A recent issue I found while attempting to use this module is that it lacks pyenv support instead resolving to the default python installation on mac. This makes it so that imports can't properly resolve among other issues.

Python wiki entry

Let's add an entry to the python wiki about this!

https://wiki.python.org/moin/IntegratingPythonWithOtherLanguages

errors when setting properties on a class that extends python class

import { python } from "https://deno.land/x/[email protected]/mod.ts";

const gi = python.import("gi");
gi.require_version("Gtk", "4.0");
const Gtk = python.import("gi.repository.Gtk");

class MainWindow extends Gtk.ApplicationWindow {
  constructor() {
    super();
    this.a = 4; // error here
  }
}
console.log(new MainWindow());

error: Uncaught TypeError: 'set' on proxy: trap returned falsish for property 'a'
    this.a = 4;
          ^
    at new MainWindow (file:///home/mrcool/dev/deno/lab/b.ts:10:11)
    at file:///home/mrcool/dev/deno/lab/b.ts:14:13

more info here #53 (comment)

Leaking semaphore

/Users/elias/.pyenv/versions/3.11.2/lib/python3.11/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown

Every time my script shuts down i get a warning about a leaked semaphore. I am guessing we are forgetting to free some python memory somewhere somehow.

Warnings when running pip.import more than once

import { pip } from 'https://deno.land/x/[email protected]/ext/pip.ts';
await pip.import('numpy');

Second and consecutive runs of this code give me the following warnings:

WARNING: Target directory /Users/admin/Library/Caches/deno/plug/pip/numpy already exists. Specify --upgrade to force replacement.
WARNING: Target directory /Users/admin/Library/Caches/deno/plug/pip/numpy-1.26.0.dist-info already exists. Specify --upgrade to force replacement.
WARNING: Target directory /Users/admin/Library/Caches/deno/plug/pip/bin already exists. Specify --upgrade to force replacement.

My version of deno is 1.37.1

Synchronous Python & async JavaScript?

In contrast to most JavaScript code, Python libraries often work synchronously and may even expect one's callback function to resolve synchronously, even if that doesn't make sense in JavaScript environment. One example might be the audible Python package that performs all HTTP requests synchronously and expects a synchronous callback to show an image to the user and get their response.

I have no experience with Deno's FFI yet, but it seems to me that calling synchronous Python functions asynchronously should be possible, as the Python interpreter runs in a different thread. Similarly, I would expect "converting" asynchronous JS functions to synchronous Python callbacks should be possible, if one blocks the Python interpreter until the JS promise is settled.

Is this something that sounds viable to you? And if so, is this something you'd be interested in adding to this library?

Jupyter rich content output

https://docs.deno.com/runtime/manual/tools/jupyter#rich-content-output

Working with spacy segmentation fault

I tried to use deno_python with spacy (https://spacy.io/)

import { python } from "https://deno.land/x/[email protected]/mod.ts";

const spacy = python.import("spacy");
console.log(spacy);

But got a segmentation fault

 48182 segmentation fault  deno run -A --unstable spacy.ts

Working well with

 python test.py

Environment:
deno 1.26.0 (release, x86_64-apple-darwin)
v8 10.7.193.3
typescript 4.8.3
Python 3.10.6

Do you have any idea about this issue @DjDeveloperr?

`findLib` is slow on Windows

Not sure about other platforms, but findLib is pretty slow on Windows. It takes up 2s at startup on my machine. I think we should not use a command (where) to look for Python, instead try to load all supported Python versions in order and return the suitable library.

error: Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'packages_distributions')

Trying to run this for the first time.

Code

import { pip } from "https://deno.land/x/python/ext/pip.ts";

const np = await pip.import("numpy");
const plt = await pip.import("matplotlib", "matplotlib.pyplot");

const xpoints = np.array([1, 8]);
const ypoints = np.array([3, 10]);

plt.plot(xpoints, ypoints);
plt.show();

Terminal

PS C:\Users\MyUser\Downloads> $env:DENO_PYTHON_PATH='C:\Users\MyUser\AppData\Local\Programs\Python\Python310\python310.dll'; deno run  -A --unstable --allow-net=deno.com .\deno1.js
Installing numpy
WARNING: Target directory C:\Users\MyUser\AppData\Local\deno\plug\pip\numpy already exists. Specify --upgrade to force replacement.
WARNING: Target directory C:\Users\MyUser\AppData\Local\deno\plug\pip\numpy-1.25.2.dist-info already exists. Specify --upgrade to force replacement.
WARNING: Target directory C:\Users\MyUser\AppData\Local\deno\plug\pip\bin already exists. Specify --upgrade to force replacement.

[notice] A new release of pip available: 22.3.1 -> 23.2.1
[notice] To update, run: deno.exe -m pip install --upgrade pip
error: Uncaught (in promise) TypeError: Cannot read properties of undefined (reading 'packages_distributions')
    const packages = importlib.metadata.packages_distributions();
                                        ^
    at Pip.import (https://deno.land/x/[email protected]/ext/pip.ts:130:41)
    at eventLoopTick (ext:core/01_core.js:183:11)
    at async file:///C:/Users/MyUser/Downloads/deno1.js:3:12
PS C:\Users\MyUser\Downloads> deno --version      
deno 1.36.1 (release, x86_64-pc-windows-msvc)
v8 11.6.189.12
typescript 5.1.6
PS C:\Users\MyUser\Downloads>

How to pass kwargs

Haven’t found this in the examples.

If I wanted to pass linewidth=2 to the plot function of matplotlib, how can I do this from Deno code?

buffer support

https://docs.python.org/3/c-api/buffer.html

Support iterating over PyObject not working

Hi.

I not sure about "support iterating over PyObject" PR but after the playground

PR: https://github.com/denosaurs/deno_python/pull/6/files

As you can see test method return a list of int but still throw error that list is not iterable.

Environment:

Deno 1.18.2 (release, x86_64-apple-darwin)
v8 9.8.177.6
Typescript 4.5.2
Python 3.9
OS: MacOS Big Sur

Segfault

I seem to be getting a segfault when python code tries to callback into or return values to Javascript code.

Seems like it is related to deno_python missing these important calls:

c.PyEval_InitThreads()
or
c.PyGILState_Ensure()

My environment is bunjs on Ubuntu Jammy WSL 2.
Also happens sporadically on another machine which is a MacBook Pro M1.

[Question] Is there any way to call TS/JS function in Python land?

Hi @eliassjogreen @DjDeveloperr

I'm trying to access TS/JS function by writing FFI in Rust and getting JS/TS function in v8 Context.

Unfortunately, rusty_v8 does not expose Isolate::Get_Current() method
denoland/rusty_v8#469.

Is there any way to call TS/JS function in Python land?

python_dono ?!

Is possible to import deno module in python code ?

segfault when overriding a class method in jupyter kernel

To reproduce

pip install ipykernel

a.ts

#!/usr/bin/env -S deno run --allow-all --allow-ffi --allow-env=DENO_PYTHON_PATH --unstable-ffi
import { NamedArgument, PyObject, python } from "jsr:@denosaurs/python";

const Kernel = python.import("ipykernel.kernelbase").Kernel;

class RKernel extends Kernel {
}

const p = PyObject.from(RKernel);
p.setAttr("implementation", "R");
p.setAttr("implementation_version", "1.0");
p.setAttr("language", "rust");
p.setAttr("language_version", "1");
p.setAttr("language_info", {
  name: "R",
  mimetype: "text/x-rust",
  file_extension: ".rs",
});
p.setAttr("banner", "R");
p.setAttr("kernel_name", "irust");

// Commenting this out, fixes the segfault
p.setAttr(
  "do_execute",`
  python.callback(() => {
  }),
);

if (import.meta.main) {
  const IPKernelApp = python.import("ipykernel.kernelapp").IPKernelApp;
  IPKernelApp.launch_instance(new NamedArgument("kernel_class", p));
}

chmod +x && ./a.ts

I followed with gdb

#0  0x00007ffff7e1dfb0 in __strncmp_sse42 () from /lib64/libc.so.6
#1  0x00007ffff4bf79f1 in find_signature () from /lib64/libpython3.12.so
#2  0x00007ffff4c79c70 in _PyType_GetTextSignatureFromInternalDoc ()
   from /lib64/libpython3.12.so
#3  0x00007ffff4b8fedd in _PyObject_GenericGetAttrWithDict ()
   from /lib64/libpython3.12.so
#4  0x00007ffff4b89be5 in _PyObject_LookupAttr ()
   from /lib64/libpython3.12.so
#5  0x00007ffff4b9215f in builtin_getattr () from /lib64/libpython3.12.so
#6  0x00007ffff4b91ba2 in cfunction_vectorcall_FASTCALL ()
   from /lib64/libpython3.12.so
#7  0x00007ffff4b88b7c in PyObject_Vectorcall ()
   from /lib64/libpython3.12.so
#8  0x00007ffff4b71ea5 in _PyEval_EvalFrameDefault ()
   from /lib64/libpython3.12.so
#9  0x00007ffff4b6bcbb in _PyObject_FastCallDictTstate ()
   from /lib64/libpython3.12.so
#10 0x00007ffff4b9ad04 in slot_tp_init () from /lib64/libpython3.12.so
#11 0x00007ffff4b696cb in type_call () from /lib64/libpython3.12.so
#12 0x00007ffff4b9e049 in _PyObject_Call () from /lib64/libpython3.12.so
#13 0x00007ffff4b76e45 in _PyEval_EvalFrameDefault ()
   from /lib64/libpython3.12.so
#14 0x00007ffff4bb0738 in method_vectorcall ()
   from /lib64/libpython3.12.so
#15 0x00007ffff4b9e145 in _PyVectorcall_Call ()
   from /lib64/libpython3.12.so
#16 0x00005555580bf052 in ffi_call_unix64 ()
#17 0x000055555adfbb27 in ffi_call_int.llvm ()
#18 0x000055555adfb89f in ffi_call ()
#19 0x000055555adce43f in <extern "C" fn(A0) .> R as v8::support::CFnFrom<F>>::mapping::c_fn ()
#20 0x000055555952d6df in Builtins_CallApiCallbackGeneric ()

If I check what its comparing, its comparing the function name to the function docs, but the function docs are an invalid pointer

(gdb) info registers
rax            0x15                21
rbx            0x55555c661e40      93825110777408
rcx            0x0                 0
rdx            0x14                20
rsi            0x55555c661e40      93825110777408
rdi            0x5555              21845
rbp            0x7ffffffeabd0      0x7ffffffeabd0
rsp            0x7ffffffeaba8      0x7ffffffeaba8
r8             0xfcfe              64766
r9             0x7ffff72481e0      140737339752928
r10            0xd                 13
r11            0x14                20
r12            0x5555              21845
r13            0x14                20
r14            0x0                 0
r15            0x7ffff4f55f80      140737303109504
rip            0x7ffff7e1dfb0      0x7ffff7e1dfb0 <__strncmp_sse42+48>
eflags         0x10283             [ CF SF IF RF ]
cs             0x33                51
ss             0x2b                43
ds             0x0                 0
es             0x0                 0
fs             0x0                 0
gs             0x0                 0
fs_base        0x7ffff7c8c480      140737350517888
gs_base        0x0                 0
(gdb) x/s 0x55555c661e40
0x55555c661e40: "JSCallback:anonymous"

rdi register have an invalid pointer

The 2 (name and doc) should come from the same buffer which is this one https://github.com/denosaurs/deno_python/blob/main/src/python.ts#L479 docs: https://docs.python.org/3/c-api/structures.html#c.PyMethodDef

I also inspected that buffer when the segfault happened, and it seemed correct

Uncaught TypeError: Unsupported type: function

import { NamedArgument, python } from "https://deno.land/x/[email protected]/mod.ts";

const gi = python.import("gi");
gi.require_version("Gtk", "4.0");
const Gtk = python.import("gi.repository.Gtk");

function onActivate(app: any) {
  const win = Gtk.ApplicationWindow(new NamedArgument("application", app));
  win.present();
}

const app = Gtk.Application();
app.connect("activate", onActivate); // error here

app.run();

error: Uncaught TypeError: Unsupported type: function
        throw new TypeError(`Unsupported type: ${typeof v}`);
              ^
    at Function.from (https://deno.land/x/[email protected]/src/python.ts:533:15)
    at PyObject.call (https://deno.land/x/[email protected]/src/python.ts:760:57)
    at Proxy.object (https://deno.land/x/[email protected]/src/python.ts:258:20)
    at file:///home/mrcool/dev/deno/lab/a.ts:13:5

Numpy example issue

Thanks for creating deno_python. I tried to run NumPy example but faced this issue.
Could you help me to resolve this issue?

error: Uncaught (in promise) PythonError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.8 from "/usr/local/bin/deno"
  * The NumPy version is: "1.22.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: dlopen(/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so, 2): Symbol not found: _PyBaseObject_Type
  Referenced from: /usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so
  Expected in: flat namespace
 in /usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-darwin.so

Traceback:
['  File "/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/__init__.py", line 144, in <module>\n    from . import core\n', '  File "/usr/local/Cellar/[email protected]/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/numpy/core/__init__.py", line 49, in <module>\n    raise ImportError(msg)\n']
  throw new PythonError(errorMessage);
        ^
    at maybeThrowError (https://deno.land/x/[email protected]/src/python.ts:641:9)
    at Python.importObject (https://deno.land/x/[email protected]/src/python.ts:717:7)
    at Python.import (https://deno.land/x/[email protected]/src/python.ts:727:17)
    at file:///Users/hoangpq/WebstormProjects/deno-working/server.ts:3:25

But working well on Python REPL

Environment:
OS: macOS
Python version: 3.8 (Homebrew)
Deno version: 1.18.0

Could bunpy be updated along with deno_python?

The bunpy npm package is still on the v0.3.3 release; it would be good if it updates continued to be published to npm as releases are made (particularly as bunpy has stopped working for me since updating python).

CC @DjDeveloperr

Inspecting a JScallback returns an error

This requires this pr #65 to work, otherwise it will segfault

import { python } from "/home/mrcool/dev/deno/others/deno_python/mod.ts";

const inspect = python.import("inspect");
inspect.signature(python.callback(() => {}));

no signature found for builtin <built-in method JSCallback:anonymous of NoneType object at 0x7f218cf558a0>
error: Uncaught (in promise) PythonError: no signature found for builtin <built-in method JSCallback:anonymous of NoneType object at 0x7f218cf558a0>  File "/usr/lib64/python3.12/inspect.py", line 3335, in signature
    return Signature.from_callable(obj, follow_wrapped=follow_wrapped,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/inspect.py", line 3075, in from_callable
    return _signature_from_callable(obj, sigcls=cls,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/inspect.py", line 2592, in _signature_from_callable
    return _signature_from_builtin(sigcls, obj,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.12/inspect.py", line 2382, in _signature_from_builtin
    raise ValueError("no signature found for builtin {!r}".format(func))

PythonError type name is lost

value.toString in python for an exception loses the type information which is particularly handy for debugging

suggested alternative

  let errorMessage = (value ?? type).getAttr("__repr__").call().toString();

should void also be added to PythonConvertible

Currently this python.callback((_, _) always gives a type error, becuase it seems the jscallback default to void which is not a PythonConvertible.

deno_python/src/python.ts

Line 61 in 025e9da

export type PythonConvertible =

Currently the workaround is to always annotate the type, mostly like this python.callback((_, button: Gtk_.ToggleButton): undefined

If void can be add as a PythonConvertible this would remove this unnecessary typing.

Combination of positional parameters and named argument throw segmentation fault

Hi @DjDeveloperr

I try this test but got 67010 segmentation fault in MacOS Big Sur.

const { Test } = python.runModule(`
class Test:
  def test(self, *args, **kwargs):
    return all([len(args) == 3, "name" in kwargs])
`);

const t = new Test();
const d = python.dict({ a: 1, b: 2 });
const v = t.test(1, 2, new NamedArgument("name", "vampire"), d);
assertEquals(v.valueOf(), true);

After investigating the issue. I saw that we have a segfault when we try to new tuple but not fill all items
https://github.com/denosaurs/deno_python/blob/main/src/python.ts#L568

TypeError: 'builtin_function_or_method' object does not support vectorcall

This is the reproduction I have, you need to keep moving the slider fast until it throws the above error or sefaults

import {
  Adw,
  Gtk,
  Gtk_,
  kw,
  NamedArgument,
  python,
} from "https://raw.githubusercontent.com/sigmaSd/deno-gtk-py/13f3da6c4890d62e09312747c905fa85263f5ca8/mod.ts";

class MainWindow extends Gtk.ApplicationWindow {
  #slider: Gtk_.Scale;
  constructor(kwArg: NamedArgument) {
    super(kwArg);
    this.set_default_size(600, 250);
    this.#slider = Gtk.Scale();
    this.#slider.set_range(0, 10);
    this.#slider.connect("value-changed", this.slider_changed);

    this.set_child(this.#slider);
  }

  slider_changed = python.callback(
    (_kwargs, slider: Gtk_.Scale): undefined => {
      slider.get_value(); // this line trigger the error/segfault
    },
  );
}

class App extends Adw.Application {
  #win: MainWindow | undefined;
  constructor(kwArg: NamedArgument) {
    super(kwArg);
    this.connect("activate", this.onActivate);
  }
  onActivate = python.callback((_kwarg, app: Gtk_.Application): undefined => {
    this.#win = new MainWindow(new NamedArgument("application", app));
    this.#win.present();
  });
}

const app = new App(kw`application_id=${"com.example.com"}`);
app.run(Deno.args);

warnings dont't show up

some warnings don't show up when using this library, for example:

a.py

import gi
gi.require_version('Gtk', '4.0')
from gi.repository import Gtk
from gi.repository import Gio

def on_activate(app):
    win = Gtk.ApplicationWindow(application=app)
    win.get_style_context()
    win.present()

app = Gtk.Application()
app.connect('activate', on_activate)

app.run(None)

a.ts

import {
  Gtk,
  NamedArgument,
  python,
} from "https://raw.githubusercontent.com/sigmaSd/deno-gtk-py/0.1.4/mod.ts";

const activ = python.callback((_, app) => {
  const win = new Gtk.ApplicationWindow(new NamedArgument("application", app));
  win.get_style_context();
  win.present();
});
const app = new Gtk.Application();
app.connect("activate", activ);
app.run();

this warning only appear on the python version

DeprecationWarning: Gtk.Widget.get_style_context is deprecated
  win.get_style_context()

I'm not really sure where does it come from , maybe the python binary itself is needed for such things ? or its lost in some return argument in ffi ?

Support slices and ellipsis

There's currently no way to use slices (:) or ellipsis (...).

Would something similar to NamedArgument be able to solve this?

import { python, Slice } from "https://deno.land/x/[email protected]/mod.ts";

const np = python.import("numpy");
const slice = np.array([1, 2, 3]).__getitem__(new Slice(1,3,null)); // equivalent to [1:3]
console.log(slice); // returns [2, 3]

problem with javascript unicode characters sent as arguments to python

reproduction examples

> python.builtins.eval("'中文'#foo", {})
中文
> python.builtins.eval("'中文'", {})
Uncaught PythonError: ('EOL while scanning string literal', ('<string>', 1, 3, "'中文"))
> python.builtins.str("'中文'#foo")
'中文'
python.builtins.str("'中文'")
'中
> python.builtins.str("中文")
Uncaught PythonError: <class 'str'> returned a result with an error set

How to override class member, methods

import { python } from "jsr:@denosaurs/python";

const m = python.runModule(`
class A:
  a = 4
`);

class B extends m.A {
  a = 5;
}

const b = new B();
console.log(b.a); // results 4 instead of 5

There is a workaround by setting it after super,

import { python } from "jsr:@denosaurs/python";

const m = python.runModule(`
class A:
  a = 4
`);

class B extends m.A {
  constructor() {
    super();
    this.a = 5;
  }
}

const b = new B();
console.log(b.a); // the expected 5

But this not workaround is not always available

For example I want to make a jupyter kernel using deno + deno_python, (for example the echo kernel here https://jupyter-client.readthedocs.io/en/stable/wrapperkernels.html)

but it seems like jupyter inspects the class without instantiating it, so it never reaches the super call, so in this case I want a way to override class members directly

variables have unexpected type

import { python } from "https://deno.land/x/[email protected]/mod.ts";

const m = python.runModule(`
b = True
  `);

console.log(typeof m.b, m.b == true, /*works*/ m.b === true /*doesn' work*/);

function true false

I expected the type of b to be bool but any variables from python always have type function