Support pyproject.toml and poetry dependencies about piku HOT 10 OPEN

I got bored and took a stab at it. It seems to work for me, but I've only started touching piku today. So this code is almost certainly not ready to get merged. So I'm making it a diff here instead. I'm very unlikely to work on making this friendly enough to get merged, as I'm not sure I'm going to stick with piku at this point.

Do not expect me to clean this up and merge this. I encourage someone to do that, if they wish.

How it's supposed to work:

Poetry is mean and won't let you specify where the venv lives. You can specify the directory venv's will be created, but you can not control the name of the directory. Piku does not like this.

The only way to have full control of the name is to put poetry in the same dir as the app, and then it's called .venv

Well piku doesn't work that way, piku needs to be able to put the ENV wherever it wants, and it needs to know the name of it outside the deploy_ function, so we do a symlink from .venv out to wherever piku wants it: join(ENV_ROOT,app).

Then it's just a matter of running poetry install.

--- piku-upstream.py    2024-06-09 23:21:28.421502747 +0000
+++ piku-poetry.py      2024-06-09 23:22:49.873828550 +0000
@@ -394,6 +394,8 @@
                 workers.pop("preflight", None)
             if exists(join(app_path, 'requirements.txt')) and found_app("Python"):
                 settings.update(deploy_python(app, deltas))
+            if exists(join(app_path, 'pyproject.toml')) and found_app("Python"):
+                settings.update(deploy_poetry(app, deltas))
             elif exists(join(app_path, 'Gemfile')) and found_app("Ruby Application") and check_requirements(['ruby', 'gem', 'bundle']):
                 settings.update(deploy_ruby(app, deltas))
             elif exists(join(app_path, 'package.json')) and found_app("Node") and (
@@ -679,6 +681,41 @@
         call('pip install -r {}'.format(requirements), cwd=virtualenv_path, shell=True)
     return spawn_app(app, deltas)

+def deploy_poetry(app, deltas={}):
+    """Deploy a Python ppoetry application"""
+    virtualenv_path = join(ENV_ROOT, app)
+    symlink_path = join(APP_ROOT,app,'.venv')
+    if not exists(symlink_path):
+        first_time = True
+        symlink(virtualenv_path,symlink_path,target_is_directory=True)
+    first_time = False
+    requirements = join(APP_ROOT, app, 'pyproject.toml')
+    env_file = join(APP_ROOT, app, 'ENV')
+    # Set unbuffered output and readable UTF-8 mapping
+    env = {
+        'POETRY_VIRTUALENVS_IN_PROJECT':'1',
+        'PYTHONUNBUFFERED': '1',
+        'PYTHONIOENCODING': 'UTF_8:replace'
+    }
+    if exists(env_file):
+        env.update(parse_settings(env_file, env))
+    # Install dependencies for poetry
+    #echo("----> virtualenv path at:{}".format(virtualenv_path), fg='green')
+    if not exists(virtualenv_path):
+        first_time = True
+        call("poetry install", cwd=join(APP_ROOT,app),env=env,shell=True)
+    # TODO: improve version parsing
+    # pylint: disable=unused-variable
+    version = int(env.get("PYTHON_VERSION", "3"))
+
+    activation_script = join(virtualenv_path, 'bin', 'activate_this.py')
+    exec(open(activation_script).read(), dict(__file__=activation_script))
+
+    if first_time or getmtime(requirements) > getmtime(virtualenv_path):
+        echo("-----> Running poetry install for '{}'".format(app), fg='green')
+        call('poetry install', cwd=virtualenv_path, shell=True)
+    return spawn_app(app, deltas)
+

 def deploy_identity(app, deltas={}):
     env_path = join(ENV_ROOT, app)

Again: do not expect me to clean this up and make a merge request at this point in time, if ever.

LOL I just saw a possible bug, first_time = False should be above the symlink creation, since it sets it True. This shouldn't matter in practice, since first_time will get reset when the venv doesn't actually exist and poetry install goes and makes it.

from piku.

Well, thanks for this! It’s enough for someone to take a stab at it—TBH I don’t have the bandwidth for testing it extensively, but if I start using poetry I’ll have a good starting point…

from piku.

I had zero problems getting https://github.com/piku/sample-python-app to work.

from piku.

Hi! I don't use poetry (I tend to not be able to install too many extra packages on restricted environments), but if you do and want to take a stab at a pull request, that would be most welcome.

You can start around L395. and make sure to update the deployment function currently starting at L646.

from piku.

@birlorg can you publish your fork and I'll take it for a test drive?

from piku.

@danihodovic

Maybe, I rarely use Github, let's see if I can figure out how to do that easily.

UPDATE: I think I did it?

Github and my piku host didn't get along with a git push. So I manually applied my diff. I haven't run it to know if it has any syntax errors, I didn't test at all. You will have to fix those yourself, if there are any, but they should be stupid little things I would assume. Since it worked at least once on my host.

For replication purposes: I used Ubuntu 24.04 LTS. I installed poetry via pipx install poetry as the piku user. pipx was installed via apt-get install pipx as root.

My commit on my repo: birlorg@f62089c

from piku.

It was a pain to set up a project with Poetry, Django, Redis and Celery so I'm trying Dokku instead.

from piku.

I'm curious, since I have exactly the same stack except for Poetry. @danihodovic, what didn't work for you?

from piku.

I guess the breaking point was lack of poetry support and missing documentation for how to run Django with Celery. The example project in this repository didn't work. On the other hand I found Dokku rather well-documented and memory constraints are not a problem of mine.

I like that Piku provides an alternative for those that don't want to run docker containers.

from piku.

The example project in this repository didn't work.

That's good feedback thanks.

The other stuff is probably out of scope without somebody championing Poetry and Celery I guess.

from piku.