piku / piku Goto Github PK
View Code? Open in Web Editor NEWThe tiniest PaaS you've ever seen. Piku allows you to do git push deployments to your own servers.
Home Page: http://piku.github.io
License: MIT License
The tiniest PaaS you've ever seen. Piku allows you to do git push deployments to your own servers.
Home Page: http://piku.github.io
License: MIT License
Notice this in the code but no minimal mention of it in the docs. (i.e. "scale up/down worker processes (ps:scale) at will.")
@piku.command("ps:scale")
@argument('app')
@argument('settings', nargs=-1)
the piku.py
help shows (somewhat incorrectly?):
ps:scale Show application configuration
It should be showing that this takes arg(s) that specify the number to scale each type to.
This was what I was able to do to scale up the worker
type processes (I have piku
as an alias)
piku ps:scale test worker=2
The log is full of this:
web.1 | [uwsgi-daemons] respawning "./redirector.py" (uid: 1000 gid: 33)
web.1 | Starting redirector HTTPServer.
web.1 | Server params: https://exceltohtml.net/ 52401
web.1 | Traceback (most recent call last):
web.1 | File "./redirector.py", line 24, in <module>
web.1 | HTTPServer(("127.0.0.1", int(port)), Redirect).serve_forever()
web.1 | File "/usr/lib/python3.5/socketserver.py", line 440, in __init__
web.1 | self.server_bind()
web.1 | File "/usr/lib/python3.5/http/server.py", line 138, in server_bind
web.1 | socketserver.TCPServer.server_bind(self)
web.1 | File "/usr/lib/python3.5/socketserver.py", line 454, in server_bind
web.1 | self.socket.bind(self.server_address)
web.1 | OSError: [Errno 98] Address already in use
web.1 | [uwsgi-daemons] throttling "./redirector.py" for 31 seconds
Replicate this bug using this repository.
Just going to put this out there. Don't worry, I don't expect anybody to work on this except me. ;)
At README.md installation refers to a way to install piku cli using piku-bootstrap
but its only available at commit https://github.com/piku/piku/blob/60da61456ed3f44c5d6691ed7a55ecbe0dfff382/piku-bootstrap
Readme reference:
To make life easier you can also install the piku helper CLI. Install it into your path e.g. ~/bin to run it from anywhere.
./piku-bootstrap install-cli ~/bin
Ok... for grins I tried to pushing the python_example to my Piku setup on a raspberrypi.
I get an odd error remote: Error: Invalid Procfile for app 'python_example'.
...
Where can I look to get more info on what is failing?
dwight : ~/piku/examples/python (master)
🍺 $ git push piku master
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 431 bytes | 215.00 KiB/s, done.
Total 5 (delta 2), reused 0 (delta 0)
remote: -----> Deploying app 'python_example'
remote: Your branch is up-to-date with 'origin/master'.
remote: Error: Invalid Procfile for app 'python_example'.
To piku.local:python_example
672e0fe..dabd96b master -> master
the Procfile
:
🍺 $ cat Procfile
wsgi: main:app
worker: python worker.py
no logs that would help...
🍺 $ ssh [email protected] logs python_example
No logs found for app 'python_example'.
NGINX_STATIC_MAPPINGS
is being written into LIVE_ENV
. The problem with that is that if you use NGINX_STATIC_PATHS
when deploying with static files, you get an invalid LIVE_ENV
that looks like this:
NGINX_STATIC_PATHS=/static/:conf/staticfiles/
NGINX_STATIC_MAPPINGS=
location /static/ {
sendfile on;
sendfile_max_chunk 1m;
tcp_nopush on;
directio 8m;
aio threads;
alias /home/piku/.piku/apps/djangosandbox/conf/staticfiles/;
}
APP=djangosandbox
This causes commands such as piku run
to fail:
% piku run djangosandbox ./manage.py
Error: malformed setting ' location /static/ {
', ignoring file.
Traceback (most recent call last):
File "./manage.py", line 8, in <module>
from django.core.management import execute_from_command_line ImportError: No module named 'django'
fresh copy of Jessie 07-05 for Raspberry Pi... when I do python piku.py setup
it gives me an import error.
piku@raspberrypi:~ $ python piku.py setup
Traceback (most recent call last):
File "piku.py", line 16, in <module>
from shutil import copyfile, rmtree, which
ImportError: cannot import name which
@rcarmo this bug can be assigned to me to fix.
remote: -----> Installing node version '10.16.0' using nodeenv
remote: * Environment already exists: /home/piku/.piku/envs/excelhtmltable
remote: * Install node (10.16.0)..cp: cannot create regular file '/home/piku/.piku/envs/excelhtmltable/bin/node': Text file busy
remote: cp: cannot create regular file '/home/piku/.piku/envs/excelhtmltable/bin/node': Text file busy
remote: Traceback (most recent call last):
remote: File "/usr/bin/nodeenv", line 9, in <module>
remote: load_entry_point('nodeenv==0.13.4', 'console_scripts', 'nodeenv')()
remote: File "/usr/lib/python3/dist-packages/nodeenv.py", line 890, in main
remote: create_environment(env_dir, opt)
remote: File "/usr/lib/python3/dist-packages/nodeenv.py", line 731, in create_environment
remote: install_node(env_dir, src_dir, opt)
remote: File "/usr/lib/python3/dist-packages/nodeenv.py", line 605, in install_node
remote: copy_node_from_prebuilt(env_dir, src_dir)
remote: File "/usr/lib/python3/dist-packages/nodeenv.py", line 522, in copy_node_from_prebuilt
remote: callit(['cp', '-a', src_dir + '/%s-v*/*' % prefix, env_dir], True, env_dir)
remote: File "/usr/lib/python3/dist-packages/nodeenv.py", line 461, in callit
remote: % (cmd_desc, proc.returncode))
remote: OSError: Command cp -a /home/piku/.piku/env...ltable/src/node-v*/* /home/piku/.piku/envs/excelhtmltable failed with error code 1
This does not actually break anything because it harmlessly tries to install over the existing binary, but it would be good to put in a check to skip the install if the correct node version is already installed.
Currently pushing any worker type the following is thrown:
remote: Warning: found both 'wsgi' and 'web' workers, disabling 'web'
Probably we need some basic tests to catch this kind of stuff.
@rcarmo if you go into settings you can add the new logo to the "social preview":
This means when github.com/rcarmo/piku
gets shared on Twitter or other social media the logo will show up. Small thing but I think it helps with marketing.
Hello ,
I congratulate you on your work.
But I have a question to which I have no answer: can we use ruby on rails?
thank you
Jbd
@rcarmo what would you think of a PR which supported arbitrary versions of Python via pyenv?
I was thinking that if PYTHON_VERSION
contains a detailed version number like 2.7.15
then pyenv could be invoked to do the work of installing the custom python version and sandboxing the app in there. This means users can be specific about exactly which version of Python they want to run their app under as some in the 3.x series have quite different features.
I could install pyenv during bootstrap if piku-bootstrap
gets merged, or otherwise it could be installed at this point if not already installed. Here are the installation instructions.
A similar thing could be done for nodejs via nvm down the track too.
Hello, thanks for this fantastic project @rcarmo! I just successfully deployed my first piku instance. 🎉
I'm about to issue a PR for something else but I wanted to write this down here while I can as it surprised me. I tried to push a non-master branch to my piku instance and nothing happened whereas on other PaaS this is not a problem and the app deploys as expected.
This is due to a typo in my previous PR: https://github.com/piku/piku/pull/97/files#diff-bc9247ae52bec05571146b5895110a89R714
Sending a patch now.
$ git remote show piku
Usage: piku.py [OPTIONS] COMMAND [ARGS]...
Try "piku.py --help" for help.
Error: No such command "git-upload-pack".
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Would be good to pass some of the basic git commands through.
I pushed NodeJS app to repo and get error:
remote: File "/home/piku/piku.py", line 929, in git_hook
remote: do_deploy(app)
remote: File "/home/piku/piku.py", line 234, in do_deploy
remote: if len(workers):
remote: TypeError: object of type 'NoneType' has no len()
Probably should be same with Go app, which has no Procfile. Imho in this case workers is None:
workers = parse_procfile(procfile)
if len(workers):
This is more of a discussion thread - the former owner of @piku has agreed to forego his username so that we can build a GitHub org under that name, and I was thinking of splitting up this repository into two or three things in different repos:
Thoughts? I haven’t set up the org properly yet (just created it before breakfast to hold on to the name), but would like to get feedback before starting to break repo URLs and suchlike :)
On a fresh install of Buster I'm getting this...
pi@pikuserver:~ $ curl https://piku.github.io/get | sh
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 215 100 215 0 0 1990 0 --:--:-- --:--:-- --:--:-- 1990Downloading piku-bootstrap here and running it.
Looks like this is your first time running piku-bootstrap.
This script will self-install dependencies into /home/pi/.piku-bootstrap now.
Hit enter to continue or ctrl-C to abort.#> Virtualenv setup not found. Installing it into /home/pi/.piku-bootstrap.
#> Downloading & installing Virtualenv.
#> Setting up the virtual environment.
#> Piku repo not found. Installing it into /home/pi/.piku-bootstrap/piku.
Cloning into '/home/pi/.piku-bootstrap/piku'...
remote: Enumerating objects: 12, done.
remote: Counting objects: 100% (12/12), done.
remote: Compressing objects: 100% (12/12), done.
remote: Total 1418 (delta 3), reused 0 (delta 0), pack-reused 1406
Receiving objects: 100% (1418/1418), 862.07 KiB | 2.36 MiB/s, done.
Resolving deltas: 100% (822/822), done.
#> ansible-playbook binary not found. Installing it into /home/pi/.piku-bootstrap.
Failed to install Ansible.
See /home/pi/.piku-bootstrap/install.log for details (you may be missing a C compiler).
The install log shows this interesting part at the end...
arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-9NJ3qw/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-armv7l-2.7/c/_cffi_backend.o c/_cffi_backend.c:2:10: fatal error: Python.h: No such file or directory #include <Python.h> ^~~~~~~~~~ compilation terminated. error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1 ----------------------------------------
ERROR: Command errored out with exit status 1: /home/pi/.piku-bootstrap/virtualenv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-ye1mun/cffi/setup.py'"'"'; file='"'"'/tmp/pip-install-ye1mun/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-zvEFd6/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-18etu7/overlay --compile --install-headers /home/pi/.piku-bootstrap/virtualenv/include/site/python2.7/cffi Check the logs for full command output.
ERROR: Command errored out with exit status 1: /home/pi/.piku-bootstrap/virtualenv/bin/python /home/pi/.piku-bootstrap/virtualenv/local/lib/python2.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-18etu7/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.8,!=1.11.3; platform_python_implementation != '"'"'PyPy'"'"'' Check the logs for full command output.
From reading piku.py
the SCALING
file is created by piku.py
in the ENV_ROOT directory.
I wonder if this would be useful to expose this into the APP_ROOT so that it could be created alongside the ENV
file so that the process scaling could be created at the beginning instead of having to bump the processes with the ps:scale
command. I know this is a one time deal since the ENV_ROOT directory is persistent between deployments, just might be useful to have this as something that could be specified before deployment time.
I'm considering hacking this in sometime in the near future, probably enforcing a threading model. I could do it inside nginx
too, but I might need some TCP port servers and not just HTTP.
I was able to successfully publish the example...
dwight : ~/python-copy (master)
🍺 $ git push piku master
Counting objects: 15, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (12/12), done.
Writing objects: 100% (15/15), 10.80 KiB | 2.16 MiB/s, done.
Total 15 (delta 0), reused 0 (delta 0)
remote: -----> Creating app 'python_example'
remote: -----> Deploying app 'python_example'
remote: Your branch is up-to-date with 'origin/master'.
remote: -----> Python app detected.
remote: -----> Creating virtualenv for 'python_example'
remote: Already using interpreter /usr/bin/python3
remote: Using base prefix '/usr'
remote: New python executable in python_example/bin/python3
remote: Also creating executable in python_example/bin/python
remote: Installing setuptools, pip...done.
remote: -----> Running pip for 'python_example'
remote: Downloading/unpacking bottle (from -r /home/piku/.piku/apps/python_example/requirements.txt (line 1))
remote: Running setup.py (path:/tmp/pip-build-39t24ff4/bottle/setup.py) egg_info for package bottle
remote:
remote: warning: no files found matching 'LICENSE.txt'
remote: Installing collected packages: bottle
remote: Running setup.py install for bottle
remote: changing mode of build/scripts-3.4/bottle.py from 644 to 755
remote:
remote: warning: no files found matching 'LICENSE.txt'
remote: changing mode of /home/piku/.piku/envs/python_example/bin/bottle.py to 755
remote: Successfully installed bottle
remote: Cleaning up...
remote: -----> Spawning 'python_example:wsgi.1'
remote: -----> Setting HTTP to listen on 127.0.0.1:9080
remote: -----> Spawning 'python_example:worker.1'
To 192.168.1.18:python_example
* [new branch] master -> master
but when I try to curl
the running example I get the following...
dwight : ~/python-copy (master)
🍺 $ curl 192.168.1.18:9080
curl: (7) Failed to connect to 192.168.1.18 port 9080: Connection refused
so doing a piku --help
gives you all the commands you can send piku. I'm not sure it is appropriate to include the internal git-hooks since that is plumbing used to make piku work.
...
git-hook INTERNAL: Post-receive git hook
git-receive-pack INTERNAL: Handle git pushes for an app
git-upload-pack INTERNAL: Handle git upload pack for an app
...
IMHO
First of all, very nice idea! Thanks for that.
I'm wondering if and how I can help to get Node.js support to piku.
ansible-playbook was not able to be installed at my box due missing gcc ( aarch64-linux-gnu-gcc )
(virtualenv) root@piku:~# pip install -q "ansible==2.7.10"
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
ERROR: Command errored out with exit status 1:
command: /root/.piku-bootstrap/virtualenv/bin/python /root/.piku-bootstrap/virtualenv/local/lib/python2.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-Ip7tPt/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.8,!=1.11.3; platform_python_implementation != '"'"'PyPy'"'"''
cwd: None
Complete output (124 lines):
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting setuptools>=40.6.0
Using cached https://files.pythonhosted.org/packages/6a/9a/50fadfd53ec909e4399b67c74cc7f4e883488035cfcdb90b685758fa8b34/setuptools-41.4.0-py2.py3-none-any.whl
Collecting wheel
Using cached https://files.pythonhosted.org/packages/00/83/b4a77d044e78ad1a45610eb88f745be2fd2c6d658f9798a15e384b7d57c9/wheel-0.33.6-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8
Using cached https://files.pythonhosted.org/packages/0d/aa/c5ac2f337d9a10ee95d160d47beb8d9400e1b2a46bb94990a0409fe6d133/cffi-1.13.1.tar.gz
Processing ./.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
Building wheels for collected packages: cffi
Building wheel for cffi (setup.py): started
Building wheel for cffi (setup.py): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /root/.piku-bootstrap/virtualenv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-x9EONj/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-x9EONj/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-cd16r2 --python-tag cp27
cwd: /tmp/pip-install-x9EONj/cffi/
Complete output (48 lines):
unable to execute 'aarch64-linux-gnu-gcc': No such file or directory
unable to execute 'aarch64-linux-gnu-gcc': No such file or directory
No working compiler found, or bogus compiler options passed to
the compiler from Python's standard "distutils" module. See
the error messages above. Likely, the problem is not related
to CFFI but generic to the setup.py of any Python package that
tries to compile C code. (Hints: on OS/X 10.8, for errors about
-mno-fused-madd see http://stackoverflow.com/questions/22313407/
Otherwise, see https://wiki.python.org/moin/CompLangPython or
the IRC channel #python on irc.freenode.net.)
Trying to continue anyway. If you are trying to install CFFI from
a build done in a different context, you can ignore this warning.
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-aarch64-2.7
creating build/lib.linux-aarch64-2.7/cffi
copying cffi/api.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/__init__.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/lock.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/commontypes.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/model.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/verifier.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/recompiler.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/vengine_gen.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/error.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/ffiplatform.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/cparser.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/pkgconfig.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/_cffi_include.h -> build/lib.linux-aarch64-2.7/cffi
copying cffi/parse_c_type.h -> build/lib.linux-aarch64-2.7/cffi
copying cffi/_embedding.h -> build/lib.linux-aarch64-2.7/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-aarch64-2.7/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-aarch64-2.7
creating build/temp.linux-aarch64-2.7/c
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-md9kAN/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-aarch64-2.7/c/_cffi_backend.o
unable to execute 'aarch64-linux-gnu-gcc': No such file or directory
error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for cffi
Running setup.py clean for cffi
Failed to build cffi
Installing collected packages: setuptools, wheel, pycparser, cffi
Running setup.py install for cffi: started
Running setup.py install for cffi: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /root/.piku-bootstrap/virtualenv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-x9EONj/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-x9EONj/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-4VveU0/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-Ip7tPt/overlay --compile --install-headers /root/.piku-bootstrap/virtualenv/include/site/python2.7/cffi
cwd: /tmp/pip-install-x9EONj/cffi/
Complete output (48 lines):
unable to execute 'aarch64-linux-gnu-gcc': No such file or directory
unable to execute 'aarch64-linux-gnu-gcc': No such file or directory
No working compiler found, or bogus compiler options passed to
the compiler from Python's standard "distutils" module. See
the error messages above. Likely, the problem is not related
to CFFI but generic to the setup.py of any Python package that
tries to compile C code. (Hints: on OS/X 10.8, for errors about
-mno-fused-madd see http://stackoverflow.com/questions/22313407/
Otherwise, see https://wiki.python.org/moin/CompLangPython or
the IRC channel #python on irc.freenode.net.)
Trying to continue anyway. If you are trying to install CFFI from
a build done in a different context, you can ignore this warning.
running install
running build
running build_py
creating build
creating build/lib.linux-aarch64-2.7
creating build/lib.linux-aarch64-2.7/cffi
copying cffi/api.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/__init__.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/lock.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/commontypes.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/model.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/vengine_cpy.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/verifier.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/recompiler.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/vengine_gen.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/error.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/ffiplatform.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/setuptools_ext.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/cffi_opcode.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/backend_ctypes.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/cparser.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/pkgconfig.py -> build/lib.linux-aarch64-2.7/cffi
copying cffi/_cffi_include.h -> build/lib.linux-aarch64-2.7/cffi
copying cffi/parse_c_type.h -> build/lib.linux-aarch64-2.7/cffi
copying cffi/_embedding.h -> build/lib.linux-aarch64-2.7/cffi
copying cffi/_cffi_errors.h -> build/lib.linux-aarch64-2.7/cffi
running build_ext
building '_cffi_backend' extension
creating build/temp.linux-aarch64-2.7
creating build/temp.linux-aarch64-2.7/c
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fdebug-prefix-map=/build/python2.7-md9kAN/python2.7-2.7.16=. -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/ffi -I/usr/include/libffi -I/usr/include/python2.7 -c c/_cffi_backend.c -o build/temp.linux-aarch64-2.7/c/_cffi_backend.o
unable to execute 'aarch64-linux-gnu-gcc': No such file or directory
error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /root/.piku-bootstrap/virtualenv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-x9EONj/cffi/setup.py'"'"'; __file__='"'"'/tmp/pip-install-x9EONj/cffi/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-4VveU0/install-record.txt --single-version-externally-managed --prefix /tmp/pip-build-env-Ip7tPt/overlay --compile --install-headers /root/.piku-bootstrap/virtualenv/include/site/python2.7/cffi Check the logs for full command output.
----------------------------------------
ERROR: Command errored out with exit status 1: /root/.piku-bootstrap/virtualenv/bin/python /root/.piku-bootstrap/virtualenv/local/lib/python2.7/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-Ip7tPt/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- 'setuptools>=40.6.0' wheel 'cffi>=1.8,!=1.11.3; platform_python_implementation != '"'"'PyPy'"'"'' Check the logs for full command output.
Following up on #90, I'm partial to picking out a simple static file generator to maintain all the docs:
gh-pages
docsify
(which is cute and simple and lightweight but is lousy for SEO)When shipping an ENV file with an application,
FOO=$HOME/bar
...is not correctly expanded. Need to ensure it is done correctly for critical system variables like HOME
, PATH
, etc.
Putting this here so I have something to PR against (preparing a patch now).
I did everything to set up my Raspbery compatible device with piku. All went fine till last testing part.
When I log in to box it did not give me output you show at RASPBERRY_PI_QUICKSTART.md
After this I found that repository is not created at piku machine after push. I actually cannot push at all.
(response was there is no app/repository)
PS. Also there was problem with .ssh directory rights, but I did solve it.
sample-language-app
)Can you implement a web interface for analytics?
Ugh, I've introduced a truly facepalming bug in #77. 🤦♂️
A bunch of vars are being leaked into LIVE_ENV
at write time.
My LIVE_ENV
looks like this at the moment:
APP=pypostgres
NGINX_IPV6_ADDRESS=[::]
PATH=/home/piku/.piku/envs/pypostgres/bin:/home/piku/.piku/envs/pypostgres/bin:/usr/lib/git-core:/usr/lib/git-core:/usr/lib/git-core:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
NGINX_CUSTOM_CLAUSES=
VIRTUAL_ENV=/home/piku/.piku/envs/pypostgres
NGINX_BLOCK_GIT=location ~ /\.git { deny all; }
NGINX_COMMON=
listen [::]:443 ssl http2;
listen 0.0.0.0:443 ssl http2;
ssl on;
ssl_certificate /home/piku/.piku/nginx/pypostgres.crt;
ssl_certificate_key /home/piku/.piku/nginx/pypostgres.key;
server_name pypostgres.piku.mccormickit.com;
# These are not required under systemd - enable for debugging only
# access_log /home/piku/.piku/logs/pypostgres/access.log;
# error_log /home/piku/.piku/logs/pypostgres/error.log;
# Enable gzip compression
gzip on;
gzip_proxied any;
gzip_types text/plain text/xml text/css application/x-javascript text/javascript application/xml+rss application/atom+xml;
gzip_comp_level 7;
gzip_min_length 2048;
gzip_vary on;
gzip_disable "MSIE [1-6]\.(?!.*SV1)";
# set a custom header for requests
add_header X-Deployed-By Piku;
location /static {
sendfile on;
sendfile_max_chunk 1m;
tcp_nopush on;
directio 8m;
aio threads;
alias /home/piku/.piku/apps/pypostgres/static;
}
location ~ /\.git { deny all; }
location / {
uwsgi_pass pypostgres;
uwsgi_param QUERY_STRING $query_string;
uwsgi_param REQUEST_METHOD $request_method;
uwsgi_param CONTENT_TYPE $content_type;
uwsgi_param CONTENT_LENGTH $content_length;
uwsgi_param REQUEST_URI $request_uri;
uwsgi_param PATH_INFO $document_uri;
uwsgi_param DOCUMENT_ROOT $document_root;
uwsgi_param SERVER_PROTOCOL $server_protocol;
uwsgi_param REMOTE_ADDR $remote_addr;
uwsgi_param REMOTE_PORT $remote_port;
uwsgi_param SERVER_ADDR $server_addr;
uwsgi_param SERVER_PORT $server_port;
uwsgi_param SERVER_NAME $server_name;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Port $server_port;
proxy_set_header X-Request-Start $msec;
}
HOME=/home/piku
NGINX_IPV4_ADDRESS=0.0.0.0
NGINX_ACL=
this is a fresh installed piku on a raspberry pi. I've tried changing the port but still get this message? Any clues/help?
I'm seeing this error (every couple of minutes):
wsgi.1 | *** Starting uWSGI 2.0.15 (32bit) on [Thu Nov 14 13:57:34 2019] ***
wsgi.1 | compiled with version: 8.3.0 on 14 July 2019 14:52:34
wsgi.1 | os: Linux-4.19.75-v7+ #1270 SMP Tue Sep 24 18:45:11 BST 2019
wsgi.1 | nodename: pikuserver
wsgi.1 | machine: armv7l
wsgi.1 | clock source: unix
wsgi.1 | pcre jit disabled
wsgi.1 | detected number of CPU cores: 4
wsgi.1 | current working directory: /home/piku/.piku/uwsgi-enabled
wsgi.1 | detected binary path: /usr/local/bin/uwsgi
wsgi.1 | chdir() to /home/piku/.piku/apps/myapp
wsgi.1 | your processes number limit is 7733
wsgi.1 | your memory page size is 4096 bytes
wsgi.1 | detected max file descriptor number: 1024
wsgi.1 | lock engine: pthread robust mutexes
wsgi.1 | thunder lock: disabled (you can enable it with --thunder-lock)
wsgi.1 | uWSGI http bound on 0.0.0.0:9080 fd 6
wsgi.1 | probably another instance of uWSGI is running on the same address (0.0.0.0:9080).
wsgi.1 | bind(): Address already in use [core/socket.c line 769]
the ProcFile
is this:
wsgi: main:app
worker: python3 myapp.py
main.py
is a minimal flask app.
import flask
app = flask.Flask(__name__)
@app.route("/")
def hello():
return "Hello!"
if __name__ == '__main__':
HTTP_PORT = int(os.environ.get('PORT', 8000))pro
BIND_ADDRESS = os.environ.get('BIND_ADDRESS', '127.0.0.1')
DEBUG = 'true' == os.environ.get('DEBUG', 'false').lower()
logging.info('Bind Address: {}:{}' % (BIND_ADDRESS, HTTP_PORT))
logging.info('Debug set to {}' % DEBUG)
app.run(host=BIND_ADDRESS, port=HTTP_PORT, debug=DEBUG)
the ENV
file is this
# bind to the local machine only
BIND_ADDRESS=0.0.0.0
PORT=9080
I'm seeing this WARNING when starting a test app to piku...
web.1 | *** Operational MODE: no-workers ***
web.1 | !!!!!!!!!!!!!! WARNING !!!!!!!!!!!!!!
web.1 | no request plugin is loaded, you will not be able to manage requests.
web.1 | you may need to install the package for your language of choice, or simply load it with --plugin.
web.1 | !!!!!!!!!!! END OF WARNING !!!!!!!!!!
web.1 | spawned uWSGI master process (pid: 3974)
web.1 | [uwsgi-daemons] spawning "python main.py" (uid: 1001 gid: 33)
web.1 | 14-Jun 08:58:38-INFO: started main.py
web.1 | * Serving Flask app "main" (lazy loading)
web.1 | * Environment: production
web.1 | WARNING: This is a development server. Do not use it in a production deployment.
web.1 | Use a production WSGI server instead.
web.1 | * Debug mode: on
web.1 | 14-Jun 08:58:39-INFO: * Running on http://127.0.0.1:9191/ (Press CTRL+C to quit)
web.1 | 14-Jun 08:58:39-INFO: * Restarting with stat
web.1 | 14-Jun 08:58:41-INFO: started main.py
web.1 | 14-Jun 08:58:42-WARNING: * Debugger is active!
web.1 | 14-Jun 08:58:42-INFO: * Debugger PIN: 921-136-616
my ProcFile
:
web: python main.py
worker: python worker.py
I've also tried wsgi: main:app
with the same results.
my main.py
is a very simple Flask app.
import flask
import requests
app = flask.Flask(__name__)
@app.route("/")
def hello():
logging.info('root restapi endpoint executing')
return "<b>Not implemented yet...</b>"
if __name__ == '__main__':
app.run(host='127.0.0.1', port=9191, debug=True)
very simple requirements.txt
file:
requests
flask
when I open piku.local:9191 I get
This site can’t be reached
piku.local refused to connect.
any suggestions?
Undefined name have the potential to raise NameError at runtime.
flake8 testing of https://github.com/rcarmo/piku on Python 3.6.3
$ flake8 . --count --select=E901,E999,F821,F822,F823 --show-source --statistics
./piku.py:264:17: F821 undefined name 'deploy_java'
deploy_java(app, deltas)
^
1 F821 undefined name 'deploy_java'
1
I've just noticed that #81 introduced a dependency on yaml
, which means that the docs need to account for python3-yaml
(the OS package) being installed.
I suggest that checking requirements should be done after an app's language has been detected. It makes it sound more like a PaaS than a know machine that knows what to check requirements for.
Just a logical suggestion.
Like instead of:
remote: -----> Checking requirements: ['java', 'lein', 'clojure']
remote: ['/usr/bin/java', '/usr/bin/lein', '/usr/local/bin/clojure']
remote: -----> Clojure app detected.
remote: -----> Building Clojure Application
to:
remote: -----> Clojure app detected.
remote: -----> Checking requirements: ['java', 'lein', 'clojure']
remote: ['/usr/bin/java', '/usr/bin/lein', '/usr/local/bin/clojure']
remote: -----> Building Clojure Application
Similarly to setting up static routes it should be possible to set up websocket proxying via an environment variable.
https://www.nginx.com/blog/websocket-nginx/
upstream websocket {
server 192.168.100.10:8010;
}
location /wsurl/ {
proxy_pass http://websocket;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
I think it's possible these days to have an app serve both HTTP and WS with the same server & port in which case the URL can default to / and the port & IP will be the same as the default http config (so the setting could be on/off for websockets).
@rcarmo I frequently need this for projects, so I will probably take a shot at implementation at some point in the future if nobody else beats me to it.
Ok, so there have been changes since I installed this on my raspberry pi. Is it time to start tagging releases or is it still too early?
Not 100% sure about this but I am seeing the following during git push
:
remote: Error unexpected '{' in field name in static path spec: should be /url1:path1[,/url2:path2], ignoring.
When I have a the following setting:
NGINX_STATIC_PATHS=/static:static
It appears to be due to the brace characters not being handled correctly by .format()
Hi great work on this. Is it possible to push an arbitrary dockerized app to Piku and skip any auto detection on the source code by simply detecting a Dockerfile and building an image out of that?
Originally posted by @IRobL in #11 (comment)
The Procfile documentation is spread out in various places. Would be good to bring it together into a single place like the ENV file.
It is probably good if you use the virtualenv from Ubuntu repository during installation.
I also noticed that piku's check_requirements(binaries) does not recognize binaries that are exported on the .bashrc file.
So install whatever you need using package manager.
$sudo apt-get install virtualenv maven nodejs etc.
It would be great to be able to do git push
deployments of static sites (e.g. frontend only), just like GitHub pages.
This could be implemented by installing the Nginx config in the case where a) wsgi
and web
workers are not defined, and b) NGINX_STATIC_PATHS
is defined. In this case the root level proxy
stanza should be excluded from the config as there would be no listening server to proxy through to.
Installation does not validate if git client it's installed, does not exit or notify installation error.
root@piku:~# ./piku-bootstrap
Looks like this is your first time running piku-bootstrap.
This script will self-install dependencies into /root/.piku-bootstrap now.
Hit enter to continue or ctrl-C to abort.
#> Virtualenv setup not found. Installing it into /root/.piku-bootstrap.
#> Downloading & installing Virtualenv.
#> Setting up the virtual environment.
#> Piku repo not found. Installing it into /root/.piku-bootstrap/piku.
**./piku-bootstrap: 39: ./piku-bootstrap: git: not found**
#> ansible-playbook binary not found. Installing it into /root/.piku-bootstrap.
Usage:
....
So #94 drove home the point that we need automated linting and tests, and I'm thinking about doing that - either in CI or via some other GitHub app.
Suggestions?
I'm needing to schedule a couple of weekly tasks to run, is using APScheduler a recommended way (https://devcenter.heroku.com/articles/clock-processes-python) or is there an easier way?
Looking at the code it looks like web
and wsgi
are the only ones that get processed and the rest are created as the tag name.
sorry, fairly new to Heroku process types
I've pushed the example in examples/python/
to my piku instance and it works great. The only change I made was to add the variable NGINX_SERVER_NAME
so that I know where to see the result.
However, when I modified main.py
and changed the output tables to add some random text I noticed that the text does not show up in the page after I commit and push to piku/master again. Looking at the logs it does not seem like uwsgi is restarting the server. What should I look for to diagnose this, or is it expected behaviour?
piku-bootstrap install-helper ~/bin/
should make a symlink to the script.
I have tried adding support for Java but once I deploy a Java Application without an isolated environment an error File or Directory not found. Adding a virtualenv path piku automatically spins a python virtual environment and detects a Python App instead a Java App.
I know the directory must exist for the Live config and scaling files.
I did a dummy creation of the app's directory under .piku/envs(app) to enable Java deployments.
I will create the PR soon, but make sure you have uswgi integration plugins. I am using jwsgi and jvm plugins.
recently started using a f-strings in a project and pushed it to piku and noticed that is was giving syntax errors. python3.5.3 installed.
would it causes issues to install 3.6?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.