rave-engine / python3-android Goto Github PK

Python 3 cross-compilation tools for Android.

License: Do What The F*ck You Want To Public License

Shell 86.04% Makefile 13.96%

python3-android's Introduction

Python 3 Android

This is an experimental set of build scripts that will crosscompile Python 3 for an ARM Android device.

This project is not maintained anymore. Do check out @GRRedWing's and @yan12125's forks for a continuation of development on Python 3 for Android.

Usage

make clean for good measure.
For every NDK/API Level/Toolchain combination you wish to build for:
- Edit env to match your (desired) configuration.
- make to build everything!
- (Optional) make test to setup an Android emulator and run automated Python regression tests.

Requirements

Building requires:

A working host toolchain that is able to compile Python (for hostpython).
Patience.

Testing requires:

Java 6 to use the Android SDK manager.
awk and tr for some setup wizardry.
Even more patience.

Both require:

A working bash and basic *nix utilities like cp and touch.
wget to fetch files.
tar to extract files.

FAQ

The build is failing with something about license terms!

Read the license terms, edit env and set ANDROID_AGREE_LICENSE_TERMS=y if you agree with them, and re-run.

python3-android's People

Contributors

Stargazers

Watchers

Forkers

checko m4rtink grredwings yan12125 tempdban nethunterjack masterloyalty samimohsin alfian878787 cattalk2 clintonkildepstein waynus yzbjack

python3-android's Issues

Resulting python3.3 binary segfaults.

Currently, the resulting python3.3 binary in $BUILD/bin crashes when launched. It seems to trigger a segfault in Bionic:

F/libc    (19156): Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 19156 (python3.3)
I/DEBUG   (  273): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
...
I/DEBUG   (  273): pid: 19156, tid: 19156, name: python3.3  >>> bin/python3.3 <<<
I/DEBUG   (  273): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 00000000
W/NativeCrashListener( 1032): Couldn't find ProcessRecord for pid 19156
I/DEBUG   (  273):     r0 00000000  r1 00000000  r2 00000000  r3 00000000
I/DEBUG   (  273): AM write failure (32 / Broken pipe)
I/DEBUG   (  273):     r4 00000000  r5 bef22a4c  r6 00000000  r7 b6f4dfd8
I/DEBUG   (  273):     r8 00000001  r9 00d3a030  sl 00000000  fp 00d3a020
I/DEBUG   (  273):     ip 0018dc2c  sp bef229d0  lr b6f2f4a1  pc b6f270b0  cpsr 600e0030
I/DEBUG   (  273):     d0  0000000000000000  d1  0000000000000000
I/DEBUG   (  273):     d2  0000000000000000  d3  0000000000000000
I/DEBUG   (  273):     d4  0000000000000000  d5  0000000000000000
I/DEBUG   (  273):     d6  0000000000000000  d7  ac5327c000000000
I/DEBUG   (  273):     d8  0000000000000000  d9  0000000000000000
I/DEBUG   (  273):     d10 0000000000000000  d11 0000000000000000
I/DEBUG   (  273):     d12 0000000000000000  d13 0000000000000000
I/DEBUG   (  273):     d14 0000000000000000  d15 0000000000000000
I/DEBUG   (  273):     d16 c1d4ea5d87124dd3  d17 3f50624dd2f1a9fc
I/DEBUG   (  273):     d18 41aa6eb704000000  d19 0000000000000000
I/DEBUG   (  273):     d20 0000000000000000  d21 0000000000000000
I/DEBUG   (  273):     d22 0000000000000000  d23 0000000000000000
I/DEBUG   (  273):     d24 0000000000000000  d25 0000000000000000
I/DEBUG   (  273):     d26 0000000000000000  d27 0000000000000000
I/DEBUG   (  273):     d28 0000000000000000  d29 0000000000000000
I/DEBUG   (  273):     d30 0000000000000000  d31 0000000000000000
I/DEBUG   (  273):     scr 00000010
I/DEBUG   (  273): 
I/DEBUG   (  273): backtrace:
I/DEBUG   (  273):     #00  pc 000230b0  /system/lib/libc.so (strlen+83)
I/DEBUG   (  273):     #01  pc 0002b49d  /system/lib/libc.so (strdup+4)
I/DEBUG   (  273):     #02  pc 00012f48  /data/py3/bin/python3.3

Seeing as the fault address is 0, it's very likely a NULL pointer being passed around somewhere.

find fails at -mindepth and other problems

I know this is experimental, but I am reporting minor problems I found when using it....

in clean.sh change the order of arguments to find since -mindepth must be before -type (at least on my platform Ubuntu 14.04, find complains.)
in openssl-1.0.0l.sh, this aborts if the directory already exists (when I was trying to clean/fetch/build after already failing once):

([[ ! -d "${PREFIX}/share" ]] && mkdir "${PREFIX}/share") || exit 1

I am not sure what the script intends to do, reuse an existing directory or make another, so I am not sure how to fix it. Anyway, I commented out building openssl to get by this step.

cross compiling python fails at the step:
python3 -S -m sysconfig --generate-posix-vars
......
ImportError: cannot import name 'new_module'
I haven't solved this yet, but it seems like it might be Python 2 versus 3, or some step I missed.

Thanks. (I am actually interested in cross compiling Python statically, I am not sure whether your script is doing that, since it is building shared libraries.)

How to build for Android 5.1

I built it on ArchLinux (64bit). Then I copy the files to my android phone and run python in sh.
But when I run $ ./python3.4 I got this error:

error: only position independent executables (PIE) are supported.

Then I search on the web about this question and changed the env file to add -fPIE:

# Compile flags.
export CFLAGS_EXTRA="-fPIE"
export CPPFLAGS_EXTRA=""
export CXXFLAGS_EXTRA=""
export LDFLAGS_EXTRA="-fPIE -pie"
export MAKEOPTS_EXTRA=""

But I can't build it! When I run make I got this error:

arm-linux-androideabi-gcc -shared --sysroot /home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build-tools/10d-21-arm-linux-androideabi-4.9/sysroot -L/home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build/10d-21-arm-linux-androideabi-4.9/lib -L/home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build-tools/10d-21-arm-linux-androideabi-4.9/lib -fPIE -pie --sysroot /home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build-tools/10d-21-arm-linux-androideabi-4.9/sysroot -L/home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build/10d-21-arm-linux-androideabi-4.9/lib -L/home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build-tools/10d-21-arm-linux-androideabi-4.9/lib -fPIE -pie -Wl,-hlibpython3.4m.so -o libpython3.4m.so Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/bitset.o Parser/metagrammar.o Parser/firstsets.o Parser/grammar.o Parser/pgen.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/capsule.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/bltinmodule.o Python/ceval.o Python/compile.o Python/codecs.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/import.o Python/importdl.o Python/marshal.o Python/modsupport.o Python/mystrtoul.o Python/mysnprintf.o Python/peephole.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/random.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o   Python/thread.o Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o  Modules/_threadmodule.o  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o  Modules/pwdmodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/_weakref.o  Modules/_functoolsmodule.o  Modules/_operator.o  Modules/_collectionsmodule.o  Modules/itertoolsmodule.o  Modules/atexitmodule.o  Modules/_stat.o  Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o  Modules/zipimport.o  Modules/faulthandler.o  Modules/_tracemalloc.o Modules/hashtable.o  Modules/symtablemodule.o  Modules/xxsubtype.o Python/frozen.o   -ldl   -lm ; \
if test libpython3.4m.so != libpython3.4m.so; then \
    ln -f libpython3.4m.so libpython3.4m.so; \
fi
/home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build-tools/10d-21-arm-linux-androideabi-4.9/sysroot/usr/lib/crtbegin_dynamic.o:crtbrand.c:function _start: error: undefined reference to 'main'
collect2: error: ld returned 1 exit status
arm-linux-androideabi-gcc -shared --sysroot /home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build-tools/10d-21-arm-linux-androideabi-4.9/sysroot -L/home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build/10d-21-arm-linux-androideabi-4.9/lib -L/home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build-tools/10d-21-arm-linux-androideabi-4.9/lib -fPIE -pie --sysroot /home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build-tools/10d-21-arm-linux-androideabi-4.9/sysroot -L/home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build/10d-21-arm-linux-androideabi-4.9/lib -L/home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/build-tools/10d-21-arm-linux-androideabi-4.9/lib -fPIE -pie -Wl,--no-as-needed -o libpython3.so -Wl,-hlibpython3.so libpython3.4m.so
arm-linux-androideabi-gcc: error: libpython3.4m.so: No such file or directory
Makefile:609: recipe for target 'libpython3.so' failed
make[1]: *** [libpython3.so] Error 1
make[1]: Leaving directory '/home/d/data3/sceext/_/o/201512_android/python3_android/python3-android/src/Python-3.4.3'
makefile:30: recipe for target 'python-3.4.3' failed
make: *** [python-3.4.3] Error 1

Python regression tests fail.

This is a meta-bug for all failing regression tests as a continuation of issue #2.

This is the current output of python3.3 lib/test/regrtest.py -v (warning: long):
http://txt.shiz.me/N2QzNDI3
http://txt.shiz.me/YmYwOTdj

Some observations:

A fair amount of tests actually succeed in the first place! Yay.
Some tests (17: aifcs, 24: audiooop, 82: ctypes, 299: sunau) seem to fail because of the dynamic modules not being linked properly against their requirements. This can be fixed easily by adding those libraries to their setup.py stanzas. Should be fixed by 5097284.
Some tests (6: types, 11: all, 13: _locale, 63: codecs, 89: decimal, 119: float, 180: locale, 187: mailbox, 289: strftime, 369: zipfile) break because of Bionic's awful locale support. This is a somewhat harder fix, but probably doable. Mostly fixed by 9e1b39d.
Some tests (6: types, 71: complex, 98: difflib, 119: float, 123: format, 124: fractions, 190: math, 211: optparse) fail because of floating point rounding errors. I don't know exactly where to start fixing those, but once it's fixed centrally, all those tests should pass. Some of this has been fixed by setting a proper locale: see 9e1b39d
Some tests (37: bz2, 229: pipes, 233: platform, 235: poll, 236: popen, 269: select, 298: subprocess) fail because their use of subprocess and shell=True, directly or indirectly. This fails because Android has its shell at /system/bin/sh, not /bin/sh as subprocess expects. Should be an easy fix. Fixed by 1e59bbd and 907f1f9.
A few (date)time-related tests (38: calendar, 105: email, 156: imaplib, 181: logging, 333: unicode) seem to fail because of a ValueError: character U+1f70025 is not in range [U+0000; U+10ffff], where the character sometimes differs. Not quite sure what's causing this.
multiprocessing.SemLock doesn't seem to be implemented, causing at least one test (72: concurrent_futures) to fail. sem_open seems to be unimplemented in Bionic, so I'm not sure if we can fix this.
100: distutils fails because it expects a compiler to be installed, and because, at least on my phone, it writes to temporary .sh files which aren't marked executable, and then tries to execute them.
Some tests (248: pwd) fail because of Bionic's broken user-related functions (getpwent() and friends) support.
~~47: cmd_line fails because of broken UTF-8. This may be a result of the broken locale implementation.~~ Fixed by f89ccb5.
~~Some tests (105: email) fail because of an unset timezone. This may be a result of the broken locale implementation.~~ Fixed by 9e1b39d.
Signals/faults seem to be broken, failing 112: faulthandler and 211: os.
Android doesn't set $HOME, breaking 239: posixpath.
Test 286: ssl ~~and 316: threading~~ seem to hang the entire test run. They had to be skipped.

ossaudiodev.c:33:27: fatal error: sys/soundcard.h: No such file or directory

The file can be found in the NDK source tree in the include directory linux/soundcard.h. I guess the python sourcecode needs a patch to search for this file in the correct location.

Fatal Python error: Py_Initialize: can't initialize sys

Hi, I build Python-3.4.2 follow the build.sh, but run failed.

python python3.4.2
android 4.4.3

export PYTHONHOME=/data/Python-3.4.2

/data/Python-3.4.2/bin/python3

Fatal Python error: Py_Initialize: can't initialize sys
sys.excepthook is missing
object : ValueError('character U+7461642f is not in range [U+0000; U+10ffff]',)
type : ValueError
refcount: 1
address : 0x76cf7660
lost sys.stderr
[1] + Stopped (signal) python