Comments (10)
First of all Termux is Andorid app and all Android apps has directories specificated by Android SDK. We just took the values of such methods as getCacheDir as a standard and turn them back without referring to the sdk.
The use of the XDG specification in applications other than Termux is unacceptable. Also I'm not sure how this approach can break current applications and why it cannot be used
from platformdirs.
For all intents and purposes, termux is a full unix environment. Moving app specific directories outside their default directories has other issues like inability to access the data through Android storage access framework as termux only exports the $HOME
directory to other apps/file manager. All other unix packages other than python/pip
packages store their application data in this way. This has always been the default even with python/pip
packages that don't use platformdirs
.
Termux Packaging policy: 7 says:
- Patches: hardcoded path references
Software often relies on paths defined by Filesystem Hierarchy Standard:
/bin
/etc
/home
/run
/sbin
/tmp
/usr
/var
These paths do not exist in Termux and have been replaced by prefixed
equivalents. Termux installation prefix is/data/data/com.termux/files/usr
and can be considered as virtual rootfs.
Home directory is stored outside of prefix:
/data/data/com.termux/files/home
Don't hardcode home and prefix, use shortcuts
@TERMUX_HOME@
and
@TERMUX_PREFIX@
respectively. Patch files are preprocessed before
being applied.Directories
/run
and/sbin
should be replaced by
@TERMUX_PREFIX@/var/run
and@TERMUX_PREFIX@/bin
respectively.
It would be great, if you follow established norms rather than creating new ones.
from platformdirs.
I don't create new norms 👍
In addition to termux, there are still a large number of applications using python
- Apps were built with p4a or beeware
- Python IDEs (PyDroid, QPython etc)
They all need to define directories, but they don't have the same file hierarchy as termux, so summarizing all cases, it seems to me more rational to use Android (Google) standards rather than add exceptions for each application.
if SAF is your only problem, you can always make a link from home to the directories you need:
$ pwd
/data/data/com.termux/files/home
$ ln -s ../../shared_prefs shared_prefs
P.S. Android support was added in 2.1.0, not in #63
from platformdirs.
There are two scenarios:
- Python pkg is used to build android apps(p4a)
- Python pkg is used inside a android app(development environment like Termux/Pydroid)
In scenario 1, The pkg can be used on any platform like unix or mac to create apk
s for use on android. They have their own mechanism to determine the needed android storage paths. Platformdirs in this case, has nothing to do with android, but with the actual development platform(say unix).
In scenario 2, a android app is used to simulate a full linux distro. I would guess more than 95% python packages used in android are only used in termux
. Other use cases(like scenario 1 are extremely rare). Even developmental environmental packages you mentioned like pydroid, actually used termux shell as base. All these apps have accessible os.environ
to get export
ed shell variables. It's only logical to consider these environments full unix
environments and use XDG specs. At the very least, if a XDG variable is available, it should be checked and retrieved and used.
Other than SAF:
- The current
platformdirs
tries to treat android not as aunix
distro but as a single app pip
and other packages, which usedappdirs
or otherwise used XDG directories in termux.- New packages will lose existing config
- Backups based on old configs won't get restored
- The config directory and cache directory(
/data/data/com.termux/shared_prefs
and/data/data/com.termux/cache/
) currently in use are for the app (termux
) cache and config itself. For eg, configurations related to height of the shell or whether to use full screen or what extra keyboard keys should be shown etc(List here). This shouldn't be used by internal packages. - Shared preferences use expensive java methods to write. That directory shouldn't be polluted by sub-packages(though they may not use the same expensive methods).
This is creating a new norm by treating all packages inside termux use the same config directories as the parent app and ignores a parent-child/master-slave relationship. At the very least, XDG
shell variables should be respected.
from platformdirs.
In scenario 1, The pkg can be used on any platform like unix or mac to create apks for use on android ... Platformdirs in this case, has nothing to do with android, but with the actual development platform(say unix).
I think you are confusing something. Platformdirs, of course, directly executed on android, becuse it was packed to APK via p4a or beeware so APKs buit via these packages should be taken into account.
They [p4a] have their own mechanism to determine the needed android storage paths.
Yes, it has, but methods only determine the path to the internal/external storage, not to specific dirs, e.g. a data/cache and other dirs. Also, android
module (with pyjnius) can be replaced by more fast, lightweight platformdirs, not requiring compilation
In scenario 2, a android app is used to simulate a full linux distro. I would guess more than 95% python packages used in android are only used in termux. Other use cases(like scenario 1 are extremely rare). Even developmental environmental packages you mentioned like pydroid, actually used termux shell as base. All these apps have accessible os.environ to get exported shell variables. It's only logical to consider these environments full unix environments and use XDG specs. At the very least, if a XDG variable is available, it should be checked and retrieved and used.
Yes, they may be less popular than Termux, but 1+ million downloads on each (Pydroid, QPython) should say something.
Pydroid didn't use termux shell as base and similar IDEs not is used to simulate a full linux distro and they are not required to have XDG env vars
- The current platformdirs tries to treat android not as a unix distro but as a single app
- pip and other packages, which used appdirs or otherwise used XDG directories in termux.
- New packages will lose existing config
- Backups based on old configs won't get restored
You are talking about backward compatibility, but what kind of compatibility can we talk about if Android was not considered in appdirs. All this is on the shoulders of developers moving from appdirs to platformdirs
The config directory and cache directory(/data/data/com.termux/shared_prefs and /data/data/com.termux/cache/) currently in use are for the app (termux) cache and config itself. For eg, configurations related to height of the shell or whether to use full screen or what extra keyboard keys should be shown etc(List here). This shouldn't be used by internal packages.
This is creating a new norm by treating all packages inside termux use the same config directories as the parent app and ignores a parent-child/master-slave relationship.
Why packages cannot use this directory? All packages data contains in separated folders, and won't conflict with termux xml preferences. Sub-foldering each package, i think, follow parent-child/master-slave relationship
Shared preferences use expensive java methods to write. That directory shouldn't be polluted by sub-packages(though they may not use the same expensive methods).
Of course, it can be used, and even polluted, and this won't affect performance in any way. Again, this won't conflict with the Termux settings
Summing up, we can say that this approach is a problem for backward compatibility with appdirs, but it is a working for every method of porting python to Android
from platformdirs.
The problem I see here is platformdirs
gives undue preference to scenario 1(python built apks) than scenario 2(shell apk using python and other unix tools), but scenario 2 is disproportionately more common.
Even if you want to support scenario 1, the use of shared_preferences
for config and cache
for cache is completely arbitrary. Hard coded paths are never considered wise in any android development. While the code uses Context().getFilesDir()
, it fails to call getCacheDir()
or getSharedPreferences()
, but adds known strings to the path retrieved from getFilesDir()
. What if the cache and config directory changes relative to the files directory in any current vendor modified OS or other custom android OS or even a future android update?
More importantly, if it fails to find paths using jnius
, it resorts to hacky way of using regular expressions to build what it thinks should be the correct path. I think this is the point, where it should look for XDG
configs instead of trying to create hard coded arbitrary paths.
Pydroid didn't use termux shell as base and similar IDEs not is used to simulate a full linux distro and they are not required to have XDG env vars
They're still shells. They have env variables. If it's not required by them to have XDG env vars, it's also not required by them to have ANDROID_DATA
or ANDROID_ROOT
env vars. If these are not present, platformdirs automatically falls back to using unix package directories with XDG configs.
If others come across this issue, as a hacky temporary fix for termux, change the android data variable, before pip install
:
export ANDROID_DATA=''
Expect some packages like am
to break.
Since only Python packages are a issue, instead of changing default shell envs, we can also change python os.environ
using sitecustomize.py
, which will be invoked on each python execution:
The file should be placed in
$PREFIX/lib/pythonX.Y/site-packages/sitecustomize.py
import os
# Get $PREFIX
prefix=os.environ.get('PREFIX')
# Fail Android check
# https://github.com/platformdirs/platformdirs/blob/73e00ef12bb50bb886fe8a135b7a54085efd131a/src/platformdirs/__init__.py#L27
os.environ['ANDROID_DATA']=''
# Fix XDG DIRS
# https://github.com/termux/termux-packages/wiki/Termux-file-system-layout
os.environ['XDG_DATA_DIRS']=prefix+'/share'
os.environ['XDG_CONFIG_DIRS']=prefix+'/etc'
os.environ['XDG_RUNTIME_DIR']=prefix+'/var/run'
from platformdirs.
Even if you want to support scenario 1, the use of shared_preferences for config and cache for cache is completely arbitrary. Hard coded paths are never considered wise in any android development. While the code uses Context().getFilesDir(), it fails to call getCacheDir() or getSharedPreferences(), but adds known strings to the path retrieved from getFilesDir(). What if the cache and config directory changes relative to the files directory in any current vendor modified OS or other custom android OS or even a future android update?
I know, there are a lot of GSIs and ROMs, but I have never seen such a vendor with modified getCacheDir
and other, and I would like to consider cases practically, not theoretically.
If AOSP has changed paths, I will definitely find out about it as quickly as i will find out about Termux will abandon xdg
More importantly, if it fails to find paths using jnius, it resorts to hacky way of using regular expressions to build what it thinks should be the correct path. I think this is the point, where it should look for XDG configs instead of trying to create hard coded arbitrary paths.
Yes, it is a hacky way, but it works today correctly. Moreover, android.os.Environment.get* uses the same method of getting directories
They're still shells. They have env variables. If it's not required by them to have XDG env vars, it's also not required by them to have ANDROID_DATA or ANDROID_ROOT env vars. If these are not present, platformdirs automatically falls back to using unix package directories with XDG configs.
ANDROID_DATA
and ANDROID_ROOT
defined in AOSP in init.environ.rc
, so all apps has these vars
It seems to me it would be more correct if you put everything directly on the code, platformdirs always open for PR 👍
from platformdirs.
ANDROID_DATA and ANDROID_ROOT defined in AOSP in init.environ.rc, so all apps has these vars
Not exactly. Termux, Pydroid and other IDEs doesn't have to, just like XDG env variables doesn't need to be present. Termux does have this present, because they're explicitly added to the shell environment for android related unix packages, but it isn't needed and it can be modified easily.
Are you willing to break up android support to these two scenarios? Say, check for prefixed environments using $PREFIX
shell env and fallback to unix/XDG spec directories in these dev environments?
from platformdirs.
Not exactly. Termux, Pydroid and other IDEs doesn't have to, just like XDG env variables doesn't need to be present. Termux does have this present, because they're explicitly added to the shell environment for android related unix packages, but it isn't needed and it can be modified easily.
Doesn't have to? These variables adds to environment of each unix (android) process. Termux creates its own environment, so copies these variables, and as you have already said yourself, removal or modification is fraught with
some packages like am to break.
Are you willing to break up android support to these two scenarios? Say, check for prefixed environments using $PREFIX shell env and fallback to unix/XDG spec directories in these dev environments?
Yep 🤝
from platformdirs.
Today I found that pip can't read my configuration in ~/.config/pip/pip.conf
on termux, it is definitely a breaking change for termux users. And every application on termux stores its config files in ~/.cache
except those apps depend on platformdirs. I think #72 is a good idea, hopefully it can be accepted.
from platformdirs.
Related Issues (20)
- Testing could fail with pytest-randomly HOT 1
- Windows support link broken HOT 2
- `user_runtime_dir` for root on Unix
- user_runtime_dir for *BSD HOT 3
- Feature: add user_downloads_dir HOT 1
- Feature: user_desktop_dir HOT 2
- Cannot import name '__version__' from 'platformdirs.version' HOT 6
- Unix user_log_dir doesn't create log dir HOT 1
- Optionally redirect `user_*_dir()` calls to `site_*_dir()` for root on Unix HOT 2
- Add `site_state_dir()` counterpart for `user_state_dir()` HOT 1
- Add `site_log_dir()` counterpart for `user_log_dir()`
- pre-commit updates of pyproject.toml:[build-system]:requires HOT 3
- Support Homebrew prefixes
- HOME and StartMenu? HOT 1
- Missing changelog for 4.0 HOT 3
- Feature Request: Be able to create directories HOT 1
- multipath cannot work for path HOT 5
- Adding to stdlib/pathlib? HOT 13
- Paths can be overridden using environment variables on Linux, but not MacOS? HOT 8
- `MacOS.site_data_path` returns `:` separated path under Homebrew with multipath HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from platformdirs.