Allow XDG_CONFIG_HOME on macOS about platformdirs HOT 40 CLOSED

This is a contentious topic seems and given there's no agreement here I'll need to decline the request to add XDG support on Windows for now.

from platformdirs.

I am in complete support of using XDG directories on all platforms if defined. I have been working with the XDG spec and trying to adapt it to all platforms in order to make multi-platform development easier.

One of my primary areas of concern is finding a way to determine which python packages can be shared across platforms (pure python) and which need to have distinct directories. XDG talks about using a shared directory structure for a user across multiple platforms and how some directories need to be distinct and some can be shared.

In my case I have a $HOME directory that I am sharing across all platforms and my dot files and .config directory are used for all platforms. In contrast, $HOME/.local/bin is for Linux and $HOME/.local/bin_win is for Windows and set using SDG dirs (Not really because the define the executable directory but don't provide the variable for it yet. I'm working on it. However for other dirs it works great.)

from platformdirs.

I'm a soft -1 for doing any changes related to this on Windows, and -0 for adopting XDG-when-set on MacOS (as long as there's a way to opt out, or if it's made opt-in). I'd lean toward having pip not enable such support on MacOS.

There's absolutely no reason to follow XDG for Windows; and doing so will only complicate the story for files on Windows. For pip, this will mean additional caveats to the configuration file story on Windows.

I'm not sure I'd want pip to support XDG configuration on MacOS either, since we're already technically out of sync with the Apple's specification for where configuration files should be; in more ways than one. Adding XDG as an additional layer of complexity on that would not be something I'd be enthusiastic about, to put it mildly.

I have to disagree with the logic. There is absolutely a reason to support XDG on both MacOS and Windows. It gives developers options. Regular "I don't know anything about computers" users won't care. But, I certainly do.

I share my HOME directory tree across platforms. During development I hate to have to go digging around in AppData to find things. Being able to control that would be very helpful. You may think that Windows has standard directories for everything, but many application don't use them. For years Python only installed in C:\Python.

And along came WSL. Now I can run Linux and Windows at the same time and share files. Being able to have a project directory seen by both platforms is great for development. With XDG dirs I can have python tools install into $HOME/.local/bin or $HOME/.local/bin_win.

Please remember that adding support for XDG dirs doesn't affect anyone that doesn't use them (for MacOS and Windows). On Linux if not set they should default appropriately.

from platformdirs.

Got surprised I could not find a way to specify PDM config dir on macOS, in order to change it from

~/Library/Application Support/pdm

to much more appropriate

~/.config/pdm

which is effectively XDG_CONFIG_HOME, as per XDG Base Directory Specification

Digging into PDM code led me eventually to this thread, as well as I found your referring some macOS Guidelines:

https://github.com/platformdirs/platformdirs/blob/b866d404c74d3467ab7a16e2be974612b14145b7/src/platformdirs/macos.py#L10-L11

Having config stored at

~/Library/Application Support

is anything but Unix thing. It's also not exactly macOS native thing, as all plist configs should go to

~/Library/Preferences

but I guess it's fine for some cross platform, especially, GUI apps, to use the former.

Because macOS is certified Unix (inspired by BSD at POSIX level), it's not unnatural to expect basic Unix conformance, which is luckily the case (notwithstanding its infamously being known to deviate from all kinds of standards).
It also implies one should be able to live a decent life, without need to know about Library and other macOS-specific stuff (given they enjoy Unix-only workflows).

The concern I can immediately come up with (there might be more) is related to dotfiles management, e.g. using one of tools:

• Home Manager
• stow
• dotdrop
• dotbot
• chezmoi

Besides:

$HOME/.config

$HOME/.local/share

which rules out their placement to Application Support. For this to work,
it's expected for CLI apps to have their config dirs configurable, if they opt for Application Support relative path.

With all respect to the maintainers: it's disappointing that the issue was closed, and, from what I see, PRs are not too much welcome. Please kindly correct me if I'm wrong, @gaborbernat.

from platformdirs.

I think we were 🤔

from platformdirs.

from platformdirs.

Closing per above.

from platformdirs.

I want (some version of) this change personally too, but I think we have to be quite careful -- the Apple guidelines are clear, so I'm:

• +1 on "follow XDG_* if set, even on macOS"
• But -1 for any suggestion of following the XDG-recommended behavior (which says what to do when XDG_ is unset) on macOS
• But +0 on "add a way for libraries using platformdirs to opt-in to that behavior even on macOS", which is sometimes useful -- e.g. ~/Library/Application Support has a space in it, so for libraries that install scripts with shebangs and put them there, they literally won't execute (unless they use the newer ~/Library/ApplicationSupport spaceless symlink), and for those, always defaulting to ~/.local/share, a la XDG_DATA_HOME's default seems potentially reasonable

(Not sure you were suggesting anything about default behavior, but yeah just mentioning my own opinion on all 3 decisions).

from platformdirs.

Would you accept a PR to extend the XDG_* environment variables to Windows as well as Mac? I've ran into cases on Windows where I would like to override the cache directory using an environment variable. My thought on the implementation would be to
have all systems obey the environment variable like Unix currently does and to revert to the current logic if it is not set.

from platformdirs.

I'm a soft -1 for doing any changes related to this on Windows, and -0 for adopting XDG-when-set on MacOS (as long as there's a way to opt out, or if it's made opt-in). I'd lean toward having pip not enable such support on MacOS.

There's absolutely no reason to follow XDG for Windows; and doing so will only complicate the story for files on Windows. For pip, this will mean additional caveats to the configuration file story on Windows.

I'm not sure I'd want pip to support XDG configuration on MacOS either, since we're already technically out of sync with the Apple's specification for where configuration files should be; in more ways than one. Adding XDG as an additional layer of complexity on that would not be something I'd be enthusiastic about, to put it mildly.

from platformdirs.

My opinion is that it should be the applications's responsibility to enable this behaviour and not exposed as an env var to end users

I fundamentally disagree with this personally -- regardless of mechanism, applications make crappy choices about where to put things sometimes (e.g. if platformdirs made this choice and someone hated it, they should be able to control their own system and apps running on it).

So I at times have agreed with @pradyunsg (that if we support XDG everywhere it should be opt-in by the app), but no matter what, I am -1 on any change that does not provide a user-facing way to control the choice (i.e. in addition to the app-maintainer's choice).

Fundamentally, libraries should not be configured from the environment.

(I don't disagree with this though, I think some day platformdirs may unfortunately need a config file).

from platformdirs.

As a Windows user, I would dislike the adoption of specs from other platforms.

My opinion is that it should be the application's responsibility to enable this behaviour and not exposed as an env var to end users

I agree with this, and with the compromise of respecting env vars for an application given its name.

from platformdirs.

Please remember that adding support for XDG dirs doesn't affect anyone that doesn't use them (for MacOS and Windows).

We hear your points, but please respect the decision of the maintainer team on this topic 👍 Unless the current maintainers come back that they've changed their minds there's nothing else to do.

from platformdirs.

I do not believe this to be the case to the extent you wish to be.

I suppose it was meant that macOS is POSIX certified? But at that level we are just splitting hairs and getting nowhere further to solving a major pain point that users are complaining about here.

The commenters in this issue have given plenty of use cases for supporting XDG_CONFIG_HOME on macOS.

Being technically correct about this or that is not the same as "doing the right thing". To me, open source is about respecting user freedom.

from platformdirs.

(if we were to advertise a way to mutate these dirs I would instead prefer we have an explicit PLATFORMDIRS_* set of vars for the fully blessed case) But I could be convinced off of that too and just stick with these if others are for it.

I would prefer this as well. Using XDG on windows feels wrong and that way we don't run into issues causing unintended behavior for other applications that are using those environment variables.

I'm a soft -1 for doing any changes related to this on Windows

Does that also include adding PLATFORMDIRS_* env vars?

from platformdirs.

@gaborbernat is the maintainer team also against implementing it on MacOS? If not - might be worth keeping this open?

from platformdirs.

What about a opt-in environment variable PLATFORMDIRS_USE_XDG? Poeple who don't have it will not be affected. It will be a quite simple patch:

--- __init__.py.bak     2023-03-25 16:36:36.753498600 +0800
+++ __init__.py 2023-03-25 16:38:20.665189000 +0800
@@ -19,6 +19,9 @@


 def _set_platform_dir_class() -> type[PlatformDirsABC]:
+    if os.getenv("PLATFORMDIRS_USE_XDG"):
+        from platformdirs.unix import Unix as Result
+
     if sys.platform == "win32":
         from platformdirs.windows import Windows as Result
     elif sys.platform == "darwin":

from platformdirs.

(Not sure you were suggesting anything about default behavior, but yeah just mentioning my own opinion on all 3 decisions).

For the record, I'm +1, +0, and +1 on those three points.

from platformdirs.

Apple does not follow the XDG spec, so at best what I've seen is people roughly mapping it, see https://stackoverflow.com/questions/3373948/equivalents-of-xdg-config-home-and-xdg-data-home-on-mac-os-x

If anyone has better links/documentation on what exactly would mean following the XDG spec when the XDG env-vars are set feel free to clarify.

from platformdirs.

How do you identify if the Python app is CLI or not?

from platformdirs.

from platformdirs.

It gets tricky because platformdirs might be a dependency of one of your libraries, so the caller codebase might not have direct access to it, and could not necessarily tell in what env type is used. We could make it a global state I guess, something like

platformdirs.set_cli_type(False)

So then GUIS could call part of their setup that code, even if it's used by a transient dependency.

from platformdirs.

from platformdirs.

I'd personally rather have one global flag. Are we aware of other languages/libraries respecting XDG on macOS as precedence?

from platformdirs.

from platformdirs.

Feel free to open a pull request 👍

from platformdirs.

How should backwards compatibility be handled?

from platformdirs.

What do you suggest?

from platformdirs.

Pushed my proposal for backwards compat to #35, which is mostly complete (just needs more tests, a bit of bikeshedding, and a rebase).

from platformdirs.

* unless they use the newer `~/Library/ApplicationSupport` spaceless symlink

This symlink is created by iTerm2, it's not a new addition to the OS.

from platformdirs.

I personally would be alright with this, however, considering its wide-ranging implications I'd like to get input of some of the downstream users; e.g. pip cc @pradyunsg, pytest @asottile @nicoddemus. Plus the opinion of other maintainers: @RonnyPfannschmidt @ofek @Julian.

from platformdirs.

I felt icky about it when seeing the email notification last night, but I can't place my finger on why yet :)
I am obviously pro doing it on macOS, so I slept on it personally and couldn't think of a reason Windows users should get left out -- I think to me the justification is that a user who sets this envvar, regardless of platform, likely intends to use it. So yeah +0 for now from me for doing this on all OSes.
I would feel less good however about proactively telling users that this is the "recommended" way to get this behavior on all OSes, because indeed these envvars do not have wide adoption on Windows, and even macOS to some extent, so other software will not work this way -- (if we were to advertise a way to mutate these dirs I would instead prefer we have an explicit PLATFORMDIRS_* set of vars for the fully blessed case) But I could be convinced off of that too and just stick with these if others are for it.

from platformdirs.

I don't think pytest uses platformdirs/appdirs yet though many of its consumers do

so my opinion is less rooted but I'll give it anyway:

• personally (and I know I'm wrong so no need to tell me!) platform-specific directories confuse me and my users so I tend to just use xdg everywhere (the pre-commit cache for instance on every platform will end up in ~/.cache/pre-commit)
• this makes support and triage much easier since it cuts out the "oh you're on windows, uh look in hmm %APPDATA%\Local, or was it Roaming err..."

that said, I think having some way to configure the directories via an environment variable is a desirably feature, and XDG is fairly well specced to enable such a feature. there's an additional question as to how you handle XDG considering a "default" -- and in that case you'd probably want to ignore XDG's recommended defaults and use the platform-specific defaults. the other edge case is when the xdg variable is present but empty -- which seems to imply a bit more "I want xdg" but typically empty should be treated the same as missing (though a popular "standard" (NO_COLOR) gets this wrong as well)

anyway, I'm not a maintainer here -- nor am I currently a direct consumer -- nor am I a maintainer of a direct consumer -- so feel free to ignore my thoughts on this :) -- I'm going to unsubscribe from notifications and if you need me please @-mention directly

from platformdirs.

Are there any well know / used standards for this on windows?

IMHO the platform specific override mechanism should be used

from platformdirs.

CC @zooba in case he cares to opine.

from platformdirs.

if we were to advertise a way to mutate these dirs I would instead prefer we have an explicit PLATFORMDIRS_* set of vars for the fully blessed case [...]

I was subscribed to this issue as a macOS user who'd potentially benefit from this change.

My opinion is that it should be the applications's responsibility to enable this behaviour and not exposed as an env var to end users that would affect every application which depends on platformdirs that they might have installed, by creating a parallel XDG not-spec. Fundamentally, libraries should not be configured from the environment.

from platformdirs.

I don't disagree with providing an escape hatch and I totally sympathise that applications can and do make crappy choices - I disagree that it should take the form of a platformdirs-specific global override. platformdirs could auto-generate (and respect) APP_NAME_* env vars for an application given its name, but the same env var should not be used for all applications serviced by platformdirs, unless it is part of a standard (like XDG), or unless you are setting out to, actually, standardise configuration overrides.

from platformdirs.

I understand.

from platformdirs.

Because macOS is certified Unix

I do not believe this to be the case to the extent you wish to be.

from platformdirs.

Sometimes freedom from incorrect implementation is more important than freedom to do broken things out of preference

from platformdirs.