Deprecation warning is pretty intense about dask HOT 11 CLOSED

I got here by searching "Must install dask-expr to activate query planning" which is the error if you try to use dask.dataframe under dask>=2024.3.0. Solved by requiring dask[dataframe] (actually in my case, dask[dataframe,diagnostics,distributed]).

from dask.

(to be clear I'm saying "this concerns me / seems odd" not "change this now please")

from dask.

I don't mind having an option that much but we don't have a way to easily set this for users. I doubt that most users are actually maintaining a dask config in their home directory.

I do like the idea of supporting this dask config set CLI command that does this for the user

from dask.

Coiled implements this FWIW

(test-env) benchmarks:~$ coiled config set foo bar
Updated [foo] to [bar], config saved to /Users/mrocklin/.config/dask/coiled.yaml

from dask.

If we're having to update our own CI due to our warning then I imagine that we're affecting any other project's CI that's depending on us. If an upstream project did this to us I'd feel pretty sad.

from dask.

The deprecation warning is indeed a problem for RAPIDS, and will likely require us to pin to dask-2024.1.1 until after the April release. Although dask-cudf was already planning to provide full support for "dataframe.query-planning": True for RAPIDS-24.04, we cannot support a True default just yet (it must be an opt-in situation for the first release - since down-stream RAPIDS libraries have negative developer cycles to deal with the regressions).

I can't say that I have any brilliant ideas to mitigate this problem for us at the moment. It certainly makes sense to be "loud" about the legacy dask.dataframe deprecation. However, it would be really nice if there was a way that we could still use dask>=2024.2.0 (I'd be very open to suggestions).

from dask.

I agree wholeheartedly with the spirit of @mrocklin reporting this issue. While I realize that there's no perfect solution to evolving APIs and handling deprecations, I had a similar reaction to this change. I want to dutifully stay on near-latest versions of dask because it's the right thing to do, but this deprecation warning pollutes all the CLIs I've written, and there doesn't appear to be a great way to address it.

Will there soon be a version of dask/dask-expr that makes the suggested configuration the default? I'm fine with that, and then writing "hello world" programs would again be noise-free, right?

from dask.

I believe this is closed now by

• #10925
• #10921

Please reopen if I'm mistaken. :)

from dask.

I'm sorry @milesgranger but I haven't kept up with those PRs and looking briefly at the descriptions I'm not sure what they do. Can you help me understand what the new behavior is?

from dask.

Apologies, I should have been more explicit about how those are actually helpful. :)

#10925 adds the dataframe.query-planning-warning config option to avoid the warning altogether.
#10921 adds the convenience dask config set CLI mentioned in the last bit of the opening comment. ie. dask config set dataframe.query-planning-warning False

from dask.

Cool. Looking now at the code for those two PRs, should we add the dask config set language to the warning message?

Also, FWIW I'd still prefer that we just not present this warning message at all. I don't think it's necessary and I suspect that it's somewhat harmful.

from dask.