Proposal: pins/run_constrained support about conda-devenv HOT 8 CLOSED

As I understand this will not affect transitive dependencies.
We could document this behavior but I think it a bit strange.
Say we have:

name: foo
pins:
  - pytest ==6.2.5
  - pytest-mock ==3.0.0
  - pytest-lazy-fixture ==0.6.0
dependencies:
  - pytest-lazy-fixture

As I see it will respect pytest-lazy-fixture ==0.6.0 but not pytest ==6.2.5 (and pytest is a dependency of pytest-lazy-fixture).

from conda-devenv.

As I understand this will not affect transitive dependencies.

Good point.

From your example, with the original proposal, it would generate:

# environment.yml
name: foo
dependencies:
  - pytest-lazy-fixture ==0.6.0

And indeed pytest would be unpinned.

We should resolve/find that pytest-lazy-fixture depends on pytest, and output pytest ==6.2.5 to the environment.yml as well (as well all the other transitive dependencies).

Not sure how to tackle this in an efficient manner though. 🤔

from conda-devenv.

pins = ...  # {package: version_spec, ...}
dependencies = ...  # {package, [version_spec, version_spec, ...], ...}
updated_pins_in_deps =True
while updated_pins_in_deps:
  updated_pins_in_deps = False
  new_dependencies = conda_solve(dependencies)
  transitive_deps = set(new_dependencies) - set(dependencies)
  for dep in transitive_deps:
    if dep in pins:
      updated_pins_in_deps = True
      dependencies[dep].append(pins.pop(dep))

from conda-devenv.

Unless I'm missing something, we need to do a full blown solve of the environment, correct?

Because the same package might have different dependencies depending on its version (say pytest 6 depends on type_extensions, pytest 7 does not), so we need to account for that.

If the only solution is in that direction, then probably best to rethink the whole solution with lock files in mind (or even scrap this idea altogether in favor of using full lock files).

from conda-devenv.

... we need to do a full blown solve of the environment, correct?

I think so. That or fetching the repo data and parsing it, that could be more performant but we will need to handle channel priority and that kind of conda configurations.

from conda-devenv.

Unless I'm mistaken, it is not just a matter of parsing versions, we do need to solve the full environment in order to obtain a reliable list of versions and dependencies.

For example, if I depend on package X, I need to find its pinned version, say 1.1, and from that, I need to be able to fully solve its dependencies for version 1.1, which in turn brings a whole world of possible dependencies and their versions, which in turn need to be fully solved again.

Just to make it clear that we are no longer in the realm of "just list some packages and pin them according to a list of pins", we do need to solve the full environment in order to produce a reliable list of dependencies.

For that reason I think we need to rethink this in terms of lock files, possibly going straight to a lock file solution, instead of this which would be an interim solution.

from conda-devenv.

You are right, do get it done properly we need the actual solve.

from conda-devenv.

Thanks for the inputs @prusse-martin, I'll close this then. 👍

from conda-devenv.