Comments (13)
I just stumbled across https://github.com/stephank/yarn-plugin-nixify
Maybe we can get some inspiration from that.
@happysalada
from dream2nix.
Question/feature request: I recently started using direnv and it's great for development, but I wonder if direnv and dream2nix can't be joined into a single devShell + prod package provider?
Ideally, when I'm in a repo and direnv runs, dream2nix provides a devShell output that not only includes the correct runtime devDependencies in the shell, but also for the project. For Node.js that would mean replacing node_modules (with a symlink to the Nix store?).
That would mean that when you checkout another commit, it will automatically update all dependencies.
from dream2nix.
@wmertens Could you check #186 and let me know if that works for you?
from dream2nix.
I added my test comments in that PR. Another issue: node_modules/.bin
doesn't get created, and thus packages that provide a bin
attribute in package.json
don't provide binaries (e.g. eslint).
IMHO, ideally, the .bin
directory would get populated and also added to the PATH.
from dream2nix.
Found a big issue: peerDependencies don't work, for example in apollo-server-errors which can't find graphql because of the symlinking.
There are a couple of ways to solve this:
- treat peerDependencies as dependencies?
- copy modules instead of symlinking
- override require or provide a loader (ESM) that gets the list of modules and converts then to the correct nix store path, and wrap
node
with it
The latter has the advantage that you don't have to create new module copies just because their dependency versions are different between projects
UPDATE:
The translator only looks at package-lock.json's dependencies
, which strips the peerDependencies. It looks like it would be better to look at the packages
attribute, which seems to contain the entire tree as it should be created and does retain the peerDependencies.
from dream2nix.
Issue: the package-json translator uses the default npm package and not the one from the specified nodejs version
from dream2nix.
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/yarn-plugnplay-and-direnv-packaging/19759/19
from dream2nix.
I think the correct translation of package-lock.json is by looking at the packages
object, starting at the key ""
for $pkg
. Each $pkg
has the keys dependencies
and peerDependencies
, which should be concatenated.
For each dependency $dep
, use the string prefixes $pkg/node_modules/$dep
down to node_modules
to find the resolved dependency.
Then build the package as normally. Its node_modules will include symlinks to the peerDependencies, and its full set of dependencies determines its input hash, as usual.
Also keep track of dev: true
in each $pkg
, so that for the built package the devDependencies are left out, but not for the devShell.
from dream2nix.
peerDependencies aren't resolved because they were removed a bit ago, because:
#117 (comment)
I'd use an injection temporarily
from dream2nix.
Wouldn't the algorithm I lay out be working whether peer deps are auto-installed or not?
from dream2nix.
I just verified with npm v8, it works as I explained. I'll try to make it work. Basically npm does all the heavy lifting of deciding which versions to use and all we need to do is provide each package with all the deps+peer deps it should be able to see according to npm.
This will make it work almost the same as npm, except that each module is in the store and symlinked, so any deps that it did not specify will not be visible to it. (which would be a bug in that module imho)
from dream2nix.
Wouldn't the algorithm I lay out be working whether peer deps are auto-installed or not?
Your algorithm assumes lock file version >= 2, vs our current package-lock translator only works with lock file version <= 2.
But anyways, I just checked and we install exactly the dependencies of the lock file, nothing else. So, if you have a lock file, you should not run into a missing peerDependencies issue.
But the problem you are likely running into, is that the build toolchain used by your package doesn't cope well with symlinked dependencies. You can try setting installMethod="copy"
via an override on your failing package, which transforms all dependencies to actual copies.
There was still some problems with linked executables when using installMethod="copy"
, which I fixed in #194
@wmertens It would be great if you could let me know if the suggestions together with #194 work for you.
from dream2nix.
If you have any projects still failing because of peerDependencies, please open a github issue providing the project, so I can have a look myself.
from dream2nix.
Related Issues (20)
- pdm: missing `editables` module when entering devShell HOT 15
- Pip installed CLI tool not in PATH HOT 3
- WIP - pdm add spyder # python3.12-pyqt5-5.15.10 - pdm.sourceSelector.<function body> is not of type 'string'. HOT 2
- WIP - pdm - Python version from d2n nixpkgs HOT 1
- [help] usage of multi-derivation-package HOT 1
- Question: Python package with libusb dependency HOT 2
- error: infinite recursion encountered # nix run github:nix-community/dream2nix#__pythonEnv extract-msg HOT 2
- realpath: command not found HOT 1
- How to make the lock building process for python proxy aware?
- Error: attribute 'overrideAttrs' missing when running nodejs-granular-v3 devshell
- bad (null) hash for package from extra pypi index
- Unable to provide nativeBuildInputs to nodejs transitive dependencies
- error: No valid filename found in the list of filenames: - PyQt5_Qt5-5.15.13-py3-none-macosx_10_13_x86_64.whl
- Can't import fsspec gcsfs
- python local developpement example using only requirements.txt and no pyproject.toml
- How to build pip package from source that requires rust toolchain HOT 2
- Support maturin pip build backend HOT 1
- current `dontStrip = true` default causes some python packages to hold on to GCC as a runtime dependency HOT 1
- pdm: not supporting editable dependencies HOT 1
- pdm: groups at the top of pdm.lock don't update once lockfile exists HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from dream2nix.