pnpm / pnpm Goto Github PK
View Code? Open in Web Editor NEWFast, disk space efficient package manager
Home Page: https://pnpm.io
License: MIT License
Fast, disk space efficient package manager
Home Page: https://pnpm.io
License: MIT License
node_modules/.store/[email protected]
This is how modules are stored. However, once you switch node versions (eg, from node 4.2.4 to 5.3.0), you'll be greeted by this:
> require('node-sass')
Error: The `libsass` binding was not found in /xxx/node_modules/.store/[email protected]/vendor/darwin-x64-47/binding.node
This usually happens because your node version has changed.
This sucks. To mitigate this, compiled modules should have the node major version appended to their store ID:
node_modules/.store/[email protected]@node4
While not a complete solution, this allows you to simply run pnpm install
to rebuild modules as needed.
In addition to this, there should be integrity checks that symlinks are linking to the correct versions... I guess.
It'd be nice for the store to be in ~/.pnpm/store
.
The store is currently defined here:
https://github.com/rstacruz/pnpm/blob/20df53b3193d0b548a21d7287e0f9af3b62f8143/bin/pnpm-install#L93
But I'd like to have some features here:
.pnpmrc
~
's.store/[email protected]
=> xxx
, automatically figuring out the relative path to do so. However, it should be amended such that if the store path is NOT inside the package dir (ie, it's in home), the symlink will be absolute, not relativenode_modules/.store
makes sense by default, but there should be an option to put it wherever you want. my main thought is that i'd like to be able to rm -rf node_modules
. if i don't want to rm -rf
the store, then i would place it outside the node_modules
directory.
When you do this:
npm install node-sass
find node_modules/.bin
npm2 and pnpm will get you the node-sass
binstub:
node_modules/.bin
node_modules/.bin/node-sass
but npm3, with its flat dependency tree model, gets you binstubs of deep dendencies:
node_modules/.bin
node_modules/.bin/har-validator
node_modules/.bin/mkdirp
node_modules/.bin/node-gyp
node_modules/.bin/node-sass
node_modules/.bin/nopt
node_modules/.bin/rimraf
node_modules/.bin/sassgraph
node_modules/.bin/semver
node_modules/.bin/sshpk-conv
node_modules/.bin/sshpk-sign
node_modules/.bin/sshpk-verify
node_modules/.bin/strip-indent
node_modules/.bin/uuid
node_modules/.bin/which
node_modules/.bin/window-size
I'm planning on not supporting npm3's model on this, and keeping pnpm / npm2's behavior. While it's possible to support this in pnpm, I personally don't see the benefit of npm3's model and would rather avoid complexity that will not be useful.
Any objections? (cc @davej @misterbyrne @indexzero and others who may be lurking)
Will this work with nvm
?
Cool project, must train muscle memory to prepend a _p_ ๐
https://github.com/rstacruz/pnpm/blob/20df53b3193d0b548a21d7287e0f9af3b62f8143/lib/got.js#L21-L24
This is where downloads are coursed through. It uses got.stream(). It needs to have some way to throttle connections to maybe 4 connections at a time so not to flood registry.npmjs.org with requests.
packages like pg-then
will require('pg')
even though they don't declare it as a dependency. This works with npm since pg-then
ends up being installed in the same node_modules folder as pg
but when they are symlinked from a global store this doesn't work. And babel-runtime
does require('babel-runtime')
. haha
These packages should declare these dependencies in "peerDependencies" but they don't because with npm they can get away without it.
And BTW the hackery I mentioned in #1 won't be necessary if pnpm
support "peerDependencies" and if packages use it correctly
you can do two pnpm install
's at the same time at the same project, though it will probably break things along the way. a lockfile will solve this.
After digging, this behavior is already possible by just using --config
instead of --userconfig
because of this line of rc (used by registry-url). Should this difference be noted in the readme, or for the sake of parity should this be brought up as an issue on rc?
I using babel@6
and fbjs-scripts
(it required babel 5), when I use pnpm install them, [email protected]
will load [email protected]
, not 6.x
.
This project can reproduce the problem. (try npm run build
)
I know that this may sound controversial but at least native npm cli is doing so and this behavior should not be broken.
They should be installed when possible.
I was using pnpm to install exec-time and got this:
exec-time git+https://github.com/capablemonkey/exec-time#e39279713fcb87a9c9e27f5a81d913d7b6853c3d ERROR โ
mocha 1.21 finding ยท
underscore 1.6.0 finding ยท
! git+https://github.com/capablemonkey/exec-time#e39279713fcb87a9c9e27f5a81d913d7b6853c3d: hosted packages not supported
Hey.
Using npm outdated
doesn't work as expected with the linked dependencies.
standard has this structure:
.
'- standard
|- standard-engine
| '- eslint
'- eslint-config-standard-react
'- eslint-plugin-react
unfortunately, eslint
now can't see eslint-config-standard-react
in our setup because of the store format:
.
'- node_modules
|- .store
| |- standard
| |- standard-engine
| |- eslint
| |- eslint-config-standard-react
| '- eslint-plugin-react
'- standard
there are ways around this... and the new 0.12 store format possibly paves the way to make that possible.
also, the npm3-style flat repo might also work here.
This makes it possible to add a single dev
(or prod
) directory to a .gitignore, or a .dockerignore, or similar.
Currently with the flat hierarchy there's no delineation between prod and dev deps (same problem in npm3 tbh: npm/npm#9674 )
Deduping will perhaps be more difficult, so I get this is a big ask.
Doing pnpm install
in a path without package.json shows a cryptic error message and exits with code 1.
~ $ cd /tmp
/tmp $ pnpm i
! Cannot read property 'dependencies' of undefined
For comparison, npm
is also a little strange in its behavior in this case, but it at least exits successfully:
/tmp $ npm i
npm WARN enoent ENOENT: no such file or directory, open '/private/tmp/package.json'
npm WARN tmp No description
npm WARN tmp No repository field.
npm WARN tmp No README data
npm WARN tmp No license field.
# node_modules will be enumerated here, if it exists
Related #47.
We use GemFury to host private npm packages. It handles requests for private packages itself and proxies the rest to the npm registry. So the semver style URLs work for public packages, but not for private ones (since GemFury doesn't support them and 404s).
https://www.npmjs.com/package/registry-url
var registryUrl = require('registry-url')
function registryUrlFor (package) {
if (package.substr(0, 1) === '@') {
return registryUrl(package.split('/')[0])
} else {
return registryUrl()
}
}
registryUrlFor('@foo/bar') //=> registryUrl('@foo') => 'https://registry.foo.io'
This will be useful for private scoped modules support. See #9 @indexzero
It seems like pnpm logs only first target modules to install. Simple test case for pnpm 0.17.0
are no-deps packages as inherits
and ramda
:
$ pnpm i inherits
inherits 2.0.1 โ
$ pnpm i ramda
ramda 0.19.1 โ
$ pnpm i inherits ramda
inherits 0.19.1 โ
$ pnpm i ramda inherits
ramda 0.19.1 โ
Still getting this with 0.17 on different packages each time.
gulp ^3.8.11 ERROR โ
...
getaddrinfo ENOTFOUND registry.npmjs.org registry.npmjs.org:80
GET registry.npmjs.org/deprecated/-/deprecated-0.0.1.tgz
Hi!
I'm currently using an internal npm registry.
Do you think it would be worth exploring the possibility of making this HTTP2 enabled?
This may allow for infinite concurrency with no penalty.
For example we could use this for server/client: https://github.com/molnarg/node-http2
In a new directory enter pnpm install [email protected]
When it's finished, enter pnpm install [email protected]
Nothing happens - the previous version remains.
To see whether it's worth exploring to achieve performance improvements, I quickly implemented something which would use the tarball from the .npm cache if present (and fall back to fetching from the server if not). It only improved the install time by about 7% ๐ (for quite a big project)
However, I then tried (something a little hackier) to force the package resolver to use npm's cached metadata if present (falling back to making a request to the registry if not). This improved the speed significantly:
Current master - no cache:
35.86s user 17.93s system 83% cpu 1:04.81 total
...cache
24.04s user 11.90s system 94% cpu 38.115 total
(BTW there does appear to be a slight performance regression in today's release - at the very end it's taking longer than before)
The current module name pnpm.js
is a bit funky. I wonder would @crcn be interested in releasing the pnpm
name?
If I run npm install
in dir node_modules
, npm will not create a new node_modules
, it'll just put the installed lib under the same node_modules
. but pnpm will create a new node_modules
, I'm not sure if this is expected.
I was pretty excited to hear about this project. I can see that shrinkwrap is not supported at the moment but do you plan to support it at later point? And if so, what's the schedule?
npm install file:../foo/bar
This is a little easier.... I think... I'm not sure. First I need to investigate how npm does this. How does it handle .gitignore? .npmignore?
I think the way to do this is not to do cp -R
, but to tarball the file:
path (with proper paths excluded), and untar it at the destination directory.
Anyone know how to fix this:
fs.js:984
binding.chown(pathModule._makeLong(path), uid, gid, req);
^
TypeError: uid must be an unsigned int
at TypeError (native)
at fs.chown (fs.js:984:11)
at /usr/local/lib/node_modules/pnpm/node_modules/tar-fs/index.js:215:7
at FSReqWrap.oncomplete (fs.js:95:15)
I'm getting a lot of this type of error since getting the latest version...
It looks like a timing thing because it doesn't happen consistently - i.e. not always and when it does with different packages. Always a similar stack trace though.
! ENOENT: no such file or directory, open 'my-app/node_modules/.store/[email protected]/_/package.json'
Error: ENOENT: no such file or directory, open '/my-app/node_modules/.store/[email protected]/_/package.json'
at Error (native)
at Object.fs.openSync (fs.js:584:18)
at Object.fs.readFileSync (fs.js:431:33)
at requireJson (/src/pnpm/lib/fs/require_json.js:10:42)
at symlinkToModules (/src/pnpm/lib/install.js:274:17)
at /src/pnpm/lib/install.js:97:20
From previous event:
at /src/pnpm/lib/install.js:97:10
From previous event:
at install (/src/pnpm/lib/install.js:89:6)
at /src/pnpm/lib/install_multiple.js:14:12
at Array.map (native)
at installMultiple (/src/pnpm/lib/install_multiple.js:13:27)
at /src/pnpm/lib/install.js:222:16
From previous event:
at buildInStore (/src/pnpm/lib/install.js:222:6)
at /src/pnpm/lib/install.js:179:20
at processImmediate [as _immediateCallback] (timers.js:383:17)
npm has these:
npm WARN unmet dependency /home/travis/build/rstacruz/pnpm/node_modules/nixt/node_modules/changelog requires chalk@'^0.5.1' but will load
npm WARN unmet dependency /home/travis/build/rstacruz/pnpm/node_modules/chalk,
npm WARN unmet dependency which is version 1.1.1
npm WARN unmet dependency /home/travis/build/rstacruz/pnpm/node_modules/nixt/node_modules/changelog requires semver@'^4.0.3' but will load
npm WARN unmet dependency /home/travis/build/rstacruz/pnpm/node_modules/semver,
npm WARN unmet dependency which is version 5.1.0
pnpm's model isn't immune to this (in fact is more prone to it), so it might be prudent to have it implemented as well.
It's currently a bash script. I thought this'd speed things up... but I'm not sure it's necessary, plus it breaks Windows compatibility.
I would be up for working on this.
My thinking is that I will mock the network endpoints (registry mocker here: https://github.com/npm/npm-registry-mock) so that it's not network dependent. Could also be integrated with Travis, although I'm guessing that the performance might differ greatly based upon the VM you get assigned when the build is run.
@rstacruz: Do you have any thoughts?
npm install -g bower
Don't know how to do this yet... need to investigate.
npm prefix -g
will show you where node modules are installed globally. Should pnpm dump things into $(npm prefix -g)/lib/node_modules
? what about Windows?
How can npm prefix -g
be obtained programatically?
ied dumps your globals into ~/.node_modules, I believe. Not sure on what the pros and cons of that are at the moment, though.
This will be useful for local npm mirrors like sinopia, and could lead to significant speed boost in downloads.
> npm config get registry
http://localhost:4873/
When I run pnpm i
on a project with plenty of dependenices, it'll install a few packages and then get to a point where it throws:
getaddrinfo ENOTFOUND registry.npmjs.org registry.npmjs.org:443
GET registry.npmjs.org/shebang-regex/%5E1.0.0
except the address on the second line changes, of course, but when I run the command again, it continues to install more packages and fails on another package - never the same one.
Since pnpm is only compatible with node >= 4 anyway, why not use native promises and Object.assign()
?
(for usage with private registries)
# .npmrc
always_auth = true
//<host>/:_authToken=<token>
Relevant: http://blog.npmjs.org/post/118393368555/deploying-with-npm-private-modules
When I executed pnpm install
I've got random error like
getaddrinfo ENOTFOUND registry.npmjs.org registry.npmjs.org:443
GET registry.npmjs.org/mime-types/~2.1.8
But if I run curl https://registry.npmjs.org/mime-types/~2.1.8
I see expected json output.
With each command run library changes.
Ember CLI has a dependency checker which ensures you have the required npm modules installed.
https://github.com/quaertym/ember-cli-dependency-checker
This can currently we worked around by setting an environment variable, e.g.
SKIP_DEPENDENCY_CHECKER=true ember server
Will possibly be fixed by #46
npm install --save express
npm install rstacruz/scourjs
Okay, this is a tough one. I don't even know where to begin describing this... sorry, will come back to it later
When the package.json
in the current directory contains a postinstall
script pnpm
does not run it, but npm
does.
From npm help install
:
With the
--production
flag (or when the NODE_ENV environment variable is set to production), npm will not install modules listed in devDependencies.
pnpm is awesome.. but I'm having issue on installing.. please help.
Issue:
binding.chown(pathModule._makeLong(path), uid, gid, req);
^
TypeError: uid must be an unsigned int
at TypeError (native)
at fs.chown (fs.js:1082:11)
at /usr/local/lib/node_modules/pnpm.js/node_modules/tar-fs/index.js:215:7
at FSReqWrap.oncomplete (fs.js:83:15)
As a new developer to the code base I was hoping you could give some foot notes on where I should start looking if I wanted to implement scoped module support.
Great work with this!
I don't know if it will help you in any way, but if you see any code in https://github.com/dtrejo/urn that you'd like to "steal" / re-use, please feel free. It does "cp -R" to install packages from ~/.npm
when there is no internet.
Cheers and keep up the good work!
David
Note: just looked at the code; probably nothing you'll want to use. Thanks for pnpm!
While it seems superficial, being able to do an analog of npm ls is pretty crucial.
I'll eventually want to have an analog of npm dedupe, which'd allow certain modules that assume a flat dependency tree to work (like standard
, see #5). To do this, we need generate a dependency tree.
Rather than jump into a random feature, are there any discrete chunks of functionality that you would appreciate PRs for?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.