boltpkg / bolt Goto Github PK

I think it would be nice to be able to run:

bolt up [dep]
bolt workspaces up [dep]
...

We've actually seen this issue a few times before, but have a much better workaround now.

The problem: When running bolt publish, things will work as expected. When running bolt publish from within a script, you are unable to authenticate. This includes using bolt.publish({}) from a node script (if it is called using bolt).

The issue turned out to be the same as what was reportedhere.

...seems like yarn is setting env variable which is npm using for publishing npm_config_registry this feels like yarn issue to me.

This was confirmed to be the case here as well.

There are a few workarounds possible for others with this issue:

• Simply not using yarn or bolt to run the script. npm and npx will work fine.
• Adding a .yarnrc to the project with registry "https://registry.npmjs.org/" in it.
  • Tested, working here.
• Using the --registry flag when publishing
  • Should work, but testing has not been successful so far
• Manually overriding the npm_config_registry variable when running npm publish in bolt
  • Not tested yet, but would be useful to add into bolt.

We are using option 2 in a couple of places atm, but would like to implement option 4 in bolt itself if possible.

It looks like

bolt upgrade packageName

Is not working as expected. It should only upgrade that one package right?

I ran it just now and it upgrades and or installed lots of packages

$ bolt upgrade landkid

⚡️   bolt v0.19.2 (node v8.4.0)
yarn upgrade v1.3.2
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
[3/5] 🚚  Fetching packages...
[4/5] 🔗  Linking dependencies...

# ...

✨  Done in 18.03s.
info Added package "@atlaskit/field-text-area" dependency "@atlaskit/util-readme" at version "^3.6.5"
info Added package "@atlaskit/editor-core" dependency "@atlaskit/util-shared-styles" at version "^2.10.3"
info Added package "@atlaskit/media-viewer" dependency "jquery" at version "^3.2.1"
🏁   Done in 18.79s.

Didnt have time to dig into what it actually did, just flagging here so I can come back and look at it.

Detailed description

I have fresh monorepo with 4 packages in it.
Link to it https://github.com/olstenlarck/hela/tree/v3-major.

Basic assumption is to just run bolt install in the project root and everything to be installed and linked. But it errors with that it can't link external dependencies of the each project.

~/tunnckoCore/hela v3-major *
❯ bolt install
⚡️   bolt v0.20.7 (node v10.3.0)
🔎   [1/3] Validating project...
📦   [2/3] Installing project dependencies...
$ /home/charlike/.config/yarn/global/node_modules/bolt/node_modules/.bin/yarn config get user-agent yarn/1.7.0 npm/?node/v10.3.0 linux x64
yarn install v1.7.0
[1/4] Resolving packages...
success Already up-to-date.
Done in 0.58s.
🔗   [3/3] Linking workspace dependencies...
error Package "@hela/core" dependency "execa-pro" must be added to project dependencies.
error Package "@hela/core" dependency "sade" must be added to project dependencies.
error Cannot symlink invalid set of dependencies.

I figured out that Bolt needs to install each dependency for each project seprately, using bolt like

bolt w @hela/cli add <deps for that package>
bolt w @hela/core add <deps for that package>
bolt w @hela/dargs add <deps for that package>
bolt w @hela/config-tunnckocore add <deps for that package>

So it adds all of them to the root dependencies and once they are there, then cleaning all of the packages' node_modules (e.g. rm -rf packages/cli/node_modules) and running bolt install on the root - everything works without errors.

I can understand why this is needed. But first, there is no docs for this. And second, it would be better before doing anything and the install, to scan each package deps in the defined workpaces, add then add them to the project root and after that start the install process.

This will simplify the manual process that i've done.

So bolt install should act like that:

Oh, okey, workspaces is packages/*, i'm going to scan each package deps - there is packages/core with two deps in its package.json - I'm going to add them as dependencies to the project root (e.g. bolw w @hela/core <found deps>).
Okey, there is packages/cli with 3 dependencies, one of which is the packages/core - i'm going to add them to the project root, except the package from packages/core.
Okey, i'm done, i'll run bolt install now.

this enable to pass options to scripts from cli 🙂

Steps:

• Want to add latest @atlaskit/layer-manager dependency to the @atlaskit/tooltip package
• The @atlaskit/tooltip package.json has no dependencies or devDependencies listed, exists in packages/elements/tooltip
• The @atlaskit/layer-manager is external and is not yet use in the project
• Run bolt w @atlaskit/tooltip add @atlaskit/layer-manager in project root

Expected:

• @atlaskit/layer-manager@^2.1.2 added to dependencies in the root /package.json
• @atlaskit/layer-manager@^2.1.2 added to dependencies in /packages/elements/tooltip/package.json

Result:

• @atlaskit/layer-manager@^2.1.2 added to dependencies in the root /package.json ✅
• @atlaskit/layer-manager@^2.1.2 not added to dependencies in /packages/elements/tooltip/package.json ❌

Fails with output below:

➜  atlaskit-mk-2 git:(master) ✗ bolt w @atlaskit/tooltip add @atlaskit/layer-manager
⚡   bolt v0.14.0 (node v8.4.0)
yarn add v1.2.1
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
warning "[email protected]" has unmet peer dependency "prop-types@*".
warning "[email protected]" has unmet peer dependency "babel-core@6 || 7 || ^7.0.0-alpha || ^7.0.0-beta || ^7.0.0-rc".
[4/4] 📃  Building fresh packages...
success Saved lockfile.
success Saved 5 new dependencies.
├─ @atlaskit/[email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
└─ [email protected]
✨  Done in 8.24s.
error Package "@atlaskit/tooltip" dependency "" must match version in project dependencies. "null" vs "atlaskit/layer-manager"

Running bolt add @atlaskit/layer-manager in the /packages/elements/tooltip directory has the same result.

We have a 'dev' script defined as "dev": "node scripts/dev/index.js", and scripts like "lint:css": "yarn dev lint:css",. Running yarn lint:css --bamboo passes the --bamboo option all the way to the dev script, running bolt lint:css --bamboo loses this option somewhere.

› yarn run lint:css --bamboo
yarn run v1.9.2
$ yarn dev lint:css --bamboo
$ node scripts/dev/index.js lint:css --bamboo
Running stylelint --custom-formatter ./node_modules/stylelint-junit-formatter "./src/**/*.js"  >test-reports/stylelint-css.xml

› bolt run lint:css --bamboo
⚡️   bolt v0.20.7 (node v10.7.0)
$ yarn dev lint:css
$ node scripts/dev/index.js lint:css
Running stylelint "./src/**/*.js"

It would be good if both commands were equivalent so that our users would need to know when to use bolt and when to use yarn and could just always use bolt.

BTW Why didn't eslint detect unsed variable?

Symptoms in logs:

$ npm publish --access public npm
$ npm publish --access public ERR! This package has been marked as private
$ npm publish --access public npm ERR! Remove the 'private' field from the package.json to publish it.

In a big mono-repo subdivided in sub-directories, I'd like to be able to mass-build only some packages/workspaces under a sub-directory.

Example here: https://github.com/Offirmo/hello-world-bolt/tree/master/pro I'd like to be able to bolt w run build only the workspaces under /pro and not all my mono-repo workspaces.

Will --only-fs (currently not implemented) work on the path of the workspace or only on the workspace name?

E.g. if extract-react-types exists in the root package.json with version "^0.10.0" and I run bolt add extract-react-types@^0.11.0 it will not upgrade the package at that version.

Running the same command with yarn upgrades the package correctly: yarn add extract-react-types@^0.11.0.

Is this something bolt add should handle or is this solely supposed to be handled by bolt upgrade?

Really looking forward to using bolt as the one solution for all aspects in building a monorepo!
Just curious whether bolt build will be customizable to doing custom commands, since I have built a tool Clipped that makes it easier for setting up configurations when building js projects.

Example error: https://bitbucket.org/atlassian/atlaskit-mk-2/addon/pipelines/home#!/results/25485

If you look at the error messages there we get a bunch of colorful (completely correct) error messages:

error Package "@atlaskit/adf-utils" must depend on the current version of "@atlaskit/editor-common": "12.0.0" vs "^11.4.0"
error Package "@atlaskit/conversation" must depend on the current version of "@atlaskit/editor-common": "12.0.0" vs "^11.4.6"
error Package "@atlaskit/editor-bitbucket-transformer" must depend on the current version of "@atlaskit/editor-common": "12.0.0" vs "^11.3.8"
error Package "@atlaskit/editor-confluence-transformer" must depend on the current version of "@atlaskit/editor-common": "12.0.0" vs "^11.3.8"

The useful error message is the one down the bottom:

Error: @atlaskit/adf-utils has a dependency on @atlaskit/editor-common at ^11.4.0, however the new version of 12.0.0 leaves this range. You will need to make a new changeset that includes an update to @atlaskit/adf-utils
    at Object.updatePackageVersions (/opt/atlassian/pipelines/agent/build/node_modules/bolt/dist/modern/functions/updatePackageVersions.js:92:15)
    at <anonymous>

It would be nice to make this message much, much more prominent. Ie using color, a section break or a box around it

Difference between two cmd shims found in the same project's node_modules/.bin

Commands left out 🙂 :

• bolt help
• bolt import
• bolt version
• bolt versions
• config
  • bolt config current - thanks @sendilkumarn
  • bolt config delete - thanks @sendilkumarn
  • bolt config set - thanks @sendilkumarn
  • bolt config get - thanks @sendilkumarn
  • bolt config list - thanks @sendilkumarn
• global
  • bolt g add - @sendilkumarn 😃
  • bolt g bin - @sendilkumarn 😃
  • bolt g remove - @sendilkumarn 😃
  • bolt g list - @sendilkumarn 😃
  • bolt g upgrade - @sendilkumarn 😃
• owner -
  • bolt o add - @RajaBellebon 👍
  • bolt o list - @RajaBellebon 👍
  • bolt o remove - @RajaBellebon 👍
• tag
  • bolt tag add - @sendilkumarn 😃
  • bolt tag list - @sendilkumarn 😃
  • bolt tag remove - @sendilkumarn 😃
• team
  • bolt team add - @sendilkumarn 😃
  • bolt team create - @sendilkumarn 😃
  • bolt team destroy - @sendilkumarn 😃
  • bolt team list - @sendilkumarn 😃
  • bolt team remove - @sendilkumarn 😃

Feel free to add if I have missed any and pick any if interested ( just add "taken" next to command you start working on,

❤️

bolt add pkgName --ignore-engines

and

bolt install --ignore-engines

do not work as expected, the flags are not passed on to yarn

When a package is declared in multiple different dependency types in a workspace's package.json, e.g. dependencies and devDependencies, bolt upgrade will add a new entry to package.json with the dependency entry names concatenated:

E.g.

"dependencies,devDependencies": {
  "bunyan": "^1.8.12"
}

I'm happy to take a look and create a PR for this.

getWorkspaces({
  cwd,
  ignoreFs: [
    'packages/css-packs/*',
    'build/*',
    'website'
  ]
});

via @jamiebuilds

So here: https://github.com/boltpkg/bolt#commands

It says:

Note: Bolt is under active development and some of these commands have not yet been implemented.

Followed by a table of bolt commands.

Request: Could we pls have a ✅ or ❌ next to each command to indicate whether that command has been implemented

Reason: Would like to start using bolt, even if some of the it isn't ready yet. Just would like to know at a glance which parts are, and which parts aren't.

[lints]

getting below the error

c:\Users\javascript\Documents\GitHub\bolt\src\types.js. Error: {"exitCode":8,"stderr":".flowconfig:19 Unsupported config section: "lints"\r\n","stdout":""}

The error displayed is:

$ bolt
⚡️   bolt v0.19.2 (node v8.9.4)
🔎   [1/3] Validating project...
📦   [2/3] Installing project dependencies...
yarn install v1.3.2
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 📃  Building fresh packages...
✨  Done in 0.14s.
🔗   [3/3] Linking workspace dependencies...
error my-bin is linked to a location outside of project node_modules: ../../packages/my-bin/index.js

Demo repo here: https://github.com/bengummer/bolt-binary-issue-demo

yarn install works in the demo repo, bolt install doesn't

Feature Request

bolt clear

Add bolt clean or clear command to clean all node_modules from root and nested packages/workpsaces

The idea was supposed to be that bolt would make a best effort to publish all packages that it can and return a list of all the packages it attempted to publish with a flag that shows whether it was published or not.

We made a mistake here in that packages that have a version that is behind npm will log warnings, but are not returned in the list of packages that were attempted.

The fix will involve a little bit of refactoring the getUnpublishedPackages function to instead return all the packages with some data about their current status (unpublished, unableToPublish, upToDate) so that when we return the list at the end, we can also include the unableToPublish ones

I just tried running bolt on a different machine with yarn 0.22.x installed and it fails to run any lifecycle events.

Looks like

yarn run -s script

Doesnt work properly in older yarns. Do we want to support that?

We can make it

yarn run script -s -- other args

And that should be compatible with both.

Yarn 1.6 fixes the long-running issue of native packages being rebuilt on every install. This should make installs dramatically faster.

We were discussing some ideas of how this might work in the Skate repo.
The idea would be to return a list of packages that had changed since master

We think it would look something like:

$ bolt changed
@scope/foo
@scope/bar
zee

but would support a couple of flags

$ bolt changed --glob
(@scope/foo|@scope/bar|zee)

$ bolt changed --fs
packages/elements/foo
packages/fabric/bar
packages/zee

$ bolt changed --json
["@scope/foo", "@scope/bar", "zee"]

This would allow you to make some really nice commands such as

# build only changed packages
BABEL_ENV=es bolt ws --only "$(bolt changed --glob)" exec  -- babel src --out-dir es

show a message, saying updated version of bolt available please update by running yarn global add bolt ?

Feature request: enhance bolt output to discriminate between concurrent workspace commands

Right now when executing a multi-workspace command, it's hard to know what's going on, especially since commands are executed concurrently.

Example 1:

What workspace caused this error? It's not easy to tell...

Example 2:

That would be extremely convenient to be able to know which workspace is the source of each line...

Maybe a prefix?

Currently, we just silently continue.

Not sure if it should be a separate issue or not, but as a part of this, I'd also like publish to return a list of published packages. This will help solve an issue we have currently where we report that everything was published as expected, even when some fail.

Would be nice. Thanks for the good work.

Follow on from the hack added in 69c1c55

We need to find a way to correctly load the users preferred npm registry.

We should be able to just use npm config to find it.

Some more checks we should perform

• peerDeps and devDeps are the same version range (I think this will be a requirement for internal deps only?)
• internal deps can only have caret, tilde and pinned version ranges (but can be anything that semver.satisifies the current version)

Hi guys from bolt ,

From last 2 weeks i've been experimenting lots of tooling related with monorepo management and all of them have some Pros/Cons , after doing some test drive with bolt/rush/lerna ... i started using lerna as my main monorepo manager and after 1 week using it i was missing the simplicity of bolt... there are lots of things going on... i know ... but after that i moved back to bolt and hope that you do not run away and get more people using it and contributing with you on this nice nice tool and journey.

I was suprised that also this guys moved to bolt...

https://bitbucket.org/atlassian/atlaskit-mk-2/src/8dc8b67506e8f91089dd04e7fee4bf576635c553/package.json?at=master&fileviewer=file-view-default

Thanks a lot ...

When i run bolt publish, bolt produce following output (It does not publish packages):

warn No unpublished packages to publish
🏁   Done in 0.04s.

• My packages are in packages located which is in package.json workspaces defined!
• bold init and bolt install worked without any issues!

package.json:

{
  "name": "app-packages",
  "version": "0.1.0",
  "main": "index.js",
  "private": true,
  "workspaces": [
    "packages/*"
  ]
}

packages folder contains:

packages/a/packages.json
packages/a/index.js

packages/b/packages.json
packages/b/index.js

When I run bolt changeset the first question I'm asked is what packages I would like to include.
I select one package, and then I seem to be asked:
"What kind of change is this for "
for every other package in my build.

The answer (for the patch type changes I am making anyway) is always None, None, None, None, None, etc.

Can this be optimised at all?

Just happened in CI twice now where network connectivity issues caused yarn to fail, but bolt install didnt return a non-zero code.

Will look into this

I use bolt in a monorepo but instead of using it to publish packages to npm registry, I use it to deploy web applications to docker. Because bolt only create symlinks, if I run bolt install in a directory and for example I tar the directory. The tar archive will not contain all node_modules required to run the application.

My idea is introducing bolt install --full. It will install alllllll dependencies off a project and no symlinks.

Hi guys, I'd love to see the example updated, or my question answered, using bolt+babel, do you have an example of how the package.json should be structured in order to provide the best packaging experience using es6 so a package can be consumed in different use cases, e.g. browser/node.

#36 refactored bolt install and moved a bunch of validations into a shared module.

The problem is that these validations previously caused Bolt to exit early, now they aren't run until the linking phase.

These validations should be hoisted up

Not 100% sure, but I've seen a few little things that haven't made sense that make me thing that maybe some messages aren't arriving in order. And not just with tasks that are running in parallel.

Installing for example:

⚡️   bolt v0.20.5 (node v8.9.0)
🔎   [1/3] Validating project...
📦   [2/3] Installing project dependencies...
🔗   [3/3] Linking workspace dependencies...
$ /Users/lbatchelor/src/bolt-pkg/bolt/node_modules/.bin/yarn config get user-agent yarn/1.6.0 npm/? node/v8.9.0 darwin x64
yarn install v1.6.0
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
success Already up-to-date.
✨  Done in 2.40s.
💥  success Installed and linked workspaces.
🏁   Done in 11.42s.

It says '[3/3] Linking workspace dependencies...' before we see any output from the yarn install step itself.

I might have a little dig later, it might be something in spawn not flushing a buffer when a task finishes maybe?

I'd like to use the npm_config_user_agent (already set by npm and yarn) to detect whether bolt is being run in my project's preinstall script.

Currently npm / yarn set the following values for the npm_config_user_agent env var

• yarn (and bolt because it is unchanged): yarn/1.3.2 npm/? node/v8.9.4 darwin x64
• npm: npm/5.6.0 node/v8.9.4 darwin x64

Ideally bolt will prepend bolt/<bolt version> to the existing env var value set by yarn.

When running

bolt add --dev pkgA pkgB pkgC

pkgB and pkgC will be added correctly, but pkgA will not.

After some investigation, we think that meow might be taking pkgA and assigning that as the value of the --dev flag, which will still be truthy, so everything else will work as expected.

Just putting some of our discussion about bolt publish here for further discussion and visibility.

Current state

bolt publish currently only accepts one flag (--access) and will currently do the following:
* fetch the latest version of each (non-private) package in the project (npm info packageName version)
* compares these versions to the ones on disk
* for each package who's local version is semver.gt than the version on npm
* npm publish is called (with --access flag passed in)

Feature request

• Users would like to be able to pass the --tag flag in as well.
  • To support this we might need to think about how the flags get applied to all the packages being published (should it apply to all of them?)

Going forward

I think the first thing we should think about is how different users will be using the publish feature.

1. They have a tool that is bumping versions and dependencies and need to publish all packages that are ahead of npm (i.e bolt release)
2. They are doing manual versioning (manual meaning on their local machine, they could be using bolt version for example).
   • They might want to also be doing tagged releases here
3. They have a mono-repo but want lock-stepped versioning across all the packages and want them all the release together (i.e https://github.com/electron-userland/electron-forge CC: @MarshallOfSound )

Suggested Plan

I think we can solve all of these usecases by doing the following

if: we are in a package instead of a project root:
	just pass flags on and call `npm publish`
	exit

if: we are in a non-interactive shell
	(go with current behaviour)
	get all package versions from npm
	get list of our packages that are ahead of npm version (being careful with `semver.gt`
    run `npm publish` with whichever flags were passed to us
	warn/throw if npm is ahead of us
	exit

collect list of packages that are ahead of npm
show list to user and allow them to select which they want to publish
publish only those

Any/all of this functionality could live behind flags (--all, --interactive, --warnOnNpmAhead, just wanted to get this up here for discussion

I'm interested in bolt's take on independent project versioning compared to lerna's. I see the version command is in development but can't find any further detail. I'm also interested in any planned efforts to generate package changelog's.