microsoft / beachball Goto Github PK

The status of a package.json could actually have changed since the change file has been created, so we need to verify this before publishing. We should still do the bumping - just maybe skip the publish of the package for simplicity.

In particular, this line made it not node 8 compatible:

https://github.com/microsoft/beachball/blob/master/packages/beachball/src/getChangedPackages.ts#L53

There may be others, and should replace @types/node with a version 8.x to make sure things are okay.

Running yarn test on a windows machine produces a bunch of errors.

Currently If I have 2 packages:

Package A:

{
  dependencies: {
    packageB: "1.0.3"
  }
}

And I publish package B, it correctly updates the dependencies of package A, but it doesn't publish package A with the new dependencies. This means users of package A will never see the new version of package B.

We should have some kind of option to configure beachball to be more aggressive about publishing updates when their dependencies change.

What is scoped publish?

If a package version is bumped, currently by default beachball will bump version of all its dependents. Scoped publish allows bumping version and publishing for only a subset of its dependents while other dependents remain the same version.

For example, say we have:
PackageA → PackageD
PackageB → PackageD
PackageC → PackageD

When change is made in PackageD, the feature can allow only to have PackageA, PackageB to get version bumped and published and PackageC remains the same version in package.json with dependencies: { 'PackageD' } updated to new version.

What existing problem we are solving?

Enable @fluentui/react packages to publish new version on-demand while office-ui-fabric-react remains the same publish cadence. Once scoped publish is supported, we can run our existing daily release job on all packages in the repo except updating the beachball publish command to scope publish to only non @fluentui/react packages.
When we want to publish @fluentui/react packages, we can have a separate on-demand job to beachball publish everything without scoping.

repro:

master: have change files
branch1: create a feature branch "branch1"
branch1: make changes (same projects as master change files)
master: publish and remove change files
branch1: run beachball change

actual: will think that it doesn't need change files
expected: needs change files

See microsoft/react-native-windows#3896

Steps to repro:

1. Clone react-native-windows repo at commit 7d6357f5aeb5f15d3708d306af7600fb1e7426cd
2. cd into new directory
3. Run "yarn install"
4. Run "npx beachball publish --no-push --no-publish"

Reverting commit 1c8cbe8e79ff30ffb712a36e52ad28d6b891f031, which updated our lockfile resolves the hang.

If the requested package version already exists in the registry, it would be nice if beachball could increment the version and try again instead of bailing out. (Might still have to bail out if the version which already exists is a prerelease, since those are harder to increment automatically.)

quick design:

add a beachball.config.js or package.json npm script to call a "postSuccessfulPublish" script

When trying to run beachball publish on Azure Devops. I get the following error despite having provided the access token:

This is the beachball step on my azure-pipelines.yml

• task: CmdLine@2
  inputs:
  script:
  npm install -g beachball
  beachball change --type minor
  beachball publish -- -y -n $(System.AccessToken) -r https://office.pkgs.visualstudio.com/OC/_packaging/**********************/npm/registry/
  displayName: 'beachball publish

Currently, generated change files have absolute machine paths in the "file" field. This should probably be relative.

If, in my /change directory, I have a set of Change Files like this:

{
  "type": "none",
  "comment": "A Noop Change",
  "packageName": "@scope/some-package",
  "email": "[email protected]",
  "commit": "hash",
  "date": "2019-10-30T22:32:50.474Z"
}

{
  "type": "none",
  "comment": "A Noop Change",
  "packageName": "@scope/some-package2",
  "email": "[email protected]",
  "commit": "hash",
  "date": "2019-10-30T22:32:50.474Z"
}

{
  "type": "none",
  "comment": "A Noop Change",
  "packageName": "@scope/another-package",
  "email": "[email protected]",
  "commit": "hash",
  "date": "2019-10-30T22:32:50.474Z"
}

Notice that they are all of type none.

When the CI runs beachball publish, the files will be deleted (can be simulated with beachball bump locally) but since there is nothing to publish, beachball does note commit those removed files and push back to the repo.

The result is that the "none" files were continue to live in the repo until a minor, major, or patch change is made, and then they will be commit back to the repo.

I'd expect these to be commit back, placed in the CHANGELOG.json, even if they are marked as none changes.

Right now if publishing one package in a monorepo fails, beachball doesn't retry it, which results in a really messy state that has to be fixed manually. Instead it should automatically retry at least once (could add a retry count option).

Handle partial publishes:

minispec:

• add a "sync" command that can synchronize a partially updated git repo with the current versions of the published npm packages.
• delete change files for a subset of these
• should mark partially upgraded versions to be unpublished and set latest dist-tag to the last known good
• on publish AGAIN, make sure to skip over unpublished versions and apply a patch (with a changelog entry) about the skipped version

When multiple changes are being applied in a single version update, it looks like beachball is trying to create a bullet list, but the format isn't quite right:

See 0.59.0-vnext.140 in:
https://github.com/microsoft/react-native-windows/blob/master/vnext/CHANGELOG.md

Every time I'm trying to commit in react-native-windows I'm running into extra pain with yarn change files. I've already committed everything but have other local files and it won't let me create a change file...

For example, I have a local vscode setting so browse files are stored per project instead of on my C: drive, I have to close vscode and delete the files

$ node ./node_modules/beachball/bin/beachball.js change
Target branch is "upstream/master"
ERROR: There are uncommitted changes in your repository. Please commit these files first:

• .vscode/.BROWSE.VC.DB-shm
• .vscode/.BROWSE.VC.DB-wal
  error Command failed with exit code 1.

Right now there's not an obvious indication that beachball has just committed the changefile it created. It would be helpful to add a message, something like this (showing the full output as an example):

Please describe the changes for: office-ui-fabric-react
✔ Describe changes (type or choose one) › SearchBox: allow passing in id for input
✔ Change type › Patch - bug fixes; no backwards incompatible changes.

Created and committed /path/to/change/file!
Be sure to push this commit to your remote branch.

Feature

Currently we bump the version based on the ChangeType defined by user: patch, minor, major
We should enable user to never bump above a certain type/version. This is mainly to safe guard an accidental major bump.n

For example:
We have change file with type major. if user defined max bump to be minor such as beachball publish --max-bump minor. We will bump the package version as minor instead of major

Node version: 10.15.1
Beachball version: 1.14.2

When I run beachball check|bump|publish, any command that checks against origin/master from bamboo CI, and it fails to fetch (separate issue with bamboo environment preventing fetch from working), the command completes with a success even though it should either fail or try to publish packages.

Error log:

build	06-Nov-2019 18:57:33	
build	06-Nov-2019 18:57:33	> [email protected] checkchange /home/bamboo/bamboo-agent-home/xml-data/build-dir/[REDACTED]
build	06-Nov-2019 18:57:33	> beachball check --changehint "Run 'yarn change' to generate a change file"
build	06-Nov-2019 18:57:33	
build	06-Nov-2019 18:57:33	Defaults to "origin"
build	06-Nov-2019 18:57:33	Target branch is "origin/master"
build	06-Nov-2019 18:57:33	Checking for changes against "origin/master"
build	06-Nov-2019 18:57:33	fetching latest from remotes
error	06-Nov-2019 18:57:33	Cannot fetch remote: origin
error	06-Nov-2019 18:57:33	(node:8896) UnhandledPromiseRejectionWarning: Error: Cannot fetch
error	06-Nov-2019 18:57:33	    at Object.fetchRemote (/home/bamboo/bamboo-agent-home/xml-data/build-dir/[REDACTED]/node_modules/beachball/lib/git.js:81:15)
error	06-Nov-2019 18:57:33	    at Object.getChangedPackages (/home/bamboo/bamboo-agent-home/xml-data/build-dir/[REDACTED]/node_modules/beachball/lib/getChangedPackages.js:49:15)
error	06-Nov-2019 18:57:33	    at Object.isChangeFileNeeded (/home/bamboo/bamboo-agent-home/xml-data/build-dir/[REDACTED]/node_modules/beachball/lib/validation.js:9:50)
error	06-Nov-2019 18:57:33	    at /home/bamboo/bamboo-agent-home/xml-data/build-dir/[REDACTED]/node_modules/beachball/lib/cli.js:70:41
error	06-Nov-2019 18:57:33	    at Object.<anonymous> (/home/bamboo/bamboo-agent-home/xml-data/build-dir/[REDACTED]/node_modules/beachball/lib/cli.js:107:3)
error	06-Nov-2019 18:57:33	    at Module._compile (internal/modules/cjs/loader.js:689:30)
error	06-Nov-2019 18:57:33	    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
error	06-Nov-2019 18:57:33	    at Module.load (internal/modules/cjs/loader.js:599:32)
error	06-Nov-2019 18:57:33	    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
error	06-Nov-2019 18:57:33	    at Function.Module._load (internal/modules/cjs/loader.js:530:3)
error	06-Nov-2019 18:57:33	(node:8896) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
error	06-Nov-2019 18:57:33	(node:8896) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
simple	06-Nov-2019 18:57:33	Finished task 'Check for changes' with result: Success
simple	06-Nov-2019 18:57:33	Starting task 'Publish changes' of type 'com.atlassian.bamboo.plugins.bamboo-nodejs-plugin:task.builder.npm'

From feedback we can do better with the dependent package changes.

1. prompt for dependent bump change type

At the time of calling beachball change, we should prompt user how they would like the dependent packages are going to be bumped. This will default to "patch."

This prompt should respect disabled change types (see also (2))

2. respect disabled change types

We need to add validation during bump phase to check if a version change type is disallowed and fail the bump before files are written on disk.

Sample prompt

beachball change

Change detected in A
Description of the change:
commit msg 1
commit msg 2
commit msg 3

Change type:
patch
minor
major

At this time, these packages depend on A:
B
C
D
E

These should be bumped as:
> patch (default)
minor
major

beachball change -m "same message across all package changes"

And interactively, will ask 1 more question about whether all the package changes should get the same message

When publishing fails partway through in a monorepo, the error message is not very clear/correct.

For example, suppose we're publishing a, b, and c. a succeeds, b fails, and c is never attempted. The output is like this:

Something went wrong with the publish! Manually update these package and versions:
- [email protected]
- [email protected]
- [email protected]

It would be better if it was more like this:

Something went wrong with the publish! Manually update these packages and versions:
- [email protected]
- [email protected]
These packages were successfully published:
- [email protected]

Currently we don't have a good UI for showing what is getting bumped and why and maybe how... We have a lot of that info inside the bumpInfo object now because we now bump virtually before things are written to disk. We should be able to summarize for users.

lerna publish has a mode to update all at the same time:

lerna publish minor
lerna publish major
lerna publish patch

This also resembles the npm version strategy. We should add this ability to bump as well as publish commands. We have to figure out what the best way it is to get the version as well as how to allow for an option for subset of packages to be versioned with different version policy (a la rush publish)

Currently we have user email for each change:

We should change to @githubusername (e.g. @xugao) Instead of email:

The check command says to run beachball. I'd like to be able to customize it to say run yarn checkchange.

Maybe provide a commandline arg to provide a custom message if beachball check fails? (Or at least the part about how to fix it?)

Also this error message is often going to be the first time people find out about change files. -- maybe we should provide a link to beachball's io page, or a more detailed description of what a change file is, and why its failing their CI?

@jdhuntington added the scenario testing infra for us doing change check, creating change files, etc. There are some fixtures about publish scenario tests that we need:

1. local npm registry via https://verdaccio.org/.
2. BARE git repo (git init --bare) so that git push would work

We should assert aspects about a package after package has been published to a verdaccio via npm pack

Should we have a .beachballignore file or maybe an entry in the package.json like we already have?

FUI repo recently had a failed publish which kind of looks like it was because the npm publish log output was too long...?

In the partial log output below, note how it cuts off in the middle of this last filename line:
2020-03-26T12:28:05.0490036Z npm notice 1.2kB dist/charting/dist-storybook/react-syntax-highlighter_lnpm ERR! code ELIFECYCLE

2020-03-26T12:24:56.2663729Z Publishing - @fluentui/[email protected]
2020-03-26T12:24:56.2665063Z publish command: publish --registry https://registry.npmjs.org/ --tag latest --//registry.npmjs.org/:_authToken=*** --access public
2020-03-26T12:25:07.8710529Z Published!
2020-03-26T12:25:07.8712204Z Publishing - @fluentui/[email protected]
2020-03-26T12:25:07.8719336Z publish command: publish --registry https://registry.npmjs.org/ --tag latest --//registry.npmjs.org/:_authToken=*** --access public
2020-03-26T12:25:13.2018152Z Published!
2020-03-26T12:25:13.2020090Z Publishing - [email protected]
2020-03-26T12:25:13.2021184Z publish command: publish --registry https://registry.npmjs.org/ --tag latest --//registry.npmjs.org/:_authToken=***
2020-03-26T12:25:42.6925191Z Published!
2020-03-26T12:25:42.6929325Z Publishing - @fluentui/[email protected]
2020-03-26T12:25:42.6930701Z publish command: publish --registry https://registry.npmjs.org/ --tag latest --//registry.npmjs.org/:_authToken=*** --access public
2020-03-26T12:25:51.4075014Z Published!
2020-03-26T12:25:51.4077026Z Publishing - @uifabric/[email protected]
2020-03-26T12:25:51.4085556Z publish command: publish --registry https://registry.npmjs.org/ --tag latest --//registry.npmjs.org/:_authToken=*** --access public
2020-03-26T12:26:00.2080418Z Published!
2020-03-26T12:26:00.2082353Z Publishing - @uifabric/[email protected]
2020-03-26T12:26:00.2084095Z publish command: publish --registry https://registry.npmjs.org/ --tag latest --//registry.npmjs.org/:_authToken=*** --access public
2020-03-26T12:26:19.6458975Z Published!
2020-03-26T12:26:19.6460648Z Publishing - @uifabric/[email protected]
2020-03-26T12:26:19.6462066Z publish command: publish --registry https://registry.npmjs.org/ --tag latest --//registry.npmjs.org/:_authToken=*** --access public
2020-03-26T12:28:04.8720379Z Something went wrong with the publish! Manually update these package and versions:
2020-03-26T12:28:04.8723443Z - @fluentui/[email protected]
2020-03-26T12:28:04.8724293Z - @fluentui/[email protected]
2020-03-26T12:28:04.8725107Z - [email protected]
2020-03-26T12:28:04.8725729Z - @fluentui/[email protected]
2020-03-26T12:28:04.8726356Z - @uifabric/[email protected]
2020-03-26T12:28:04.8736267Z - @uifabric/[email protected]
2020-03-26T12:28:04.8737520Z - @uifabric/[email protected]
2020-03-26T12:28:04.8738373Z - @uifabric/[email protected]
......
2020-03-26T12:28:04.8754293Z npm notice 
2020-03-26T12:28:04.8755005Z npm notice 📦  @uifabric/[email protected]
2020-03-26T12:28:04.8755811Z npm notice === Tarball Contents === 
2020-03-26T12:28:04.8756841Z npm notice 463.4kB dist/fabric-website-resources/dist/css/fabric-11.0.0.scoped.css                                                                               
2020-03-26T12:28:04.8758453Z npm notice 463.4kB dist/office-ui-fabric-react/css/fabric-11.0.0.scoped.css                                                                                      
...... many files later .......                       
2020-03-26T12:28:05.0489196Z npm notice 2.2kB   dist/todo-app/dist-storybook/react-syntax-highlighter_languages_highlight_makefile.a2aa69fba30a18ba7bae.bundle.js                             
2020-03-26T12:28:05.0490036Z npm notice 1.2kB   dist/charting/dist-storybook/react-syntax-highlighter_lnpm ERR! code ELIFECYCLE
2020-03-26T12:28:05.0490447Z npm ERR! errno 1
2020-03-26T12:28:05.0491397Z npm ERR! [email protected] publish:beachball: `beachball publish --scope "!packages/fluentui/*" "-b" "origin/master" "-n" "***" "--access" "public" "-y"`
2020-03-26T12:28:05.0491999Z npm ERR! Exit status 1
2020-03-26T12:28:05.0492257Z npm ERR! 
2020-03-26T12:28:05.0492839Z npm ERR! Failed at the [email protected] publish:beachball script.

[1:08:51 PM] ■ "F:\Repos\react-native-windows\node_modules\eslint\bin\eslint.js" "." --ext ".js,.jsx,.ts,.tsx" --config "F:\Repos\react-native-windows\vnext.eslintrc.js" --ignore-path "F:\Repos\react-native-windows\vnext.eslintignore" --color

F:\Repos\react-native-windows\vnext\src\RNTester\AlertExample.windows.tsx 16:9 warning Unexpected alias '_this' for 'this' consistent-this 41:9 warning Unexpected alias '_this' for 'this' consistent-this 62:9 warning Unexpected alias '_this' for 'this' consistent-this
✖ 3 problems (0 errors, 3 warnings)
[1:09:16 PM] ■ finished 'eslint' in 24.41s

The eslint task was marked as success even though there were warnings.

change and check do not operate correctly when the branch is in an unmergeable state. we need to give warning about this.

Currently in change file, we collect:

1. Change type: patch, minor, major, none
2. comment: author to describe the change

We should support a way for author to future group change into a particular genre (tag), they can be:

• Breaking change
• Bug fix
• Feature
• Documentation
• Performance

Provide a way for beachball user to define freely how to render CHANGELOG.md
Currently beachball renders as:

The value for "file" in the Beachball change file schema currently contains the full path on disk e.g. https://github.com/OfficeDev/office-ui-fabric-react/pull/10941/files#diff-795147d4185c2ea960552d4a0378b629R8

This exposes internals of the Users machine and is a privacy concern. Ideally, beachball only emits the relative file path.

Note that this can also lead to undesirable language being generated by Beachball and subsequently committed to a repository.

I'm willing to fix assuming the full path is not intentional i.e. part of Beachball's design.

#22 added a config option that needs to be documented.

The azure dev ops npmAuthenticate task edits the local .npmrc file.

In order to use this with beachball publish, beachball needs to be able to run publish with local edits in the repo. Currently it looks like beachball reverts any local edits before publish.

We may also need the capability of editing files during publish if we want to enable typescript workspaces where we change to a .js entry file during publish, but have .ts internal to the repo? (I could be wrong about that).

It could also allow apps to run custom prepare publish scripts as part of their CI pipelines, to possibly set version numbers in source code before publishing.

Note, with the npmAuthenticate issue at least, we'd want to make sure that beachball publish doesn't actually commit that change to the repo when commit its other changes.

When a package is marked private, we skip the publish and the change file deletion step but we still incorrectly bump all the versions of dependent packages. This leads to infinite version bumping as fast as the CI loop can run because the change file for this package will perpetually exist and not get deleted.

https://office.visualstudio.com/OC/_git/office-bohemia/commit/651fd9496aa021bb849be22bd89c6b64a75a7c72

Now that we have a better knowledge of upstreams, we should only fetch the appropriate upstream (the target upstream remote). We need to display the error messages from that git.fetchAll() call as well.

Feature

Add PR link for each change such as:

Implementation

1. Stop collecting commit hash at change file creation time: done
   

   beachball/packages/beachball/src/changefile/promptForChange.ts

   Line 86 in 01f3bae

   commit: getCurrentHash(cwd) || 'hash not available',

2. Instead, retrieve the commit hash when generating the PR link for being included in the CHANGELOG

3. get PR link using commit hash and github api. reference the code here.

There's a chance that the merge that happens will merge in new change files and the npm publish and the bumping would have been different. We need to preserve the change files that got created in between.

React Native Windows is wanting to move to a model where we have tags for "master", "latest", and "preview". Apart from these tags, we often backport fixes into previous versions. Ideally we could avoid tagging these older releases, but currently publishing forces a tag, defaulting to "latest" if none is specified. To avoid polluting latest with backports, we've been explicitly changing old branches to publish a tag based on their branch name. This ends up leading to a bunch of unnecessary NPM tags causing clutter.

Ideally, we should be able to ask Beachball while publishing not to add a dist-tag to prevent tagging our backport fixes that we still want to publish.

When a missing package is detected from registry, beachball should default to publishing the new package.

Currently, as part of a dependent package bump, nothing gets written as part of a changelog. This means that no changelog.json entry would be generated and therefore, subsequent tooling do not have a chance to look for why the version is bumped.

We should inject change entries into the "changes" key inside BumpInfo after the dependent and grouped bumping to ensure all changes are recorded.

Some automated tasks might need to generate change files.

They could manually generate a change file matching the current format, which could cause issues with future beachball changes... or they could call:

beachball change --package my-package --type patch --message "Updating version of my-package-dep to 3.1.2"

or something like that.

If the package isn't already a prerelease package, cli should query what the prerelease tag should be.

Otherwise, it should just reuse the existing prerelease tag.

Feature

Currently each changelog is generated per single package, we should provide way for user to

1. Define a group of packages and generate a single, aggregated changelog for this group of packages.
2. User can define what directory location to put the aggregated ChangeLog at.

Implementation

(more investigation still needed)
Consider expanding the version group feature to support not only grouping the version bump, but also support grouping the changelog

Right now the list of change types on the command line is a little hard to parse visually:

I think it would help to make the change type names ("Patch" etc) bold and/or change the spacing or punctuation.

I don't have a good way to simulate adding bold, but here are some ideas for changes to spacing/punctuation:

❯  Patch - bug fixes; no backwards incompatible changes.
   Minor - small feature; backwards compatible changes.
   None  - this change does not affect the published package in any way.
   Major - major feature; breaking changes.


❯  Patch  - bug fixes; no backwards incompatible changes.
   Minor  - small feature; backwards compatible changes.
   None   - this change does not affect the published package in any way.
   Major  - major feature; breaking changes.


❯  Patch: bug fixes; no backwards incompatible changes.
   Minor: small feature; backwards compatible changes.
   None:  this change does not affect the published package in any way.
   Major: major feature; breaking changes.


❯  Patch:  bug fixes; no backwards incompatible changes.
   Minor:  small feature; backwards compatible changes.
   None:   this change does not affect the published package in any way.
   Major:  major feature; breaking changes.


❯  Patch    bug fixes; no backwards incompatible changes.
   Minor    small feature; backwards compatible changes.
   None     this change does not affect the published package in any way.
   Major    major feature; breaking changes.


❯  Patch  (bug fixes; no backwards incompatible changes.)
   Minor  (small feature; backwards compatible changes.)
   None   (this change does not affect the published package in any way.)
   Major  (major feature; breaking changes.)

If the user hasn't configured a git remote for the main/upstream repo, beachball should detect this and show an error instructing how to add a remote. Something like:

Could not find a git remote for this package's [main/upstream/???] repo. Run the
following command to add the remote, then try generating change files again:

    git remote add upstream <url here>

(I'm not sure what the standard term is for an original repo when you're working from a fork, and of course the other wording is negotiable too.)

Inspired by discussion starting at microsoft/fluentui#10294 (comment) trying to debug why too many change files were generated for a PR.

If a user wishes to generate a change file and one already exists, include that info in the "no change file needed" message for clarity.