Giter Club home page Giter Club logo

jquery-release's Introduction

jQuery Project Release Automation

This script automates releases for all jQuery projects. It is designed to create consistency between projects and reduce the burden of maintaining individual release scripts.

Creating a Release

Creating a release is as simple as cloning this repository and telling the script which project to use. In order to ensure a clean and proper release is created, you should always start from a new clone of this repository.

git clone [email protected]:jquery/jquery-release.git
cd jquery-release
npm install
node release.js --remote=jquery/<project-name>

Testing the Release Script

You can do a test run of the release script by using a different remote repository. It is recommended to perform tests from a fresh clone of the project being released. The script is smart enough to detect if you're using an official repository and adjust which actions are taken so that undesired actions, such as publishing to npm, don't occur for test runs.

You can also explicitly specify --dry-run to skip actions that affect external state.

When working on features of this script, adapt the following to simplify testing a bit, replacing the paths for project and cdn:

#!/bin/sh -e
# uncomment next line to debug this script
# set -x
project=/path/to/fake-project
cdn=/path/to/fake-cdn

cd $project
git checkout main
set +e
git tag -d 0.0.1
set -e
git reset --hard safe
git checkout asdf
cd -

cd $cdn
git push -f
cd -

npm run clean
env TEST_REMOTE=$cdn node release.js --dry-run --remote=$project

Save as test-release.sh in the checkout of this repo, make it executable with chmod +x test-release.sh, then run with ./test-release.sh.

If you have access to the private repositories fake-project and fake-cdn, you can use them by dropping the --dry-run argument and updating the TEST_REMOTE environment variable to "[email protected]:jquery/fake-cdn.git".

Full Usage Options

See the usage documentation for the full set of options. You can also run the script with no parameters to see the usage.

Creating a Project-Specific Release Script

This script only performs the set of common functionality across all projects. Each project may have additional functionality. Any project-specific configuration and functionality must be defined in the build/release.js file inside the project's repository.

Here's a minimal example:

module.exports = function( Release ) {

Release.define({
	issueTracker: "trac",
	contributorReportId: 37,
	changelogShell: function() {
		return "# Amazing Changelog for v" + Release.newVersion + "\n";
	}
});

};

Required/Recommended Configuration

checkRepoState( [ callback ] )

Performs any project-specific checks to ensure the repository is in a good state to be released. For example, there is a built-in check to ensure that AUTHORS.txt is up-to-date.

This method has no return value. If a project-specific check fails, the script should use Release.abort() to prevent the release from continuing.

This method may be synchronous or asynchronous depending on the presence of callback. If present, the callback must be invoked.

generateArtifacts( callback )

Generates any release artifacts that should be included in the release. The callback must be invoked with an array of files that should be committed before creating the tag.

changelogShell()

Defines the shell for the changelog. The changelog is created by concatenating the shell, the commit log, and the issue list.

tracMilestone()

If using Trac, return a different milestone to be used in the queries to generate a changelog and list of contributors. Defaults to newVersion.

npmTags()

A function that returns an array of tags to apply to the npm release. Every release must contain at least one tag.

issueTracker

Which type of issue tracker is being used for the project. Must be either "trac" or "github".

contributorReportId

If using Trac, this defines which report will produce a list of contributors for a specific release.

See docs/trac-contributors.sql for the SQL necessary to create the Trac report.

exports.dependencies

Note: This is a property on the exports object in build/release.js.

An array of release-specific dependencies. Dependencies can be listed here instead of in devDependencies in package.json so that contributors don't need to install dependencies which are only necessary for the release.

Other Methods

define( props )

Defines new properties and methods to add to the Release object.

abort( msg [, error ] )

Aborts the release and prints the message. If an error object is provided, it is used for the stack trace, otherwise the current call stack is used.

exec( command, options )

Executes the given command. You can pass { silent: true } to suppress output on the command line.

Returns the output.

git( command, errorMessage )

Executes the given git command. If the command fails, the release will be aborted and errorMessage will be displayed.

gitLog( format )

Gets a git log using the specified format. If the log fails, the release will be aborted.

Returns an array of commits.

prompt( callback )

Prompts the user for input.

Passes the input to callback.

confirm( callback )

Prompts the user to confirm they want to continue with the release script. If the user decides not to continue, the release will be aborted and callback won't be invoked.

confirmReview( callback )

Prompts the user to review the output and confirm they want to continue with the release script. If the user decides not to continue, the release will be aborted and callback won't be invoked.

trac( path )

Gets the results of a Trac query, with tab-delimited results.

Returns the tab-delimited string.

readPackage()

Gets the contents of package.json as an object.

writePackage( package )

Saves package to package.json, preserving indentation style.

walk( methods, done )

Executes the array of methods (minimum one element) step by step. For any given method, if that method accepts arguments (method.length > 0), it will pass a callback that the method needs to execute when done, making the method call async. Otherwise the method is assumed to be sync and the next method runs immediately.

Once all methods are executed, the done callback is executed.

dist( callback )

This function is available in case the project requires more distribution than what is provided. It is called after building, but before publishing to npm.

Other Properties

isTest

Whether this is a test release. Test releases don't publish to npm and use the fake-cdn project instead of publishing to the real CDN.

project

The name of the project being released.

remote

The location of the remote repository.

preRelease

The version number for a pre-release version, or false for stable releases.

dir.base

The main directory used for the release script.

dir.repo

The directory for the local repository.

newVersion

The version being released.

prevVersion

The previous release version (used for determining what changed).

nextVersion

The version that will be set in package.json after the release.

cdnPublish

Which directory contains files to publish to the jQuery CDN. Set to false to prevent publishing to the jQuery CDN. Defaults to "dist/cdn".

npmPublish

Set to true to publish a release via npm. Defaults to false.

tagTime

Timestamp for the release tag.

branch

Which branch the release is being generated from (defaults to main).

jquery-release's People

Contributors

agcolom avatar dependabot[bot] avatar gibson042 avatar gseguin avatar jzaefferer avatar mgol avatar scottgonzalez avatar staabm avatar timmywil avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

jquery-release's Issues

changelog task failing on exceeded API rate

Trying to release QUnit 1.19.0 I'm getting a exceeded API rate error on the changelog task.

Here is a gist with the full log: https://gist.github.com/leobalter/2036c590d050ed839c93#file-gistfile1-txt-L209-L221

It allows new and clean retries, but they all fail at this point.

Also, the fail says:

But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.

I wasn't able to find anything on the docs.

Allow project-specific npm dependencies

Right now projects need to include any additional dependencies in devDependencies in order to make them available inside the release script. For some projects, these dependencies may be quite large and having them around for normal development is unnecessary. It'd be better to only have these dependencies installed when the release script runs.

Limit changelog for pre-releases

Would be nice if the changelog only included commits from one pre-release to the next, instead of all from the previous stable release to the current pre-release.

Create a test project for testing jquery-release

Make it small to speed up testing jquery-release, but include enough for the testing to be relevant. For example, needs to support grunt authors.

We can try to commit node_modules to speed up the npm install step. Should include one release-specific dependency that alyways install, but that should be tiny without transient dependencies.

Since grunt authors needs grunt, we have a rather big dependency. Should see if there's some way around that.

We can use this to also test error conditions, like an outdated AUTHORS.txt. We have to figure out to make use of that and implement it. One idea is to do branches for various error conditions, so invalid-authors would have an outdated AUTHORS.txt and nothing else. The change would be tiny (shift instead of pop on the read authors), so a rebase on master should always be trivial.

To automate testing of the release script, we could have a separate script that does something like this:

#!/bin/sh -ex
node release.js --remote=../fake-project --branch=master
# don't fail on errors anymore, capture them instead
set +x
node release.js --remote=../fake-project --branch=invalid-authors
## TODO assert that authors were invalid
node release.js --remote=../fake-project --branch=invalid-npm-owner
## TODO assert that npm owner is invalid

For that to be actually automatic, we'd need a way to skip the confirms.

Also need to remove tags and the cdn commit, something something:

../fake-project/git tag -d 0.0.1
../fake-project/git reset --hard HEAD^
../fake-cdn/git reset --hard HEAD^
../fake-cdn/git push -f

These currently assumes that everything worked, but that wouldn't always be the case.

To assert the output, redirect stdout (and stderr) to a file and grep them?

Set CWD in each task

Right now the tasks assume they know what directory we're in. To safe guard against new/changed tasks, we should explicitly set the CWD at the beginning of each task.

Make CDN publishing optional

I would like to use this for my own project, https://github.com/jzaefferer/jquery-validation

Currently there is no way to disable the CDN upload, so the script fails there. I though I could just override the relevant methods, but since those get passed by reference (aka the references are copied) to Release._walk, my overrides come to late and don't change anything. For _checkAuthorsTxt this happens to work since _checkRepoState calls it.

A flag ala npmPublish, here cdnPublish, would help. If that's acceptable, I'd put that together and test it with my plugin release script.

Automating releases with GitHub workflows

Hi! I've noticed this repo defines an automated script to release jQuery org projects and this script allows publishing the project/package to npm. npm has recently enabled publishing packages with provenance and that could be a nice addition to jQuery org releases.

The provenance holds verifiable information about the software artifacts describing where, when and how it was produced. It ensures that the artifacts users download from npm were uploaded from a reliable source.

If you are willing to make a few changes to the jQuery org release process, we can publish the npm packages with provenance using GitHub workflows! If you agree, I can support you by opening PRs.

Additional context

I'm Gabriela and I work on behalf of Google and the OpenSSF suggesting supply-chain security changes :)

Improve cdn dir handling

QUnit CDN files have naming very similar to jQuery itself, e.g. /cdn/qunit/qunit-$version.js. Currently the files would end up in something like /cdn/qunit/$version/.

Can we make the CDN path configurable? Or should I just update lib/cdn.js to hardcode a check for Release.project === "qunit"?

Tests?

I know that those would be tricky to write, but no tests at all? Seems weird and stupid. Preferably ran in all supported node versions on Travis CI.

NPM: Always set the beta tag

Stable releases should get tagged as beta as well so that anyone trying to use the closest to bleeding edge that we provide gets stable versions if they're newer than the most recent pre-release.

Add Release.grunt()

We should have abstractions around our common tooling. A Grunt abstraction will also allow us to stop handling the Windows difference directly in each project.

Replace Release.project

In cdn, use the name from package.json. In changelog, use the repo URL. Check if there's any project-specific scripts that use Release.project (that's very unlikely).

Can't check credentials for unpublished projects

When a project was never published on npm, we can't check for owners. This doesn't matter for our existing projects, since they all were published before. Its easy enough to skip by uncommenting the call to Release._checkNpmCredentials, but some day we should probably address this.

GitHub changelog generation breaks on pre releases

Since there's no milestone with a name like "3.0.0-alpha1", GitHub changelog generation breaks because it should be using "3.0.0" as the milestone. I'm not sure the best way to fix this. Perhaps we could just do something like...

var version = Release.preRelease ?
    /^\d+\.\d+\.\d+/.exec( Release.newVersion )[ 0 ] :
    Release.newVersion,

    milestone = milestones.filter(function( milestone ) {
        return milestone.title === version;
    })[ 0 ];

instead of using newVersion, so it lops off any version labels.

Speed up CDN publishing

Cloning the entire CDN repo takes a long time. We should at least allow the clone to stay on disk and pull updates, instead of forcing the __release/ dir to be deleted.

Quiet `npm install`

npm has a loglevel of warn that won't output all the http messages. I can be set with --loglevel warn, aka --quiet aka -q. I suggest we add -q to both npm install calls. It still outputs a summary of all the installed modules (and logs from install scripts), which is all we need. If something goes wrong, it'll output that as well.

create tag triggers commit hooks

as the createTag step triggers commit hooks and the used commit message just is a version-number

Release.exec( "git commit -m '" + Release.newVersion + "'",
            "Error committing release changes." );`

it violates the jquery-validate rules of every commits needs to indicate/prepare a component, the whole process fails with a error from the githook.

could we either make it possible to define the commit message or make the commands not trigger hooks (not sure this is possible in git)

//cc @jzaefferer

Should we have node_modules committed?

This apparently saves some dangerous/places a release could go wrong situation, but I think that it's going to create problems for cross platform if the deps ever end up with compiled gyp modules for instance.

It's generally not a good idea to have them committed to the best of my understanding. Thoughts?

_updateBranchVersion: Don't touch bower.json

We use package.json as our canonical source for the version property, so bower.json should only be modified in the tagged branch. Currently _updateBranchVersion will incorrectly update bower.json in master (or whatever the branch is).

I suggest to change _updateBranchVersion to call Release._versionJSON( "package.json", Release.nextVersion ); directly, instead of _setVersion as it does right now.

Verify npm credentials upfront

For projects that publish to npm, we should verify credential up front to make sure we don't have permission problems at the end.

npm owner ls <package> will list the owners and npm whoami will list the current user.

NPM: Full control over tags

jQuery 1.x shouldn't get set as latest even for stable releases. Similarly, 1.x pre-releases shouldn't get the beta tag.

Support minor releases

QUnit almost never has patch releases, so bumping bower.json and package.json to the next x.x+1.0pre before releases is something jquery-release could take over.

Major releases are relatively rare, so doing those manually is probably fine.

404-ing links in README

Seems like you've got some broken links in the readme:

You need local clones of fake-project and fake-cdn, then update both variables to point to those.

I can't seem to find the fake-project and fake-cdn projects (github is giving me a 404). Are those repos moved or do I need to use another repo?

Commit message for tag is invalid

For example, "Tagging the 1.11.0 release." is not a valid commit message, according to commitplease. Noticed this while starting the migration of jQuery UI to jquery-release.

Commitplease accepts commit messages that are just the new version number. We could use that, or come up with the "module" to put in front of the above commit message.

Make check for external dependencies extendable?

Core currently requires md5sum, UI requires unzip. Arguably those could all be replaced with node modules. Both UI and Mobile have their own implementation for creating md5 sums. Last I checked, there was no decent unzip implementation in node (archiver only does zipping; node's zlib wrapper only handles compression, not the actual archiving).

If we make the check extendable, we'd have move or duplicate it, since it can only run after _cloneRepo.

Error recovery

If a task errors out, we should log all the state so we can restart the release process. When restarting, it should be possible to pick which step to start from, but there should be a sane default if that's possible.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.