virtuslab / bazel-steward Goto Github PK

Refactor the cli interface to use --kebab-case for flags instead of current --camelCase

Commit message for dependency update should be configurable with a template.
Default template would be "Update {dependency} to {versionTo}".
We may consider what variables for template will be provided. We could allow specifying group/artifact for maven deps separately for example.

Regardless of PR grouping, we will put each update in a separate commit so the commit will always be about single version update.

Users should be able to pin specific part of the version for given library (or group of libraries).
For example pin scala library at "2.12.x" so that our update logic only considers version like 2.12.17 but not 2.13.0 nor 3.0.0 as possible to update.
It should be possible to pin major, major + minor and major + minor + patch

According to Maven specification during comparing two version there should be phase of trimming so called "null" values ie.: 0, "", "final", "ga". Currently this isn't supported in bazel-steward and it might help with comparing process.
Sources:
Maven Specification

Bazel-steward should be able to open a PR for each outdated dependency. If PR already exists it shouldn't be touched, nor should we create duplicates. We can now or in the future, update existing PRs if they have merge conflicts.

We probably should start with attaching the github action (#10) to bazel-steward itself to keep our dependencies up to date. If this works, we will reach out to the bazel-bsp team and set it up there.

Users may want to run specific commands after bumping versions. It might be to modify the files in the repository or other side effect. Users may configure to run commands before committing but after bumping so that commit can contain changes, additionally we can offer running commands and committing changes separately with a custom message.

Each hook may be limited to a specific set of dependencies.

A default use case for this could be running the buildifier after changes.

Implement existing stub class MavenDependencyExtractor.
https://github.com/bazelbuild/rules_jvm_external

Suggested implementation: use bazel query or build the project using an aspect.

Setup e2e tests:
minimal bazel repository with java and scala and some 3rd party dependencies defined in standard ways

By default we should not suggest updating stable release to prerelease versions. Users should be able to enable this feature for all dependencies, or for some specific group/platform or even a single dependency.

This issue is about a corner case when someone has, say, metamonorepo, i.e. one git repository contains multiple Bazel projects. Each project would be treated separately, with separate config.

Users should be able to configure relative paths within repositories where workspaces are located. By default it would be single element list with root directory ["."].

There are multiple ways to approach this.

• We can combine it with approach from #39, i.e. each "subworkspace" would be specified as a workspace, either as a path to the workspace in the local file system or git url + relative path using some syntax.
• We can auto detect bazel workspaces in given repository and only run for those who have the bazel-steward config file.

User should be able to specify list of allowed dependencies that should be updated (to opt in to bazel-steward for limited set of deps) and/or a list of ignored dependencies.
As in all cases when specifying deps it should be possible to use some regex/wildcard to specify groups of libraries in a general way. Also it should be "scopable" to given platform like maven or python.

We are using SemVer for versioning now. In practice there are multiple versioning schemas. See https://docs.renovatebot.com/modules/versioning/ for examples.
Bazel Steward should understand at most popular schemas. SemVer may be too strict. We should research this topic, find some examples and corner cases, and implement them as tests.
For the time being we should be good with loose-semver (handle versions that do not follow semver strictly but are similar).

1. Learn about versioning schemes
2. Implement tests based on possible use cases
3. Figure out best logic to cover most cases and implement it

Background: Bazel Steward should now what kind of dependency management rules are in given workspace and only run the extraction logic if it makes sense.

As the first step, we should check if jvm_rules_external are setup in bazel.

Suggested implementation: most likely it can be verified by existence of some specific target in the project, or by text search of *.bzl and WORKSPACE files.

Tests should be able to setup an example bazel workspace and run the basic workflow: detect some outdated dependencies and apply it to the workspace/build files. Test should check if files were modified correctly.
We can consider mocking logic for extracting available versions and testing is separately - this would make the e2e test easier to write, but it is not a requirement.

It may highly depend on packaging ecosystem, but if we can get changelog for a library, it is good to include all changes that user applies to their repository when updating the dependency from version X to Y.

Currently we are just running bazel steward on an existing cloned repository. We could wrap this logic and execute it multiple times on different repositories (first cloning them).

This is useful for companies that just would like to setup 1 job for bazel steward, configure which repositories should it run on and call it done. Without this feature, it would be required to setup a separate job for each project.

Repositories list/file could be a parameter to the Bazel Steward binary. We could accept mixed list of directories and clone urls and run the logic for each of them. By default we would pass as a single relative path (".").

MavenDependencyExtractor extracts libraries used currently in the project. Libraries may be fetched from custom repositories, not only maven central. Users may use custom internal resolvers that require credentials. We should be able to extract all this. The repsitory information needs to be passed to Coursier so that it looks for updates in the same repositories, as available versions/libraries may differ.

New way of managing dependencies on rules (replacement for #66). See: https://bazel.build/build/bzlmod
It is not yet widely adapted, it is a new feature of bazel, but it will be more widely used eventually.

Implement MavenRepository.findVersions. We likely want to use coursier for this purpose. Coursier is with scala, so we may want to wrap it's API into something more Kotlin friendly. Worst case it may require to have a small target written in Scala.

By default we are looking for versions to replace in all WORKSPACE, BUILD, .bzl and .bazel files. Users should be able to configure acceptable paths in the form of globs or regexes, with an option to exclude. The configuration could be global, but also scoped to a specific version system (like look for jvm_rules_external dependencies in this file, but for scala version in that file, and for bazel rules version in different file).

Using script lint_kotlin.sh causes damages to project configuration. After using it project misses few dependencies and is uncompilable because of of it.
The only workaround is using from terminal command bazel test //... and opening log file from the output and fixing all lint errors by hand.

Bazel Steward by default creates PRs against default branch (main/master). Users should be able to pick different branch than default, for example, default branch is main, but bazel-steward should work on dev branch. Additionally users may want to run it for multiple branches like branch 212.x and 213.x etc.

Users should be able to decide how often bazel steward runs and opens new Pull Requests.
We should make sure that each deployment strategy supports this. In case of github actions or most other CI jobs, it is just possible to specify cron expression to define frequency.

Scala Steward is able to update different artifacts with different ferequency. I am not sure if we need this though. It requires maintaining state between runs.

Sometimes a package is moved, for example to a different organization (groupId) on maven, or is renamed in a different way. Similar situations apply for npm and other ecosystems. Both scala steward and renovate have a list of such dependencies and on which version it changed. Bazel Steward should update the package identifier along with the version.

Update versions and checksums for bazel rules like jvm_rules_external, rules_scala, rules_kotlin, etc.

Depending on users needs, Bazel Steward should support more platforms than just the official instance of Github. We should support different platforms, including internal/enterprise version of services with a good documentation regarding configuration.

Example platforms:

• Github (Enterprise)
• Gitlab (Enterprise)
• Bitbucket (Cloud/Server)
• Azure Dev Ops
• AWS CodeCommit
• Gitea

Bazel Steward should be easily extendable to add different platforms as plugins.

• Auto bump existing PR to resolve conflicts unless it was edited by user
• Closing PR by users means that version should be ignored (we should not open PR again)
• If newer version appeared, close the old PR and open a new one

We have data classes for config that match json schema that we use for validation. We could consider generating these classes automatically from json schema, for example using this library: https://github.com/pwall567/json-kotlin-schema

Research public github repositories that use bazel.
Write down:

• reposistory name/url
• used languages and rules
• if it uses any dependency update system
  It should guide us in further implementation

Users should be able to configure:

• a maximum number of PRs that Bazel Steward can create or update each time it runs. It can be useful if CI runs are costly and the runs should be distributed in time.
• maximum number of open bazel steward PRs in repo. It can be useful for repositories that have 120 outdated versions and prefer to update them in chunks

Config:

pull-requests:
  - limits:
      open: 10
      updates-per-run: 5

Make sure this is easy to do and well explained.

Similar issue to #44 but for PR titles. It also relates to grouping (#50), as grouped PRs will have custom titles, similarly default 1 change PR can have their title template.

By default we create and maintain 1 PR per version update.
We should allow users to customize this, for example grouping updates of specific dependencies together, minor/patch updates in one PR, etc.
Scala Steward has it well made.

We can copy the logic of Scala Steward, we may also provide different configurable strategies, like always bumping to the latest version. It should be configurable globally or per dependency.

This issue is about fine grained control over updating files with versions. If for whatever reason it is undesired that bazel steward edits some version in some file (heuristics is wrong or anything), it should be possible to exclude some part of file from the detection/replacement mechanism.
We can use similar mechanism as linters/formatters do i.e. use comments like

block:

# bazel-steward off
MY_VERSION = "1.0.0"
MY_OTHER_VERSION = "1.0.2"
# bazel-steward on

inline

MY_VERSION = "1.0.3" # bazel-steward off

This is an extension of #61

Users should be able to opt in for automatic PR merging if all checks pass. Scala Steward uses mergify. We may use this or different approach. Most likely it is best to integrate somehow with each platform that we support.

Apply scalafix migrations on specific library updates.

Some libraries change their public api and require code changes on update. Some of these libraries provide special scalafix rules that can automatically rewrite the code to work with newer version. Scala Steward has a list of libraries, versions and scalafix rules that should be applied. We could integrate it with bazel.
It is not a trivial task. sbt has a plugin for scalafix, while bazel does not, so we would need to build something.

Bazel Steward should be able to detect newer version of bazel and suggest bumping it.

We may get information about vulnerabilities of libraries and do not suggest upgrades if version is vulnerable, or suggest an update to the newest secure version.

Additionally we can mention in the PR content that given version fixes some vunerability.

Github action should run bazel-steward on the repository and open PRs when run (depends on #9).
We should document how to add bazel-steward to a repo - put an example yaml file to configure repo with bazel steward run periodically (cron). This would be our first integration.

We need to somehow distribute bazel-steward to put it in a github action. We could use maven central and coursier. Initially it may be easier to setup github releases though.

Action could pick the latest version and allow overriding if necessary.

Depending on platform, user should be able to specify for example which reviwers, labels and milestones to assign to PRs created by Bazel Steward.

Bazel Steward should detect some config file like .bazel-steward.conf in the repository root and load it as a config and make it available to the whole application.
Format is to be decided. We can pick probably between yaml and hocon.

Explain what are the requirements and how to import project in the IDE.

Bazel Steward should understand a scheme configurable by regex (per each dependency). The purpose of this task is to allow overriding the versioning logic with a custom regex

Write new or modify existing e2e test:

1. Workspace file should depend on some library that only exists outside maven central
2. Use outdated library version
3. Assert that update was found for this version