ballerina-platform / ballerina-release Goto Github PK

Description:
Along with this fix ballerina-platform/ballerina-distribution#2105 cache is generated from bala file in the distribution. Hence no need to archive cache and store in Github packages

Description:
Automated lang merge pr creates unnecessary commit messages.

I think this can be avoided if we force push a commit to automated/dependency_version_update instead of merging. We can also avoid merge failures. Others shouldn't commit to this branch

Description:
We have to add code owners for Ballerina in Action sample repositories in ballerina-guides organization.

Description:
Currently dependency versions have been centralized and accessed through gradle.properties file for most of the repos. But there are some instances where naming of the same dependency is different from one repo to another.

Description:
Currently update tool tests are running against live. This needs to be improved to run against both live and stage.

• Introduce 3 steps/workflows to run installer tests, update tool tests against preprod, update tool tests against live
• Currently these workflows are executing during PR, this needs to be changed to on workflow dispatch. so release manager could execute after creating installers, after publishing to stage and after publishing live on demand
• Enable sudo operations for stage with sudo BALLERINA_STAGING_UPDATE=true bal update

Description:
PR ballerina-platform/module-ballerina-docker#764 Add docker file to build devcontainer image. We need to add it to the release pipeline.

• Push devcontainer image for beta2 release manually
• Add devcontainer image build and push to release pipeline
• Currently Ballerina docker images are kept in https://github.com/ballerina-platform/module-ballerina-docker repo which is ballerina docker module repository. We should move it to a separate repo

Description:
We can add another column to showcase the latest timestamped version of the modules. However, this needs to be dynamic.

Shields has the functionality to display a dynamic badge, however since the Github API is authenticated, we must have an API in the middle for this scenario.

Description:
$title

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Description:

A summary email that has information about the lag status and the pending PR status of ballerina repositories can be sent daily to a desired mail group as an effort to make CI/CD process more efficient.
Ballerina repositories updated status can be viewed from the dashboard available here and the automated mail is expected to contain an overall summary of the important information shown there.

Description:
Need to add installer tests in Full Build Pipeline, Full Build Pipeline For Updated Stages, Daily Full Build (master branch) and Daily Full Build (patch branch).

Describe your problem(s)

Describe your solution(s)

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

Description:
Currently dependency graph is represented in a dot file. We need to create a GitHub pages website to show dependencies graph

Description:
$title

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Description:
This should be improved to send a notification to the google chat in case of timing out due to changing build times.

Description:
Recently, when I reached the Github Support for a different issue, they pointed out the following improvements we can do the workflows of our extensions,

1. Use the latest set up java action and checkout action step

For the first one, they're using setup-java@v1 I recommend moving over to @v2 + using the adopt distribution. It can used cached versions of java that are already pre-installed on the image so it should be faster + it will have better logging + eliminate any chance for a download error.

Recommended action step,

- name: Checkout Repository
  uses: actions/checkout@v2
- name: Set up JDK 11
  uses: actions/setup-java@v2
  with:
    distribution: 'adopt'
    java-version: '11'

2. Streamline dependency repositories configuration

I noticed that in your build.gradle file, you're using multiple maven elements.
https://github.com/ballerina-platform/ballerina-openapi/blob/master/build.gradle#L72-L92
It's actually possible (and preferable) to replace these with a single element for all repos. When downloading packages, the repo section is ignored. This means you can replace all of the maven elements with one like this:

Recommended,

        maven {
            url = 'https://maven.pkg.github.com/ballerina-platform/*'
            credentials {
                username System.getenv('packageUser')
                password System.getenv('packagePAT')
            }
        }

Description:
$subject

The reason for this is the workflow tries to read the code owners file in the "master" branch which is not the case for some of the modules. https://github.com/ballerina-platform/ballerina-release/blob/master/.github/workflows/notify-build-failures.yml#L34

This can be fixed by using GithubPy python library as seen in https://github.com/ballerina-platform/ballerina-release/blob/master/dependabot/update_dependency_graph.py#L89-L91

Description:

Currently we have tests in ballerina-distribution repository and then we have tests in Ballerina installer tests as well.
We assume these tests will assure the functionality.

Some of tests may not catch actual bugs due to various reasons like tests are using intermediate packs and can be exit code is wrong.

Therefore we need to conduct a Mutation testing effort by introducing few bugs purposely and assure how many of them identified by our current test suite.

Description:
Currently windows installer test workflow is disabled. This needs to be enabled back and should execute with each release

Description:
Currently when we need to skip tests in one of the full build pipelines, we have to do it by changing the code file. We don't need to change the code when we use a module ignore file.

Describe your problem(s)

Describe your solution(s)

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

Description:
We need to sync central live modules in dev and stage servers. Currently live server is updated by the release workflows of each modules.

To push same stdlibs in the stage and dev nodes we need to create a separate workflow. This can be triggered after a release(ballerina-distribution). iterate module list then clone each release tag/branch and build and push

Description:
$title

Related Issues:
ballerina-platform/ballerina-lang#27680

Description:
Currently all chat notifications are send to Ballerina main chat. However, since Observability Extensions are maintained by a different team, we need to send these notifications to the Observability Team Chat.

Repos:

• https://github.com/ballerina-platform/module-ballerinax-prometheus
• https://github.com/ballerina-platform/module-ballerinax-jaeger
• https://github.com/ballerina-platform/module-ballerinax-choreo

Description:
We need to push following extensions to central along with release action

• module-ballerina-docker
• module-ballerina-c2c
• module-ballerinax-aws.lambda
• module-ballerinax-azure.functions
• module-ballerinax-datamapper
  • Remove 2.0.0 version
• ballerina-openapi
  • Remove 2.0.0 version
• module-ballerinax-prometheus
• module-ballerinax-choreo
• module-ballerinax-jaeger

Description:
Currently there is a job to publish ballerina releases and similarly we need to have a separate job to publish update tool separately

Description:
Currently, this workflow only takes the repository name and sends a templated notification, This can be improved further by including the following optional inputs,

1. Build Name (Daily Build)
2. Build Run ID (This is to point to a specific build log)
3. Custom message to be appended to the notifications - This is highly useful when the builds are complex such as "ballerina-distribution"

Description:
$subject

Choreo service: This has to be a choreo service since this includes some private information,

import ballerina/log;
import ballerina/http;

type NotificationDetails record {|
    string ballerinaVersion;
    Module[] prChecksFailed;
    Module[] prMergeFailed;
    Module[] buildCheckFailed;
    Module[] buildVersionCannotBeIdentified;
|};

type Module record {|
    string name;
    string link;
|};

final string googleChatSpaceId = "";
final string googleChatSpaceKey = "";
final string googleChatSpaceToken = "";

final http:Client clientEndpoint = check new ("https://chat.googleapis.com/v1/spaces/" + googleChatSpaceId + "/messages");

service / on new http:Listener(8090) {

    @http:ResourceConfig {
        consumes: ["application/json"]
    }
    resource function post notification(@http:Payload NotificationDetails details) returns http:Ok|http:BadGateway {
        log:printInfo("Received Notification Request : " + details.toString());
        string chatMessage = "Dependency update to lang version *" + details.ballerinaVersion + "* has failed,\n\n";
        if details.prChecksFailed.length() != 0 {
            chatMessage += "Following modules' Automated Dependency Update PRs have failed checks,\n";
            chatMessage += getModuleList(<Module[]>details.prChecksFailed);
        }
        if details.prMergeFailed.length() != 0 {
            chatMessage += "Following modules' Automated Dependency Update PRs could not be merged,\n";
            chatMessage += getModuleList(<Module[]>details.prMergeFailed);
        }
        if details.buildCheckFailed.length() != 0 {
            chatMessage += "Following modules' Timestamped Build checks have failed,\n";
            chatMessage += getModuleList(<Module[]>details.buildCheckFailed);
        }
        if details.buildVersionCannotBeIdentified.length() != 0 {
            chatMessage += "Following modules' latest Timestamped Build Version cannot be identified,\n";
            chatMessage += getModuleList(<Module[]>details.buildVersionCannotBeIdentified);
        }
        chatMessage += "\nAfter following up on the above modules, retrigger the <https://github.com/ballerina-platform/ballerina-release/actions/workflows/update_dependency_version.yml" + 
                        "|Dependency Update Workflow>";

        log:printDebug(chatMessage);
        http:Response|error resp = clientEndpoint->post("?key=" + googleChatSpaceKey + "&token=" + googleChatSpaceToken, 
                            {"text": chatMessage});
        if resp is error {
            log:printError("Unable to invoke the webhook", 'error = resp);
            http:BadGateway response = {
                body: "Unable to invoke Google Chat Webhook " + resp.message()
            };
            return response;
        } else {
            http:Ok ok = {};
            return ok;
        }
    }
}

function getModuleList(Module[] modules) returns string {
    string linkList = "";
    foreach Module module in modules {
        linkList += "<" + module.link + "|" + module.name + ">\t\t";
        string? codeOwner = codeOwnersMapping[module.name];
        if codeOwner is string {
            linkList += "<users/" + codeOwner + ">";
        } else {
            log:printError("Code Owner Mapping not found for module: " + module.name);
        }
        linkList += "\n";
    }
    return linkList;
}

map<string> codeOwnersMapping = {
    // Code owners mapping is shared privately
};

Script updated in niveathika@46971a2

Description:
Currently full build pipeline runs daily without notifying build failures. This task is to notify chat.

Describe your task(s)

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

Description:
Build full pipeline from ballerina-lang to ballerina distribution including standard library modules against a specific branch of ballerina-lang where the developer makes changes.

This should use the standard library stages where the last time ballerina-distribution repository daily-build was passed. Therefore it will reduce the build failures in the full build pipeline.

Describe your task(s)
Developers should be able to check whether their changes on ballerina-lang will break the standard library modules or not. This will reduce the build failures in standard library module repositories as well as ballerina-distribution repository.

Description:
Create a workflow to add/remove notification users on demand

Description:
Currently, the script only allows one input to skip tests in both ballerina language and standard library. IF this is divided lang tests can be skipped in urgency and tests can only be run on stdlibs. This reduces the full build pipeline if needed.

Description:
Currently when we are running the Full Build Pipeline For Updated Stages workflow, User needs to create a new branch from the latest stable version for ballerina-lang since it will build the standard library modules from that stage (Otherwise it might fail the full build).

This workflow will help the user to create a new branch from the latest stable version (might be time stamp version) by checking the last commit id of that stage.

The user will give the option to create the new branch on their own github account or in the ballerina-platform.

Description:
The event type in repository dispatch has been updated. This has to be updated in each of the connectors.

Describe your problem(s)

Describe your solution(s)

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

Description:
re-enable tests by uncommenting .github/workflows/ballerina-test-installers.yml

This was disabled in order to merge #71

Description:
Trigger ballerinax connector build workflow triggers the build for all ballerinax connectors to check for breaking changes introduced in distribution. This specific build currently use the docker:nightly image.

Describe your problem(s)
Since docker:nightly image is pushed to docker hub periodically, it's not possible to get immediate update on breaking changes.

Describe your solution(s)
A simple solution would be having an another docker tag like ballerina:breaking_changes for the purpose of checking the breaking changes. If some one want to check for breaking changes for specific distribution then they may push the distribution to the docker hub with the mentioned tag and trigger the workflow.

the ci.yml or daily-build.yml workflow files in ballerinax connectors need to be updated with the specific tag

Related PRs (optional):
#1981 #77

Description:
We need to include following improvements for current full build against lang branch workflow

• Include test skip option to build without tests
• Include option to archive finally built zips and installers (Need to build installer if this is enabled)

Description:
Sending PR from 2201.0.x automated and executed daily.
If CI checks are passing PR could be merged automatically if those are failing notifications chat should be updated

Description:
Currently the build pipeline workflow can be triggered by passing the ballerina-lang repo branch where the developer did his/her changes. However in order to trigger this workflow, he/she has to merge a PR to a branch in ballerina-platform. With this improvement, developer can use his/her forked repo to trigger the workflow.

Goal
Reduce lags in standard library modules & ballerina-distribution

Description:
Currently we maintain separate json to maintain repo owners information. As now code owners have been setup we can use this reference.

Additionally we may have to maintain a reference to map GitHub username to org email. Or else earlier in Github there was option to setup emails for GitHub orgs. If it exists and users have setup it we can use that as well

Description:
We need to create a workflow to bump ballerina-lang version in distribution repo daily. If CI tests are passing it needs to be merged. This needs to be done executed for 2201.0.x and master branch both

Description:
$title

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Description:
Currently following extensions are not packed inside the distribution. Hence for daily builds we need to publish these to stage central as we have done for connectors

• nats
• stan
• jaeger
• prometheus
• choreo
• kafka
• rabbitmq
• java.jdbc
• mssql
• mysql
• oracledb
• postgresql

Description:
Currently standard library module names and versions were read from https://github.com/ballerina-platform/ballerina-standard-library/blob/main/release/resources/stdlib_modules.json file. It should be changed to https://github.com/ballerina-platform/ballerina-release/blob/master/dependabot/resources/extensions.json file since it contains all the modules.

We need to build standard libraries only upto level 8.

In full build pipeline for patch branch, we don't need to bump standard library dependency versions. we only need to bump ballerina-lang version.

Describe your problem(s)

Describe your solution(s)

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

Description:
Setup dev.ballerina.io server

Description:
Use code owners to and lookup csv file to notify all the ballerina modules failures instead of current json lookup in the script

Description:
Currently the build pipeline workflow can be triggered for the desired branch of ballerina-lang and it will build the standard library modules with master branch. With this improvement, developer should be able to pass a standard library module with a branch where the developer did his/her changes.

Describe your solution(s)
Passing inputs for the workflow which contain the standard library module and the branch. Then access them in the python file where the build commands are executed.

Goal
To decrease lags in the standard library modules and ballerina-distribution.

Description:
Currently dependencies are represented as levels. This needs to be improved with a graph representation by using existing json file

Description:
Stdlib dependency order is required for cache generation. Currently this has been sorted in Gradle configurations. Ideally this has to be loaded from JSON based on the dependency tree

Description:
The stdlibs are published to stage environment in RC vote for module owners to test. This needs to be deleted when GA is released or RC vote fails.

Description:
$title

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Description:
$title

Suggested Labels:

Suggested Assignees:

Affected Product Version:

OS, DB, other environment details and versions:

Steps to reproduce:

Related Issues:

Description:

When a user syncs the forked repository with the original repo, if new workflows are added there, they are enabled by default. So when new scheduled automated workflows are there, they get triggered causing unnecessary actions. So better to check the actor before triggering them if only one of the workflow runs is needed overall.

Eg: Nightly syncing lang patch branch with master workflow

Describe your problem(s)

Describe your solution(s)

Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):