Giter Club home page Giter Club logo

github-release-scripts's Introduction

Eclipse Adoptium

This organization provides a home for Git repositories that contain the activities of the Adoptium Working Group, the Eclipse Adoptium Top Level Project and several Eclipse projects that fall under that top level project:

NOTE: The high-level project and issue tracking across all projects is kept in the Adoptium repo issue tracking system.


Please see the Eclipse Adoptium Project description for more information regarding the Adoptium top-level project or its sub-projects (visually depicted in the diagram below).

graph TD

subgraph Eclipse Adoptium
    classDef public fill:#CFE1F3,stroke:#333,stroke-width:4px,color:#000000
    classDef private fill:#FF0000,stroke:#333,stroke-width:4px,color:#000000
    style public fill:#CFE1F3,stroke:#333,stroke-width:4px,color:#000000
    style private fill:#FF0000,stroke:#333,stroke-width:4px,color:#000000
    subgraph Adoptium
        AdoptiumTrigger[adoptium]:::public --- website["adoptium.net"]:::public --- api["api.adoptium.net"]:::public --- blog["blog.adoptium.net"]:::public --- dash["dash.adoptium.net"]:::public
    end
    subgraph Temurin
        subgraph Build
            buildTrigger[temurin-build]:::public --- mirror["mirror-scripts"]:::public --- src["jdk, jdk8u, jdk8u-aarch32, jdk17u"]:::public --- release["github-release-scripts"]:::public --- binaries["temurin8-binaries,<br/>temurin11-binaries,<br/>temurin17-binaries,<br/>temurin19-binaries"]:::public --- installer["installer"]:::public --- build["build-jdk"]:::public
        end
        subgraph Infrastructure
            direction LR
            infraTrigger[infrastructure]:::public --- jenkins["ci-jenkins-pipelines"]:::public --- jenkinshelper["jenkins-helper"]:::public --- support["adoptium-support"]:::public
        end
    end
    subgraph Temurin Compliance
        TCKTrigger[temurin-compliance]:::private --- infra["infrastructure"]:::private --- jck8["JCK8-unzipped"]:::private --- jck11["JCK11-unzipped"]:::private --- jck17["JCK17-unzipped"]:::private --- jck19["JCK19-unzipped"]:::private
    end
    subgraph AQAvit
        AQAvitTrigger[aqa-tests]:::public --- tkg["TKG"]:::public --- test-tools["aqa-test-tools"]:::public --- stf["STF"]:::public --- systemtest["aqa-systemtest"]:::public --- bumblebench["bumblebench"]:::public --- run-aqa["run-aqa"]:::public
    end
    subgraph Incubator
        IncubatorTrigger[jdk11u-fast-startup-incubator]:::public
    end
end
Loading

Eclipse Adoptium Working Group

The Adoptium Working Group promotes and supports high-quality runtimes and associated technology for use across the Java ecosystem. Our vision is to meet the needs of Eclipse and the broader Java community by providing a marketplace for high-quality Java runtimes for Java-based applications. We embrace existing standards and a wide variety of hardware and cloud platforms.

Eclipse Adoptium Top Level Project

The mission of the Eclipse Adoptium Top-Level Project is to distribute high-quality runtimes and associated technology for use within the Java ecosystem. We achieve this through a set of Projects under the Adoptium Project Management Committee (PMC) and a close working partnership with external projects, most notably OpenJDK for providing the Java SE runtime implementation. Our goal is to meet the needs of both the Eclipse community and broader runtime users by providing a comprehensive set of technologies around runtimes for Java applications that operate alongside existing standards, infrastructures, and cloud platforms.

Eclipse AQAvit project

AQAvit is the quality and runtime branding evaluation project for Java SE runtimes and associated technology. During a release it takes a functionally complete Java runtime and ensures that all the additional qualities are present that make it suitable for production use. These quality criteria include good performance, exceptional security, resilience and endurance, and the ability to pass a wide variety of application test suites. In addition to verifying that functionally complete runtimes are release ready, the AQA tests may also serve to verify new functionality during runtime development.

Eclipse Temurin project

The Eclipse Temurin project provides code and processes that support the building of runtime binaries and associated technologies that are high performance, enterprise-caliber, cross-platform, open-source licensed, and Java SE TCK-tested for general use across the Java ecosystem.

Eclipse Temurin Compliance project

The Eclipse Temurin Compliance project is responsible for obtaining, managing, and executing the Oracle Java SE Compatibility Kit (JCK) on Eclipse Temurin binaries. The work is done on private infrastructure and using code managed in closed repositories only available to committers of Temurin Compliance. The public artefacts produced by this project are limited to an indication of whether a particular Eclipse Temurin binary is Java SE compliant or not.

Eclipse Mission Control project

Eclipse Mission Control enables you to monitor and manage Java applications without introducing the performance overhead normally associated with these types of tools. It uses data collected for normal adaptive dynamic optimization of the Java Virtual Machine (JVM). Besides minimizing the performance overhead, this approach eliminates the problem of the observer effect, which occurs when monitoring tools alter the execution characteristics of the system.

github-release-scripts's People

Contributors

adambrousseau avatar andrew-m-leonard avatar bethgriggs avatar dependabot[bot] avatar gdams avatar jerboaa avatar johnoliver avatar karianna avatar keithc-ca avatar smlambert avatar snyk-bot avatar sophia-guo avatar supahgreg avatar sxa avatar sxa555 avatar tellison avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

github-release-scripts's Issues

Ensure we are pushing AQAvit TAP files to the binaries repository for each release

As part of an official Temurin release that we want to list in the Adoptium marketplace, we need to store the TAP files alongside the JDK binary. The marketplace data should contain a link to the AQAvit TAP files (this is a requirement for listing in the marketplace).

See examples of where we refer to those results in the json files here:
https://github.com/adoptium/marketplace-data/blob/main/17/jdk_17_0_5_8.json#L22
https://github.com/adoptium/marketplace-data/blob/main/8/jdk8u352_b08.json#L22
etc

But in fact, we are currently not pushing these files to the binaries repo, so when you try to access those links:
https://github.com/adoptium/temurin17-binaries/releases/download/jdk-17.0.5%2B8/OpenJDK17U-jdk_x64_alpine-linux_hotspot_17.0.5_8.tap.zip they are not found.

We do archive TAP files for each pipeline as part of the Jenkins job, but need to ensure they are pushed to the permanent location in the binaries repo.

A caveat: All AQAvit tests must pass. If there are many failures during a release, we need to additionally include the Grinder TAP files to this zip file of stored TAP files, but this work can be done iteratively. (Ideally we work to have all tests run and pass as part of the build pipeline and no reruns are required... but in reality, we know there may always be things that require rerun).

Can we fix the timestamps on the tags in `temurinXX-binaries?

The temurinXX-binaries releases have tags created for each release, but the date associated with that tag is the last time a file in the binaries repositories (Not an item in the release) was updated. At present that means it's from August 2022 when the README.md was last updated. This is undesirable and confusing.

From endoflife-date/endoflife.date#2729 (comment):

It should be possible to fix this by using annotated tags, instead of lightweight git tags. Annotated tags (created using git -a) carry their own creation date, while lightweight tags are just a link to a specific commit, which carries the date.

We should look at whether we can test that and make it work to avoid the dates on the tags being out of sync with those on the releases.

releaseCheck is not using exact string matches, but picks up substrings

Split out from January 2024 retrospective -

https://ci.adoptium.net/job/build-scripts/job/release/job/refactor_openjdk_release_tool/8089/ was published with a name missing the jdk- prefix, but the releaseCheck.sh didn't show it as everything abnormal - it seems to have found the right release - perhaps because it was a substring of the 'real' value - something to look at it in the releaseCheck.sh script as a potential buglet.

          https://ci.adoptium.net/job/build-scripts/job/release/job/refactor_openjdk_release_tool/8089/ was published with a name missing the `jdk-` prefix, but the releaseCheck.sh didn't show it as everything abnormal - it seems to have found the right release - perhaps because it was a substring of the 'real' value - something to look at it in the releaseCheck.sh script as a potential buglet.
*** PERFORMING RELEASE CHECK TO SEE IF THERE ARE ANY UNEXPECTED PROBLEMS ***
Grabbing information from https://github.com/adoptium/temurin21-binaries/releases/tag/21.0.2+13
FILTER IS: 21.0.2%2B13
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
100 12464    0 12464    0     0   6000      0 --:--:--  0:00:02 --:--:--  6000
100 10.1M    0 10.1M    0     0  4756k      0 --:--:--  0:00:02 --:--:-- 4756k
Linux on x64: OK!
Linux on aarch64: OK!
Linux on ppc64le: OK!
Linux on s390x: OK!
Linux on arm: Not published:
AIX on ppc64: OK!
Alpine on x64: OK!
Alpine on aarch64: OK!
Windows on x64: OK!
MacOS on x64: OK!
MacOS on aarch64: OK!
Source images: OK!

Originally posted by @sxa in adoptium/temurin#13 (comment)

GitHub API limits to 30 releases

30 release limit:
https://api.github.com/repos/AdoptOpenJDK/openjdk-nightly/releases

Increased to 90 release limit through the URL:
https://api.github.com/repos/AdoptOpenJDK/openjdk-nightly/releases?per_page=90

The NPM tool has a listReleases function that George is using on the openjdk-website-backend repo (app.js):
http://github-tools.github.io/github/docs/3.1.0/Repository.html#listReleases

However, this does not appear to support the same per_page option that can be passed in - so we're currently stuck with 30 releases.

Suggested action after discussion: raise an issue on the GitHub repo for the npm tool: https://github.com/github-tools/github/issues

release tool "out of heap space" failure

https://ci.adoptium.net/job/build-scripts/job/release/job/refactor_openjdk_release_tool/8197/

Uploading OpenJDK11U-debugimage_x64_mac_hotspot_11.0.23_2-ea.tar.gz.json
Uploading OpenJDK11U-sbom_aarch64_mac_hotspot_11.0.23_2-ea.json.sig
Uploading OpenJDK11U-testimage_s390x_linux_hotspot_11.0.23_2-ea.tar.gz
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
	at java.util.Arrays.copyOf(Arrays.java:3236)
	at java.io.ByteArrayOutputStream.grow(ByteArrayOutputStream.java:118)
	at java.io.ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:93)

Night builds fail to publish: makefailurelogs not being ignored

The check against makefailurelogs does not work as expected, and continues to try and process the file:

+ [[ ! OpenJDK20U-makefailurelogs_aarch64_alpine-linux_hotspot_2023-03-14-03-30.tar.gz =~ *makefailurelogs* ]]
+ echo 'Processing non-archive file: OpenJDK20U-makefailurelogs_aarch64_alpine-linux_hotspot_2023-03-14-03-30.tar.gz'

Syntax error in releaseCheck.sh

There seems to be an issue in the releaseCheck script, at least when run on the C3jenkins node - I couldn't replicate this on my local machine:

From https://ci.adoptium.net/job/build-scripts/job/release/job/refactor_openjdk_release_tool/8046/console

Linux on x64: OK!
Linux on aarch64: Not published:
Linux on ppc64le: OK!
Linux on s390x: OK!
Linux on arm: Not published:
AIX on ppc64: Not published:
../sbin/releaseCheck.sh: 62: [: x64: unexpected operator
../sbin/releaseCheck.sh: 62: [: aarch64: unexpected operator
Windows on x64: OK!
MacOS on x64: OK!
MacOS on aarch64: OK!
Source images: OK!

Nightly builds page offering last successful build's binary for each date

For example, the nightly build of Linux x64 platform was last successful here - https://ci.adoptopenjdk.net/job/openjdk8_openj9_build_x86-64_linux/62. Every subsequent build has failed to date.

Yet the nightly build page (https://adoptopenjdk.net/nightly.html?variant=openjdk8-openj9) currently offers binaries (with different dated names) for all the dates it failed - they are actually the same binary from the last successful build ...

ben@ben-ThinkPad-W530:~$ ls -l OpenJDK8*.tar.gz
-rw-r--r-- 1 ben ben 82471252 Feb  9 14:01 OpenJDK8-OPENJ9_x64_Linux_20180902.tar.gz
-rw-r--r-- 1 ben ben 82471252 Feb 13 14:01 OpenJDK8-OPENJ9_x64_Linux_20181302.tar.gz
ben@ben-ThinkPad-W530:~$ diff OpenJDK8-OPENJ9_x64_Linux_20180902.tar.gz  OpenJDK8-OPENJ9_x64_Linux_20181302.tar.gz
ben@ben-ThinkPad-W530:~$ 

I've not checked whether this is the case across platforms and VM variants.

releaseCheck.sh no longer works

The parsing of the github web pages that is done in releaseCheck.sh has been broken in recent months, presumably because GitHub has changed the rendering of the pages and this was initially a quick hack parsing the HTML. This means that the script is unable to detect the list of artefacts within the release, and therefore says that everything is not published. Since this is visible in the release tool job it can be confusing for releasers running the job manually.

The script will therefore use an overhaul, perhaps to utilise the github API to get the appropriate information, to enable the script to work as designed.

Rename the x64_linux_largeHeap release to differentiate from x64_linux release

We have just added a couple of new builds to the build pipelines (build-355), that are of the same os but a different variation (x64_linux_largeHeap, and win32 builds). Need to update the Release.sh script so that they get named differently when listed in the website.

Currently they are are named the same as their related builds, which will cause confusion.

Multiple failures uploading to github recently

Seems to always be failing when it hits the mac x64 package on all versions. Rate limit? Something specific to that artifact? (Looks ok, not zero-bytes)
Excluding **/*_x64_mac_* seems to progress ok (Based on one run)

Exception in thread "main" org.kohsuke.github.HttpException: Server returned HTTP response code: -1, message: 'null' for URL: https://uploads.github.com/repos/adoptium/temurin19-binaries/releases/63237181/assets?name=OpenJDK-jdk_x64_mac_hotspot_2022-03-31-03-30.pkg
	at org.kohsuke.github.GitHubClient.interpretApiError(GitHubClient.java:494)
	at org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:414)
	at org.kohsuke.github.GitHubClient.sendRequest(GitHubClient.java:358)
	at org.kohsuke.github.Requester.fetch(Requester.java:76)
	at org.kohsuke.github.GHRelease.uploadAsset(GHRelease.java:249)
	at org.kohsuke.github.GHRelease.uploadAsset(GHRelease.java:224)
	at net.adoptium.release.UploadAdoptReleaseFiles$_uploadFiles_closure2.doCall(UploadFiles.groovy:89)
	at net.adoptium.release.UploadAdoptReleaseFiles$_uploadFiles_closure2.call(UploadFiles.groovy)

Request to bump to latest version of kohsuke:github-api

Currently at 1.95. Latest release is 1.127

compile 'org.kohsuke:github-api:1.95'

That version has a hardcoded upload url for release assets which blocks me from uploading to a GHE server.
https://github.com/hub4j/github-api/blob/c1bab63ebdd9c93e49a5879331234de488e91590/src/main/java/org/kohsuke/github/GHRelease.java#L140

I tried this version on my fork and it seems to work. I'm not sure what else would need to be tested in order to accept the change.

Remove non-suffixed duplicates of JSON files after website changes

The following lines should be removed from app.js after the website is updated to use the _openjdk8 suffixed versions of the JSON files:

fs.writeFileSync('releases.json', JSON.stringify(result, null, 2))
fs.writeFileSync('latest_release.json', JSON.stringify(result[0], null, 2))
fs.writeFileSync('nightly.json', JSON.stringify(result, null, 2))
fs.writeFileSync('latest_nightly.json', JSON.stringify(result[0], null, 2))

Stop pushing releases.json and latest_release.json

Since everyone should be using api.adoptopenjdk.net, and the V2 api does not rely on these files. I think it should be possible to stop pushing these files if/when we deprecate and remove the v1 api.

Should `TIMESTAMP` be required when publishing nightlies

There is a conflict between two parts of the process.
In the refactor release tool job it says Optional timestamp to add for nightly builds. against the description of the TIMESTAMP field
But if you try to run without a TIMESTAMP set the job says Nightly must have a TIMESTAMP set
Is it optional, or is it mandatory? Either way, the messages need to be consistent with each other.

Automate AQAvit tagging and release branch creation

Update and move the scripts from https://github.com/smlambert/release-aqavit/tree/main/scripts into aqa-tests repo and create a workflow for this process. Workflow can be manually run for now, but can consider adding a schedule where we run it 7 weeks before an OpenJDK release (as per this schedule).

Expected behaviour of automation:

  • For CPU, we tag and create release branches based from the expected tags defined here

  • For Feature releases and respins, we have tended to tag and create a release branch, then use that same AQAvit release branch for the CPU directly following it (March/April and Sept/Oct share release branches, and only if needed, branched a dot release from it, so if March 2024 JDK 22 was v1.0.1-release branch, then April 2024 CPU would see the update of JDK22 and addition of JDK8, JDK11, JDK17 and JDK21 information into the testenv.properties of the v1.0.1-release branch. If a breaking change was required in the branch, then create a separate branch named v1.0.1.1-release derived from the v1.0.1-release branch would be created and used for a CPU release (or a respin of a CPU or feature release). This seems to have served us well, so continue in this fashion for now.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.