cdklabs / publib Goto Github PK

View Code? Open in Web Editor NEW

52.0 16.0 21.0 4.8 MB

A unified toolchain for publishing libraries to popular package managers (formally jsii-release)

License: Apache License 2.0

Shell 21.23% JavaScript 0.77% TypeScript 76.83% Go 1.17%

publib's People

Stargazers

Watchers

publib's Issues

Maven staging timeouts

We are seeing quite a lot of timeouts when publishing to maven central:

https://github.com/aws/constructs/runs/1824718480?check_suite_focus=true

The timeout can be increased as described here:

https://issues.sonatype.org/plugins/servlet/mobile#issue/OSSRH-63605

https://help.sonatype.com/repomanager2/staging-releases/configuring-your-project-for-deployment#ConfiguringYourProjectforDeployment-DeploymentwiththeNexusStagingMavenPlugin

Maven password not escaped when generating setting.xml file

When generating the settings.xml file maven in publish-mvn.sh escape the password field to avoid illegal xml characters.

publib-npm only uses authToken

publishing to npm only works for NPM_TOKEN that match the authToken format, it doesn't allow for repositories that only allow for _auth to be used in npmrc files.

I've opened a PR for this #821

Go release incorrectly creating tags for packages with version in the package name

Tags are not being created correctly for the awscdkapigatewayv2 alpha modules. The v2 is being stripped from the name so instead of awscdkapigatewayv2alpha/v2.0.0-rc.24 we are getting awscdkapigatewayalpha/v2/v2.0.0-rc.24

Npm not using ~/.npmrc

I'd like to use npx publib-npm in the pipeline with AWS CodeArtifact, however, it is not using credentials from ~/.npmrc

Successfully configured npm to use AWS CodeArtifact repository https://xxx-xxx.d.codeartifact.us-east-1.amazonaws.com/npm/lib/ 
Login expires in 12 hours at 2023-03-22 23:06:25-04:00

Then error with publish

npx publib-npm                                                                                                                                                           
NPM_TOKEN is required

Native npm publish works properly, using auth from ~/.npmrc
npm publish dist/js/lib@${version}.jsii.tgz

With NPM_TOKEN and NPM_REGISTRY variables for the CodeArtifact registry, it errors out with

npx publib-npm          

Unable to locate credentials. You can configure credentials by running "aws configure".
```

Do we support `_auth` config in npm?

For now, I want to publish a package that is created by jsii to my hosted Nexus. However, I must config the NPM_TOKEN, I am able to publish successfully. But for other repositories, I use npm config _auth to do it.

I hope we can support npm config _auth at the same time. Because in other repositories, I use _auth to do the authentication for the private nexus repository. Due to I use aws-cdk to deploy some AWS resources, I want to use products around cdklab as well. But the experience isn't good. 😢

PyPI publisher: missing dry run support

MissingProjectException in jsii-release-maven

My CDK project can't release to Maven Central due to the following error in jsii-release-maven.

Error:  The goal you specified requires a project to execute but there is no POM in this directory (/__w/cdk-datadog-resources/cdk-datadog-resources/dist/java). Please verify you invoked Maven from the correct directory. -> [Help 1]

Am I making a mistake in the settings?
This project created by projen.

Workflow logs

https://github.com/NomadBlacky/cdk-datadog-resources/runs/2718228766

Run npx -p jsii-release@latest jsii-release-maven
  npx -p jsii-release@latest jsii-release-maven
  shell: sh -e {0}
  env:
    MAVEN_GPG_PRIVATE_KEY: ***
  
    MAVEN_GPG_PRIVATE_KEY_PASSPHRASE: ***
    MAVEN_PASSWORD: ***
    MAVEN_USERNAME: ***
    MAVEN_STAGING_PROFILE_ID: ***
npx: installed 8 in 1.873s
Importing GPG key...
gpg: keyring `/tmp/tmp.2MCJZZF6iy/secring.gpg' created
gpg: keyring `/tmp/tmp.2MCJZZF6iy/pubring.gpg' created
gpg: key C6744B5E: secret key imported
gpg: /tmp/tmp.2MCJZZF6iy/trustdb.gpg: trustdb created
gpg: key C6744B5E: public key "Takumi Kadowaki <[email protected]>" imported
gpg: key 44A2BFDA: secret key imported
gpg: key 44A2BFDA: public key "Takumi Kadowaki <***@gmail.com>" imported
gpg: Total number processed: 2
gpg:               imported: 2  (RSA: 2)
gpg:       secret keys read: 2
gpg:   secret keys imported: 2
gpg_key_id=3CF60FF5C6744B5E
6458ADFB44A2BFDA
📦 Publishing to Maven Central
/tmp/tmp.gaJGOpScSL
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 Preparing repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/apache-maven
Java version: 1.8.0_292, vendor: Amazon.com Inc., runtime: /usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64/jre
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.4.0-1047-azure", arch: "amd64", family: "unix"
[DEBUG] Created new class realm maven.api
[DEBUG] Importing foreign packages into class realm maven.api
[DEBUG]   Imported: javax.annotation.* < plexus.core
[DEBUG]   Imported: javax.annotation.security.* < plexus.core
[DEBUG]   Imported: javax.enterprise.inject.* < plexus.core
[DEBUG]   Imported: javax.enterprise.util.* < plexus.core
[DEBUG]   Imported: javax.inject.* < plexus.core
[DEBUG]   Imported: org.apache.maven.* < plexus.core
[DEBUG]   Imported: org.apache.maven.artifact < plexus.core
[DEBUG]   Imported: org.apache.maven.classrealm < plexus.core
[DEBUG]   Imported: org.apache.maven.cli < plexus.core
[DEBUG]   Imported: org.apache.maven.configuration < plexus.core
[DEBUG]   Imported: org.apache.maven.exception < plexus.core
[DEBUG]   Imported: org.apache.maven.execution < plexus.core
[DEBUG]   Imported: org.apache.maven.execution.scope < plexus.core
[DEBUG]   Imported: org.apache.maven.lifecycle < plexus.core
[DEBUG]   Imported: org.apache.maven.model < plexus.core
[DEBUG]   Imported: org.apache.maven.monitor < plexus.core
[DEBUG]   Imported: org.apache.maven.plugin < plexus.core
[DEBUG]   Imported: org.apache.maven.profiles < plexus.core
[DEBUG]   Imported: org.apache.maven.project < plexus.core
[DEBUG]   Imported: org.apache.maven.reporting < plexus.core
[DEBUG]   Imported: org.apache.maven.repository < plexus.core
[DEBUG]   Imported: org.apache.maven.rtinfo < plexus.core
[DEBUG]   Imported: org.apache.maven.settings < plexus.core
[DEBUG]   Imported: org.apache.maven.toolchain < plexus.core
[DEBUG]   Imported: org.apache.maven.usability < plexus.core
[DEBUG]   Imported: org.apache.maven.wagon.* < plexus.core
[DEBUG]   Imported: org.apache.maven.wagon.authentication < plexus.core
[DEBUG]   Imported: org.apache.maven.wagon.authorization < plexus.core
[DEBUG]   Imported: org.apache.maven.wagon.events < plexus.core
[DEBUG]   Imported: org.apache.maven.wagon.observers < plexus.core
[DEBUG]   Imported: org.apache.maven.wagon.proxy < plexus.core
[DEBUG]   Imported: org.apache.maven.wagon.repository < plexus.core
[DEBUG]   Imported: org.apache.maven.wagon.resource < plexus.core
[DEBUG]   Imported: org.codehaus.classworlds < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.* < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.classworlds < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.component < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.configuration < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.container < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.context < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.lifecycle < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.logging < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.personality < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.util.xml.Xpp3Dom < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.util.xml.pull.XmlPullParser < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.util.xml.pull.XmlPullParserException < plexus.core
[DEBUG]   Imported: org.codehaus.plexus.util.xml.pull.XmlSerializer < plexus.core
[DEBUG]   Imported: org.eclipse.aether.* < plexus.core
[DEBUG]   Imported: org.eclipse.aether.artifact < plexus.core
[DEBUG]   Imported: org.eclipse.aether.collection < plexus.core
[DEBUG]   Imported: org.eclipse.aether.deployment < plexus.core
[DEBUG]   Imported: org.eclipse.aether.graph < plexus.core
[DEBUG]   Imported: org.eclipse.aether.impl < plexus.core
[DEBUG]   Imported: org.eclipse.aether.installation < plexus.core
[DEBUG]   Imported: org.eclipse.aether.internal.impl < plexus.core
[DEBUG]   Imported: org.eclipse.aether.metadata < plexus.core
[DEBUG]   Imported: org.eclipse.aether.repository < plexus.core
[DEBUG]   Imported: org.eclipse.aether.resolution < plexus.core
[DEBUG]   Imported: org.eclipse.aether.spi < plexus.core
[DEBUG]   Imported: org.eclipse.aether.transfer < plexus.core
[DEBUG]   Imported: org.eclipse.aether.version < plexus.core
[DEBUG]   Imported: org.fusesource.jansi.* < plexus.core
[DEBUG]   Imported: org.slf4j.* < plexus.core
[DEBUG]   Imported: org.slf4j.event.* < plexus.core
[DEBUG]   Imported: org.slf4j.helpers.* < plexus.core
[DEBUG]   Imported: org.slf4j.spi.* < plexus.core
[DEBUG] Populating class realm maven.api
[INFO] Error stacktraces are turned on.
[DEBUG] Message scheme: color
[DEBUG] Message styles: debug info warning error success failure strong mojo project
[DEBUG] Reading global settings from /usr/local/apache-maven/conf/settings.xml
[DEBUG] Reading user settings from /tmp/tmp.gaJGOpScSL/mvn-settings.xml
[DEBUG] Reading global toolchains from /usr/local/apache-maven/conf/toolchains.xml
[DEBUG] Reading user toolchains from /root/.m2/toolchains.xml
[DEBUG] Using local repository at /root/.m2/repository
[DEBUG] Using manager EnhancedLocalRepositoryManager with priority 10.0 for /root/.m2/repository
[INFO] Scanning for projects...
[DEBUG] Extension realms for project org.apache.maven:standalone-pom:pom:1: (none)
[DEBUG] Looking up lifecycle mappings for packaging pom from ClassRealm[plexus.core, parent: null]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.061 s
[INFO] Finished at: 2021-06-01T11:47:19Z
[INFO] ------------------------------------------------------------------------
Error:  The goal you specified requires a project to execute but there is no POM in this directory (/__w/cdk-datadog-resources/cdk-datadog-resources/dist/java). Please verify you invoked Maven from the correct directory. -> [Help 1]
org.apache.maven.lifecycle.MissingProjectException: The goal you specified requires a project to execute but there is no POM in this directory (/__w/cdk-datadog-resources/cdk-datadog-resources/dist/java). Please verify you invoked Maven from the correct directory.
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:85)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Error:  
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MissingProjectException
Error: Process completed with exit code 1.

Publish to NPM fails with PNPM

As I upgraded the repository with PNPM as package manager, the publish to npm registrery fails with the following message:

npm notice Publishing to https://registry.npmjs.org/ with tag latest and default access
npm ERR! code E415
npm ERR! 415 Unsupported Media Type - PUT https://registry.npmjs.org/<mypackage> - Hard link is not allowed

The system cannot find path specified

npx publib-npm throws the message "The system cannot find path specified" on Windows.

npx works with other node modules like jest.

Is there any config that needs to be done on Windows?

thanks!

go: tags should include module name

For example, for github.com/aws/constructs-go, Instead of v3.3.1 the tag should be constructs/v3.3.1.

This technically allows different modules in the same repo to be tagged with different versions.

Reference: https://research.swtch.com/vgo-module#multiple-module_repositories

In the major subdirectory convention, v2/ contains the module “my/thing/v2”. A natural extension is to allow subdirectories not named for major versions. For example, we could add a blue/ subdirectory that contains the module “my/thing/blue”, confirmed by a blue/go.mod file with that module path. In this case, the source control commit tags addressing that module would take the form blue/v1.x.x. Similarly, the tag blue/v2.x.x would address the blue/v2/ subdirectory. The existence of the blue/go.mod file excludes the blue/ tree from the outer my/thing module.

Maven publishing abandons staging repositories for existing package versions

Maven publishing is creating staging repositories even if the package version being released already exists in Maven. If there are a lot of such concurrent requests, this leads to Maven blocking publishing credentials since this impacts their cleanup process and also customers since it becomes a noisy neighbor issue.

We need to fix our publishing script to check for existing version before starting the publishing process.

Related issue with Maven: https://issues.sonatype.org/browse/OSSRH-94655

Endpoint for new maven repositories

The endpoint for new maven repositories is now:

https://s01.oss.sonatype.org

This means for new maven repositories:

MAVEN_ENDPOINT must be set to https://s01.oss.sonatype.org, otherwise jsii-release-maven will fail with a "403 Forbidden" error
to get the "Staging profile ID", one has to log in at https://s01.oss.sonatype.org, not at https://oss.sonatype.org

Thought this would be useful to mention in the docs.

Sporadic maven publishing failures

In many of our projen projects, which uses jsii-release to do the publishing, we see the following maven errors:

Waiting for operation to complete...
Dec 15, 2021 12:38:42 AM com.sun.jersey.api.client.ClientResponse getEntity
..............................
SEVERE: A message body reader for Java class com.sonatype.nexus.staging.api.dto.StagingProfileRepositoryDTO, and Java type class com.sonatype.nexus.staging.api.dto.StagingProfileRepositoryDTO, and MIME media type text/html was not found

Dec 15, 2021 12:38:42 AM com.sun.jersey.api.client.ClientResponse getEntity
SEVERE: The registered message body readers compatible with the MIME media type are:
*/* ->
  com.sun.jersey.core.impl.provider.entity.FormProvider
  com.sun.jersey.core.impl.provider.entity.StringProvider
  com.sun.jersey.core.impl.provider.entity.ByteArrayProvider
  com.sun.jersey.core.impl.provider.entity.FileProvider
  com.sun.jersey.core.impl.provider.entity.InputStreamProvider
  com.sun.jersey.core.impl.provider.entity.DataSourceProvider
  com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$General
  com.sun.jersey.core.impl.provider.entity.ReaderProvider
  com.sun.jersey.core.impl.provider.entity.DocumentProvider
  com.sun.jersey.core.impl.provider.entity.SourceProvider$StreamSourceReader
  com.sun.jersey.core.impl.provider.entity.SourceProvider$SAXSourceReader
  com.sun.jersey.core.impl.provider.entity.SourceProvider$DOMSourceReader
  com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$General
  com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$General
  com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$General
  com.sun.jersey.core.impl.provider.entity.EntityHolderReader
  com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider

At first glance this feels like some configuration error with the project, but the thing is that it seems this is sporadic, since subsequent executions succeed fine.

Putting this here so we can keep track of it, it might be worth adding some more advanced retry mechanism for maven publishing.

no dry-run option for npm

there's no way to dry-run publish npm packages, as this options is available in publib-npm

Support publishing maven packages to AWS CodeArtifact

I believe the existing implementation can be extended to conditionally handle AWS CodeArtifact and use AWS CLI, like the NPM implementation, to get the AWS CodeArtifact temporary authorization token for use as MAVEN_PASSWORD.

I traced an issue I was having in projen (v0.43.0) back to this project. When attempting to build an AWS Construct Library published to AWS CodeArtifact to both NPM and Maven, only NPM was supported. I was unable to get Maven to function and thought it best to dig deeper to bring first-class support forward.

GitHub Packages support (#11, #12) provides a great foundation to extend to fully address AWS CodeArtifact support (#10)

Once implemented, an example GitHub Actions publish step could look like:

- name: Publish package
  run: npx -p jsii-release jsii-release-maven
  env:
    MAVEN_SERVER_ID: codeartifact
    MAVEN_USERNAME: ${{ github.actor }}
    MAVEN_REPOSITORY_URL: "https://my_domain-111122223333.d.codeartifact.us-west-2.amazonaws.com/maven/my_repo/'"
    AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

We could borrow from the NPM implementation (https://github.com/cdklabs/jsii-release/blob/master/bin/jsii-release-npm#L23-L38) to conditionally assign environment variable MAVEN_PASSWORD to an AWS CodeArtifact authorization token using AWS CLI if the MAVEN_REPOSITORY_URL is defined and matches an AWS CodeArtifact pattern. This then would be picked up by \${env.MAVEN_USERNAME} within mvn-settings.xml creation steps.

Support publishing go modules located in the root directory of the repo

We currently assume the DIR passed to the script must contain subdirectories where go modules are located. This is true for the mono repo use-case, but for single module repo's this is an author choice.

For example, our own jsii-runtime-go doesn't follow this pattern - and we need to support it.

Cannot publish to local npm registry

If you use a tool like Verdaccio to host a proxy npm registry on your own computer and try setting the environment variable NPM_REGISTRY to localhost:XXXX when invoking publib-npm, the command fails because it tries publishing to https://localhost:4872 instead of http://localhost:4872. Whether or not we connect to the registry over HTTPS should be configurable.

Pin dependencies

Hi there,

As #1107 was introduced by a new minor update to a maven task run in the release process I was wondering what you would think of (optionally?) pinning all used dependencies? Package managers don't seem to rapidly change their publishing workflows so I would assume what works now should also work in the future, so there is not necessarily a need to stay on top of all dependency updates. I'd love to have an option to limit the impact of dependency updates so that my main workflows can remain stable.

I'm more than happy to do a PR if you feel like this is a valuable addition to the project.

publib-maven fails during `sign-and-deploy-file` step with PINENTRY_LAUNCHED 2145 curses 1.1.1 not a tty

Command being run

npx -p publib@latest publib-maven

Uses version [email protected]

Env variables provided

env:
          MAVEN_GPG_PRIVATE_KEY: ${{ secrets.MAVEN_GPG_PRIVATE_KEY }}
          MAVEN_GPG_PRIVATE_KEY_PASSPHRASE: ${{ secrets.MAVEN_GPG_PRIVATE_KEY_PASSPHRASE }}
          MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
          MAVEN_USERNAME: ${{ secrets.MAVEN_USERNAME }}
          MAVEN_STAGING_PROFILE_ID: ${{ secrets.MAVEN_STAGING_PROFILE_ID }}

Complete error message

[INFO] Signer 'gpg' is signing 4 files
[DEBUG] 2.2.27
[GNUPG:] KEY_CONSIDERED 0E2DB99D44DEA2E3BE4A9D3326689A5EA6A5AFBF 2
[GNUPG:] BEGIN_SIGNING H10
[GNUPG:] PINENTRY_LAUNCHED 2145 curses 1.1.1 not a tty - - ? 1001/127 -
gpg: signing failed: No such file or directory
[GNUPG:] FAILURE sign 83918929
gpg: signing failed: No such file or directory
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.226 s
[INFO] Finished at: 2024-03-12T10:44:01Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:3.2.0:sign-and-deploy-file (default-cli) on project standalone-pom: Exit code: 2 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:3.2.0:sign-and-deploy-file (default-cli) on project standalone-pom: Exit code: 2
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:375)
    at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)

Additional details

This command is being run with a GitHub workflow: https://github.com/mongodb/awscdk-resources-mongodbatlas/blob/main/.github/workflows/release.yml#L120.
This publish workflow was working without issues in the past, with the last successful release on the 6th of February. I can see that a new version of maven-gpg-plugin (3.2.0) was release on the 8th of March, not sure if this had an impact.

When running the publish command locally (on macOS) I was requested to enter the GPG passphrase even when having the MAVEN_GPG_PRIVATE_KEY_PASSPHRASE env variable defined.

Any inputs or possible workarounds would be greatly appreciated.

Support publishing public scoped packages to npm

When you want to publish scoped packages to npm, you need to run npm publish --access public (https://docs.npmjs.com/configuring-your-npm-client-with-your-organization-settings)

It would be great to support this in jsii-release.

Suggestion: Backstage Application Release Orchestration Plugin

Suggestion: would be great to have this tool as a backstage plugin - https://backstage.io/plugins

support AWS CodeArtifact

AWS CodeArtifact generates the NPM_REGISTRY with a trailing /:

And the jsii-release-npm will generate NPM_REGISTRY with double slash in .npmrc

https://github.com/eladb/jsii-release/blob/4d05bf42a90e71dc170bd189bf708234a2cece00/bin/jsii-release-npm#L25

which returns the Unable to authenticate error

$ NPM_REGISTRY='pahud-112233445566.d.codeartifact.ap-northeast-1.amazonaws.com/npm/aws-repo/' NPM_TOKEN=`aws codeartifact get-authorization-token --domain pahud --domain-owner 903779448426 --query authorizationToken --output text` node_modules/jsii-release/bin/jsii-release-npm

npm notice 
npm notice 📦  [email protected]
npm notice === Tarball Contents === 
npm notice 19.3kB .jsii                        
npm notice 11.4kB LICENSE                      
npm notice 1.2kB  .projenrc.js                 
npm notice 3.8kB  lib/index.js                 
npm notice 1.7kB  test/index.test.js           
npm notice 1.0kB  test/integ.api.js            
npm notice 1.3kB  .eslintrc.json               
npm notice 2.4kB  package.json                 
npm notice 19B    version.json                 
npm notice 1.6kB  README.md                    
npm notice 346B   lib/index.d.ts               
npm notice 31B    test/index.test.d.ts         
npm notice 11B    test/integ.api.d.ts          
npm notice 397B   .mergify.yml                 
npm notice 339B   .github/workflows/build.yml  
npm notice 1.3kB  .github/workflows/release.yml
npm notice === Tarball Details === 
npm notice name:          cdk-codeartifact-demo                   
npm notice version:       0.0.0                                   
npm notice package size:  12.6 kB                                 
npm notice unpacked size: 46.1 kB                                 
npm notice shasum:        eee847e49c128493f884a17e8bf238d500c653d2
npm notice integrity:     sha512-XH+b7+/PKCedZ[...]wsLe6ajbdlCVw==
npm notice total files:   16                                      
npm notice 
npm ERR! code E401
npm ERR! Unable to authenticate, need: Bearer realm="pahud/aws-repo", Basic realm="pahud/aws-repo"

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/pahud/.npm/_logs/2020-06-12T02_32_35_628Z-debug.log
ERROR

A quick fix is to always remove the trailing slash with bash substitution like this

echo "//${NPM_REGISTRY%%/}/:_authToken=${NPM_TOKEN}" > ~/.npmrc

And it will work with both AWS CodeArtifact as well as npmjs.

Support AWS CodeArtifact for Maven

Support AWS CodeArtifact for Maven.
Add documentation and/or example how to configure AWS CodeArtifact for jsii-release

Use Case

I want to publish my CDK constructs built with JSII (NPM/PyPI/Maven) to AWS CodeArtifact - and I've now succesfully done it with NPM+PyPI 👍

The maven setup seems quite complex so I haven't (at least yet) tried to figure it out myself because my very limited knowledge of Maven and Java-ecosystem ☹️ But I can try some day!

Proposed Solution

Due to above reason I can't be sure, but to me it seems that the following hardcoded values in bin/jsii-release-maven will most probably prevent publishing to CodeArtifact:

-DrepositoryId=maven-central on line 133
-DserverId=ossrh on line 153

Basically the solution should be similar as with npm and with pypi (see my PR): Rely on environment variables (which work well in CI-environment).

Other

👋 I may be able to implement this feature request
⚠️ This feature might incur a breaking change

This is a 🚀 Feature Request

Add ability to publish Golang bindings to non-github.com repositories

Publib throws an error if someone tries to publish Go code to a non-github.com repository, such as GitHub enterprise (github.mycompany.com) or other locations like Bitbucket: https://github.com/cdklabs/publib/blob/main/src/targets/go.ts#L262

It would be nice to have the option to publish to GitHub enterprise and Bitbucket repos in addition to github.com for people that want to publish their Go code to internal repositories.

Support NPM OTP token

Problem

NPM package can't be published with publib when 2FA is configured in the target NPM account.

Details

npm publish command used in publib-npm (source) requires an --opt flag when 2FA is configured. There's no equivalent env variable according to NPM docs.

Workaround

I was able to publish an NPM package by editing publib sources in node_modules. I added the --otp flag in node_modules/.bin/publib-npm:

  npm publish ${tag} ${access} ${file} --otp <my_temp_code> 2>&1 | tee ${log}

Twine upload errors don't provide any detail

During a projen "Publish to Pypi" action I got the following error

Uploading enterprise_utils-0.0.0-py3-none-any.whl
25l
  0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/30.3 kB • --:-- • ?
  0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/30.3 kB • --:-- • ?
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.3/30.3 kB • 00:00 • 84.9 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.3/30.3 kB • 00:00 • 84.9 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.3/30.3 kB • 00:00 • 84.9 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.3/30.3 kB • 00:00 • 84.9 MB/s
25hWARNING  Error during upload. Retry with the --verbose option for more details. 
Error: Process completed with exit code 1.

There was no option for --verbose, so I included publib as a project dependency, adjusted bin/publib-pypi to use

python3 -m twine upload --verbose --skip-existing *

Here I got a more helpful error of

INFO     Response from https://upload.pypi.org/legacy/:                                                                              
         400 '[email protected]:professionalaf/enterprise-utils.git' is an invalid value for Home-Page. Error: Invalid URI See          
         https://packaging.python.org/specifications/core-metadata for more information.                                             
INFO     <html>                                                                                                                      
          <head>                                                                                                                     
           <title>400 '[email protected]:professionalaf/enterprise-utils.git' is an invalid value for Home-Page. Error: Invalid URI See 
         https://packaging.python.org/specifications/core-metadata for more information.</title>                                     
          </head>                                                                                                                    
          <body>                                                                                                                     
           <h1>400 '[email protected]:professionalaf/enterprise-utils.git' is an invalid value for Home-Page. Error: Invalid URI See    
         https://packaging.python.org/specifications/core-metadata for more information.</h1>                                        
           The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>               
         &#x27;[email protected]:professionalaf/enterprise-utils.git&#x27; is an invalid value for Home-Page. Error: Invalid URI See    
         https://packaging.python.org/specifications/core-metadata for more information.                                             
                                                                                                                                     
                                                                                                                                     
          </body>                                                                                                                    
         </html>

Which prompted me to adjust my publishToPypi options to include

publishToPypi: {
    distName: 'enterprise-utils',
    module: 'enterprise_utils',
    homepage: 'https://github.com/professionalaf/enterprise-utils',  // Added this
  },

Doing a successful twine upload with --verbose doesn't add any additional logging, so adding this flag should only assist with debugging errors

Golang: releases fail if there are no changes

We sometimes publish releases with no changes to the code. This is supported by all publishers, but the go publisher fails with the following error:

nothing to commit, working tree clean

Which makes sense.

The desired behavior in such cases is that a tag will still be applied and pushed.

Fails to report error during publib-maven

See: https://github.com/cdklabs/awscdk-asset-node-proxy-agent/actions/runs/4624066296/jobs/8230436553

Looks like publib-maven is failing with:

Error:  
Error:  Nexus Staging Rules Failure Report
Error:  ==================================
Error:  
Error:  Repository "softwareamazon-5342" failures
Error:    Rule "RepositoryWritePolicy" failures
Error:      * Artifact updating: Repository ='releases:Releases' does not allow updating artifact='/software/amazon/awscdk/cdk-asset-node-proxy-agent-v5/2.0.120/cdk-asset-node-proxy-agent-v5-2.0.120.jar'
Error:      * Artifact updating: Repository ='releases:Releases' does not allow updating artifact='/software/amazon/awscdk/cdk-asset-node-proxy-agent-v5/2.0.120/cdk-asset-node-proxy-agent-v5-2.0.120-sources.jar'
Error:      * Artifact updating: Repository ='releases:Releases' does not allow updating artifact='/software/amazon/awscdk/cdk-asset-node-proxy-agent-v5/2.0.120/cdk-asset-node-proxy-agent-v5-2.0.120-javadoc.jar'
Error:      * Artifact updating: Repository ='releases:Releases' does not allow updating artifact='/software/amazon/awscdk/cdk-asset-node-proxy-agent-v5/2.0.120/cdk-asset-node-proxy-agent-v5-2.0.120.pom'
Error:  
Error:  
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.651 s
[INFO] Finished at: 2023-04-19T01:04:06Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal org.sonatype.plugins:nexus-staging-maven-plugin:1.6.5:release (default-cli) on project dummy: Could not perform action: there are failing staging rules!: Staging rules failure! -> [Help 1]
Error:  
Error:  To see the full stack trace of the errors, re-run Maven with the -e switch.
Error:  Re-run Maven using the -X switch to enable full debug logging.
Error:  
Error:  For more information about the errors and possible solutions, please read the following articles:
Error:  [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

But the github action eventually reports success.

The reason is here:

publib/bin/publib-maven

Lines 262 to 268 in 3a4a114

 if [ ${PIPESTATUS[0]} -ne 0 ]; then 

 if cat ${release_output} | grep "does not allow updating artifact" | grep -q ".pom"; then 

 echo "⚠️ Artifact already published. Skipping" 

 else 

 error "Release failed" 

 fi 

 fi

We may be overindexing on the magic string "does not allow updating artifacts" because it is not true in this case that we are trying to publish the same artifact.

Support publishing maven packages to GitHub packages

This could be done by adding parallel options, but I think it makes the most sense to just add additional options that also apply to maven central with sensible defaults.

MAVEN_SERVER_ID : used in maven settings for credential lookup, defaults to ossrh
MAVEN_REPOSITORY_URL: used as the deployment repository. Not used for maven central

MAVEN_STAGING_PROFILE_ID would no longer be required when not using maven central
Signing is also not required for other repositories.

I don't believe the nexus staging plugin works for non-nexus base repositories, but I'm not very familiar with it.
I believe you can deploy to nexus without using it, but it's probably safer to have an alternate code flow that just uses the default maven deploy plugin.

This method should also make it possible to support CodeArtifact as was requested in #10

Golang Readme shows incorrect environment variable key

The README shows GITHUB_TOKEN as a required environment variable for publishing to golang, but the default Secret used in projen is GO_GITHUB_TOKEN (ref)

Reporting a vulnerability

Hello!

I hope you are doing well!

We are a security research team. Our tool automatically detected a vulnerability in this repository. We want to disclose it responsibly. GitHub has a feature called Private vulnerability reporting, which enables security research to privately disclose a vulnerability. Unfortunately, it is not enabled for this repository.

Can you enable it, so that we can report it?

Thanks in advance!

PS: you can read about how to enable private vulnerability reporting here: https://docs.github.com/en/code-security/security-advisories/repository-security-advisories/configuring-private-vulnerability-reporting-for-a-repository

Support publishing multi version golang modules from the same repo

Now that we implemented #27, supporting different versions per module shouldn't be far off.

	if [ ${PIPESTATUS[0]} -ne 0 ]; then
	if cat ${release_output} \| grep "does not allow updating artifact" \| grep -q ".pom"; then
	echo "⚠️ Artifact already published. Skipping"
	else
	error "Release failed"
	fi
	fi