cdklabs / publib Goto Github PK
View Code? Open in Web Editor NEWA unified toolchain for publishing libraries to popular package managers (formally jsii-release)
License: Apache License 2.0
A unified toolchain for publishing libraries to popular package managers (formally jsii-release)
License: Apache License 2.0
npx -p publib@latest publib-maven
Uses version [email protected]
env:
MAVEN_GPG_PRIVATE_KEY: ${{ secrets.MAVEN_GPG_PRIVATE_KEY }}
MAVEN_GPG_PRIVATE_KEY_PASSPHRASE: ${{ secrets.MAVEN_GPG_PRIVATE_KEY_PASSPHRASE }}
MAVEN_PASSWORD: ${{ secrets.MAVEN_PASSWORD }}
MAVEN_USERNAME: ${{ secrets.MAVEN_USERNAME }}
MAVEN_STAGING_PROFILE_ID: ${{ secrets.MAVEN_STAGING_PROFILE_ID }}
[INFO] Signer 'gpg' is signing 4 files
[DEBUG] 2.2.27
[GNUPG:] KEY_CONSIDERED 0E2DB99D44DEA2E3BE4A9D3326689A5EA6A5AFBF 2
[GNUPG:] BEGIN_SIGNING H10
[GNUPG:] PINENTRY_LAUNCHED 2145 curses 1.1.1 not a tty - - ? 1001/127 -
gpg: signing failed: No such file or directory
[GNUPG:] FAILURE sign 83918929
gpg: signing failed: No such file or directory
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.226 s
[INFO] Finished at: 2024-03-12T10:44:01Z
[INFO] ------------------------------------------------------------------------
Error: Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:3.2.0:sign-and-deploy-file (default-cli) on project standalone-pom: Exit code: 2 -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:3.2.0:sign-and-deploy-file (default-cli) on project standalone-pom: Exit code: 2
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2 (MojoExecutor.java:375)
at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute (MojoExecutor.java:351)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:171)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:163)
This command is being run with a GitHub workflow: https://github.com/mongodb/awscdk-resources-mongodbatlas/blob/main/.github/workflows/release.yml#L120.
This publish workflow was working without issues in the past, with the last successful release on the 6th of February. I can see that a new version of maven-gpg-plugin (3.2.0) was release on the 8th of March, not sure if this had an impact.
When running the publish command locally (on macOS) I was requested to enter the GPG passphrase even when having the MAVEN_GPG_PRIVATE_KEY_PASSPHRASE
env variable defined.
Any inputs or possible workarounds would be greatly appreciated.
For example, for github.com/aws/constructs-go
, Instead of v3.3.1
the tag should be constructs/v3.3.1
.
This technically allows different modules in the same repo to be tagged with different versions.
Reference: https://research.swtch.com/vgo-module#multiple-module_repositories
In the major subdirectory convention, v2/ contains the module “my/thing/v2”. A natural extension is to allow subdirectories not named for major versions. For example, we could add a blue/ subdirectory that contains the module “my/thing/blue”, confirmed by a blue/go.mod file with that module path. In this case, the source control commit tags addressing that module would take the form blue/v1.x.x. Similarly, the tag blue/v2.x.x would address the blue/v2/ subdirectory. The existence of the blue/go.mod file excludes the blue/ tree from the outer my/thing module.
there's no way to dry-run publish npm packages, as this options is available in publib-npm
I believe the existing implementation can be extended to conditionally handle AWS CodeArtifact and use AWS CLI, like the NPM implementation, to get the AWS CodeArtifact temporary authorization token for use as MAVEN_PASSWORD
.
I traced an issue I was having in projen (v0.43.0) back to this project. When attempting to build an AWS Construct Library published to AWS CodeArtifact to both NPM and Maven, only NPM was supported. I was unable to get Maven to function and thought it best to dig deeper to bring first-class support forward.
GitHub Packages support (#11, #12) provides a great foundation to extend to fully address AWS CodeArtifact support (#10)
Once implemented, an example GitHub Actions publish step could look like:
- name: Publish package
run: npx -p jsii-release jsii-release-maven
env:
MAVEN_SERVER_ID: codeartifact
MAVEN_USERNAME: ${{ github.actor }}
MAVEN_REPOSITORY_URL: "https://my_domain-111122223333.d.codeartifact.us-west-2.amazonaws.com/maven/my_repo/'"
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
We could borrow from the NPM implementation (https://github.com/cdklabs/jsii-release/blob/master/bin/jsii-release-npm#L23-L38) to conditionally assign environment variable MAVEN_PASSWORD
to an AWS CodeArtifact authorization token using AWS CLI if the MAVEN_REPOSITORY_URL
is defined and matches an AWS CodeArtifact pattern. This then would be picked up by \${env.MAVEN_USERNAME}
within mvn-settings.xml
creation steps.
As I upgraded the repository with PNPM
as package manager, the publish to npm registrery fails with the following message:
npm notice Publishing to https://registry.npmjs.org/ with tag latest and default access
npm ERR! code E415
npm ERR! 415 Unsupported Media Type - PUT https://registry.npmjs.org/<mypackage> - Hard link is not allowed
The README shows GITHUB_TOKEN
as a required environment variable for publishing to golang, but the default Secret used in projen is GO_GITHUB_TOKEN
(ref)
We are seeing quite a lot of timeouts when publishing to maven central:
https://github.com/aws/constructs/runs/1824718480?check_suite_focus=true
The timeout can be increased as described here:
https://issues.sonatype.org/plugins/servlet/mobile#issue/OSSRH-63605
During a projen "Publish to Pypi" action I got the following error
Uploading enterprise_utils-0.0.0-py3-none-any.whl
25l
0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/30.3 kB • --:-- • ?
0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0.0/30.3 kB • --:-- • ?
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.3/30.3 kB • 00:00 • 84.9 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.3/30.3 kB • 00:00 • 84.9 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.3/30.3 kB • 00:00 • 84.9 MB/s
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 30.3/30.3 kB • 00:00 • 84.9 MB/s
25hWARNING Error during upload. Retry with the --verbose option for more details.
Error: Process completed with exit code 1.
There was no option for --verbose
, so I included publib
as a project dependency, adjusted bin/publib-pypi
to use
python3 -m twine upload --verbose --skip-existing *
Here I got a more helpful error of
INFO Response from https://upload.pypi.org/legacy/:
400 '[email protected]:professionalaf/enterprise-utils.git' is an invalid value for Home-Page. Error: Invalid URI See
https://packaging.python.org/specifications/core-metadata for more information.
INFO <html>
<head>
<title>400 '[email protected]:professionalaf/enterprise-utils.git' is an invalid value for Home-Page. Error: Invalid URI See
https://packaging.python.org/specifications/core-metadata for more information.</title>
</head>
<body>
<h1>400 '[email protected]:professionalaf/enterprise-utils.git' is an invalid value for Home-Page. Error: Invalid URI See
https://packaging.python.org/specifications/core-metadata for more information.</h1>
The server could not comply with the request since it is either malformed or otherwise incorrect.<br/><br/>
'[email protected]:professionalaf/enterprise-utils.git' is an invalid value for Home-Page. Error: Invalid URI See
https://packaging.python.org/specifications/core-metadata for more information.
</body>
</html>
Which prompted me to adjust my publishToPypi
options to include
publishToPypi: {
distName: 'enterprise-utils',
module: 'enterprise_utils',
homepage: 'https://github.com/professionalaf/enterprise-utils', // Added this
},
Doing a successful twine upload with --verbose
doesn't add any additional logging, so adding this flag should only assist with debugging errors
The endpoint for new maven repositories is now:
This means for new maven repositories:
MAVEN_ENDPOINT
must be set to https://s01.oss.sonatype.org
, otherwise jsii-release-maven will fail with a "403 Forbidden" errorThought this would be useful to mention in the docs.
In many of our projen projects, which uses jsii-release
to do the publishing, we see the following maven errors:
Waiting for operation to complete...
Dec 15, 2021 12:38:42 AM com.sun.jersey.api.client.ClientResponse getEntity
..............................
SEVERE: A message body reader for Java class com.sonatype.nexus.staging.api.dto.StagingProfileRepositoryDTO, and Java type class com.sonatype.nexus.staging.api.dto.StagingProfileRepositoryDTO, and MIME media type text/html was not found
Dec 15, 2021 12:38:42 AM com.sun.jersey.api.client.ClientResponse getEntity
SEVERE: The registered message body readers compatible with the MIME media type are:
*/* ->
com.sun.jersey.core.impl.provider.entity.FormProvider
com.sun.jersey.core.impl.provider.entity.StringProvider
com.sun.jersey.core.impl.provider.entity.ByteArrayProvider
com.sun.jersey.core.impl.provider.entity.FileProvider
com.sun.jersey.core.impl.provider.entity.InputStreamProvider
com.sun.jersey.core.impl.provider.entity.DataSourceProvider
com.sun.jersey.core.impl.provider.entity.XMLJAXBElementProvider$General
com.sun.jersey.core.impl.provider.entity.ReaderProvider
com.sun.jersey.core.impl.provider.entity.DocumentProvider
com.sun.jersey.core.impl.provider.entity.SourceProvider$StreamSourceReader
com.sun.jersey.core.impl.provider.entity.SourceProvider$SAXSourceReader
com.sun.jersey.core.impl.provider.entity.SourceProvider$DOMSourceReader
com.sun.jersey.core.impl.provider.entity.XMLRootElementProvider$General
com.sun.jersey.core.impl.provider.entity.XMLListElementProvider$General
com.sun.jersey.core.impl.provider.entity.XMLRootObjectProvider$General
com.sun.jersey.core.impl.provider.entity.EntityHolderReader
com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider
At first glance this feels like some configuration error with the project, but the thing is that it seems this is sporadic, since subsequent executions succeed fine.
Putting this here so we can keep track of it, it might be worth adding some more advanced retry mechanism for maven publishing.
This could be done by adding parallel options, but I think it makes the most sense to just add additional options that also apply to maven central with sensible defaults.
ossrh
MAVEN_STAGING_PROFILE_ID would no longer be required when not using maven central
Signing is also not required for other repositories.
I don't believe the nexus staging plugin works for non-nexus base repositories, but I'm not very familiar with it.
I believe you can deploy to nexus without using it, but it's probably safer to have an alternate code flow that just uses the default maven deploy plugin.
This method should also make it possible to support CodeArtifact as was requested in #10
Hi there,
As #1107 was introduced by a new minor update to a maven task run in the release process I was wondering what you would think of (optionally?) pinning all used dependencies? Package managers don't seem to rapidly change their publishing workflows so I would assume what works now should also work in the future, so there is not necessarily a need to stay on top of all dependency updates. I'd love to have an option to limit the impact of dependency updates so that my main workflows can remain stable.
I'm more than happy to do a PR if you feel like this is a valuable addition to the project.
publishing to npm only works for NPM_TOKEN
that match the authToken
format, it doesn't allow for repositories that only allow for _auth
to be used in npmrc
files.
I've opened a PR for this #821
We currently assume the DIR
passed to the script must contain subdirectories where go modules are located. This is true for the mono repo use-case, but for single module repo's this is an author choice.
For example, our own jsii-runtime-go
doesn't follow this pattern - and we need to support it.
Hello!
I hope you are doing well!
We are a security research team. Our tool automatically detected a vulnerability in this repository. We want to disclose it responsibly. GitHub has a feature called Private vulnerability reporting, which enables security research to privately disclose a vulnerability. Unfortunately, it is not enabled for this repository.
Can you enable it, so that we can report it?
Thanks in advance!
PS: you can read about how to enable private vulnerability reporting here: https://docs.github.com/en/code-security/security-advisories/repository-security-advisories/configuring-private-vulnerability-reporting-for-a-repository
My CDK project can't release to Maven Central due to the following error in jsii-release-maven
.
Error: The goal you specified requires a project to execute but there is no POM in this directory (/__w/cdk-datadog-resources/cdk-datadog-resources/dist/java). Please verify you invoked Maven from the correct directory. -> [Help 1]
Am I making a mistake in the settings?
This project created by projen.
https://github.com/NomadBlacky/cdk-datadog-resources/runs/2718228766
Run npx -p jsii-release@latest jsii-release-maven
npx -p jsii-release@latest jsii-release-maven
shell: sh -e {0}
env:
MAVEN_GPG_PRIVATE_KEY: ***
MAVEN_GPG_PRIVATE_KEY_PASSPHRASE: ***
MAVEN_PASSWORD: ***
MAVEN_USERNAME: ***
MAVEN_STAGING_PROFILE_ID: ***
npx: installed 8 in 1.873s
Importing GPG key...
gpg: keyring `/tmp/tmp.2MCJZZF6iy/secring.gpg' created
gpg: keyring `/tmp/tmp.2MCJZZF6iy/pubring.gpg' created
gpg: key C6744B5E: secret key imported
gpg: /tmp/tmp.2MCJZZF6iy/trustdb.gpg: trustdb created
gpg: key C6744B5E: public key "Takumi Kadowaki <[email protected]>" imported
gpg: key 44A2BFDA: secret key imported
gpg: key 44A2BFDA: public key "Takumi Kadowaki <***@gmail.com>" imported
gpg: Total number processed: 2
gpg: imported: 2 (RSA: 2)
gpg: secret keys read: 2
gpg: secret keys imported: 2
gpg_key_id=3CF60FF5C6744B5E
6458ADFB44A2BFDA
📦 Publishing to Maven Central
/tmp/tmp.gaJGOpScSL
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Preparing repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/apache-maven
Java version: 1.8.0_292, vendor: Amazon.com Inc., runtime: /usr/lib/jvm/java-1.8.0-amazon-corretto.x86_64/jre
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "5.4.0-1047-azure", arch: "amd64", family: "unix"
[DEBUG] Created new class realm maven.api
[DEBUG] Importing foreign packages into class realm maven.api
[DEBUG] Imported: javax.annotation.* < plexus.core
[DEBUG] Imported: javax.annotation.security.* < plexus.core
[DEBUG] Imported: javax.enterprise.inject.* < plexus.core
[DEBUG] Imported: javax.enterprise.util.* < plexus.core
[DEBUG] Imported: javax.inject.* < plexus.core
[DEBUG] Imported: org.apache.maven.* < plexus.core
[DEBUG] Imported: org.apache.maven.artifact < plexus.core
[DEBUG] Imported: org.apache.maven.classrealm < plexus.core
[DEBUG] Imported: org.apache.maven.cli < plexus.core
[DEBUG] Imported: org.apache.maven.configuration < plexus.core
[DEBUG] Imported: org.apache.maven.exception < plexus.core
[DEBUG] Imported: org.apache.maven.execution < plexus.core
[DEBUG] Imported: org.apache.maven.execution.scope < plexus.core
[DEBUG] Imported: org.apache.maven.lifecycle < plexus.core
[DEBUG] Imported: org.apache.maven.model < plexus.core
[DEBUG] Imported: org.apache.maven.monitor < plexus.core
[DEBUG] Imported: org.apache.maven.plugin < plexus.core
[DEBUG] Imported: org.apache.maven.profiles < plexus.core
[DEBUG] Imported: org.apache.maven.project < plexus.core
[DEBUG] Imported: org.apache.maven.reporting < plexus.core
[DEBUG] Imported: org.apache.maven.repository < plexus.core
[DEBUG] Imported: org.apache.maven.rtinfo < plexus.core
[DEBUG] Imported: org.apache.maven.settings < plexus.core
[DEBUG] Imported: org.apache.maven.toolchain < plexus.core
[DEBUG] Imported: org.apache.maven.usability < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.* < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.authentication < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.authorization < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.events < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.observers < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.proxy < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.repository < plexus.core
[DEBUG] Imported: org.apache.maven.wagon.resource < plexus.core
[DEBUG] Imported: org.codehaus.classworlds < plexus.core
[DEBUG] Imported: org.codehaus.plexus.* < plexus.core
[DEBUG] Imported: org.codehaus.plexus.classworlds < plexus.core
[DEBUG] Imported: org.codehaus.plexus.component < plexus.core
[DEBUG] Imported: org.codehaus.plexus.configuration < plexus.core
[DEBUG] Imported: org.codehaus.plexus.container < plexus.core
[DEBUG] Imported: org.codehaus.plexus.context < plexus.core
[DEBUG] Imported: org.codehaus.plexus.lifecycle < plexus.core
[DEBUG] Imported: org.codehaus.plexus.logging < plexus.core
[DEBUG] Imported: org.codehaus.plexus.personality < plexus.core
[DEBUG] Imported: org.codehaus.plexus.util.xml.Xpp3Dom < plexus.core
[DEBUG] Imported: org.codehaus.plexus.util.xml.pull.XmlPullParser < plexus.core
[DEBUG] Imported: org.codehaus.plexus.util.xml.pull.XmlPullParserException < plexus.core
[DEBUG] Imported: org.codehaus.plexus.util.xml.pull.XmlSerializer < plexus.core
[DEBUG] Imported: org.eclipse.aether.* < plexus.core
[DEBUG] Imported: org.eclipse.aether.artifact < plexus.core
[DEBUG] Imported: org.eclipse.aether.collection < plexus.core
[DEBUG] Imported: org.eclipse.aether.deployment < plexus.core
[DEBUG] Imported: org.eclipse.aether.graph < plexus.core
[DEBUG] Imported: org.eclipse.aether.impl < plexus.core
[DEBUG] Imported: org.eclipse.aether.installation < plexus.core
[DEBUG] Imported: org.eclipse.aether.internal.impl < plexus.core
[DEBUG] Imported: org.eclipse.aether.metadata < plexus.core
[DEBUG] Imported: org.eclipse.aether.repository < plexus.core
[DEBUG] Imported: org.eclipse.aether.resolution < plexus.core
[DEBUG] Imported: org.eclipse.aether.spi < plexus.core
[DEBUG] Imported: org.eclipse.aether.transfer < plexus.core
[DEBUG] Imported: org.eclipse.aether.version < plexus.core
[DEBUG] Imported: org.fusesource.jansi.* < plexus.core
[DEBUG] Imported: org.slf4j.* < plexus.core
[DEBUG] Imported: org.slf4j.event.* < plexus.core
[DEBUG] Imported: org.slf4j.helpers.* < plexus.core
[DEBUG] Imported: org.slf4j.spi.* < plexus.core
[DEBUG] Populating class realm maven.api
[INFO] Error stacktraces are turned on.
[DEBUG] Message scheme: color
[DEBUG] Message styles: debug info warning error success failure strong mojo project
[DEBUG] Reading global settings from /usr/local/apache-maven/conf/settings.xml
[DEBUG] Reading user settings from /tmp/tmp.gaJGOpScSL/mvn-settings.xml
[DEBUG] Reading global toolchains from /usr/local/apache-maven/conf/toolchains.xml
[DEBUG] Reading user toolchains from /root/.m2/toolchains.xml
[DEBUG] Using local repository at /root/.m2/repository
[DEBUG] Using manager EnhancedLocalRepositoryManager with priority 10.0 for /root/.m2/repository
[INFO] Scanning for projects...
[DEBUG] Extension realms for project org.apache.maven:standalone-pom:pom:1: (none)
[DEBUG] Looking up lifecycle mappings for packaging pom from ClassRealm[plexus.core, parent: null]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.061 s
[INFO] Finished at: 2021-06-01T11:47:19Z
[INFO] ------------------------------------------------------------------------
Error: The goal you specified requires a project to execute but there is no POM in this directory (/__w/cdk-datadog-resources/cdk-datadog-resources/dist/java). Please verify you invoked Maven from the correct directory. -> [Help 1]
org.apache.maven.lifecycle.MissingProjectException: The goal you specified requires a project to execute but there is no POM in this directory (/__w/cdk-datadog-resources/cdk-datadog-resources/dist/java). Please verify you invoked Maven from the correct directory.
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:85)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Error:
Error:
Error: For more information about the errors and possible solutions, please read the following articles:
Error: [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MissingProjectException
Error: Process completed with exit code 1.
We sometimes publish releases with no changes to the code. This is supported by all publishers, but the go publisher fails with the following error:
nothing to commit, working tree clean
Which makes sense.
The desired behavior in such cases is that a tag will still be applied and pushed.
When you want to publish scoped packages to npm, you need to run npm publish --access public
(https://docs.npmjs.com/configuring-your-npm-client-with-your-organization-settings)
It would be great to support this in jsii-release.
NPM package can't be published with publib
when 2FA is configured in the target NPM account.
npm publish
command used in publib-npm
(source) requires an --opt
flag when 2FA is configured. There's no equivalent env variable according to NPM docs.
I was able to publish an NPM package by editing publib
sources in node_modules
. I added the --otp
flag in node_modules/.bin/publib-npm
:
npm publish ${tag} ${access} ${file} --otp <my_temp_code> 2>&1 | tee ${log}
An NPM_OTP_TOKEN
env variable could be supported by publib
and used as the --otp
flag value in the npm publish
command. This won't be the optimal solution, though, as the OTP token is short-lived and will have to be updated right before running the publib
command. A better solution would be to somehow allow npm publish
to run in interactive mode and prompt the user for the OTP token. (see NPM OTP option docs).
Now that we implemented #27, supporting different versions per module shouldn't be far off.
For now, I want to publish a package that is created by jsii to my hosted Nexus. However, I must config the NPM_TOKEN, I am able to publish successfully. But for other repositories, I use npm config _auth
to do it.
I hope we can support npm config _auth
at the same time. Because in other repositories, I use _auth
to do the authentication for the private nexus repository. Due to I use aws-cdk to deploy some AWS resources, I want to use products around cdklab as well. But the experience isn't good. 😢
npx publib-npm throws the message "The system cannot find path specified" on Windows.
npx works with other node modules like jest.
Is there any config that needs to be done on Windows?
thanks!
If you use a tool like Verdaccio to host a proxy npm registry on your own computer and try setting the environment variable NPM_REGISTRY to localhost:XXXX
when invoking publib-npm
, the command fails because it tries publishing to https://localhost:4872
instead of http://localhost:4872
. Whether or not we connect to the registry over HTTPS should be configurable.
I'd like to use npx publib-npm
in the pipeline with AWS CodeArtifact, however, it is not using credentials from ~/.npmrc
Successfully configured npm to use AWS CodeArtifact repository https://xxx-xxx.d.codeartifact.us-east-1.amazonaws.com/npm/lib/
Login expires in 12 hours at 2023-03-22 23:06:25-04:00
Then error with publish
npx publib-npm
NPM_TOKEN is required
Native npm publish works properly, using auth from ~/.npmrc
npm publish dist/js/lib@${version}.jsii.tgz
With NPM_TOKEN and NPM_REGISTRY variables for the CodeArtifact registry, it errors out with
npx publib-npm
Unable to locate credentials. You can configure credentials by running "aws configure".
```
jsii-release
I want to publish my CDK constructs built with JSII (NPM/PyPI/Maven) to AWS CodeArtifact - and I've now succesfully done it with NPM+PyPI 👍
The maven setup seems quite complex so I haven't (at least yet) tried to figure it out myself because my very limited knowledge of Maven and Java-ecosystem
Due to above reason I can't be sure, but to me it seems that the following hardcoded values in bin/jsii-release-maven
will most probably prevent publishing to CodeArtifact:
Basically the solution should be similar as with npm and with pypi (see my PR): Rely on environment variables (which work well in CI-environment).
This is a 🚀 Feature Request
Publib throws an error if someone tries to publish Go code to a non-github.com repository, such as GitHub enterprise (github.mycompany.com) or other locations like Bitbucket: https://github.com/cdklabs/publib/blob/main/src/targets/go.ts#L262
It would be nice to have the option to publish to GitHub enterprise and Bitbucket repos in addition to github.com for people that want to publish their Go code to internal repositories.
When generating the settings.xml
file maven in publish-mvn.sh
escape the password field to avoid illegal xml characters.
Tags are not being created correctly for the awscdkapigatewayv2 alpha modules. The v2 is being stripped from the name so instead of awscdkapigatewayv2alpha/v2.0.0-rc.24 we are getting awscdkapigatewayalpha/v2/v2.0.0-rc.24
Suggestion: would be great to have this tool as a backstage plugin - https://backstage.io/plugins
Maven publishing is creating staging repositories even if the package version being released already exists in Maven. If there are a lot of such concurrent requests, this leads to Maven blocking publishing credentials since this impacts their cleanup process and also customers since it becomes a noisy neighbor issue.
We need to fix our publishing script to check for existing version before starting the publishing process.
Related issue with Maven: https://issues.sonatype.org/browse/OSSRH-94655
AWS CodeArtifact generates the NPM_REGISTRY
with a trailing /
:
And the jsii-release-npm
will generate NPM_REGISTRY
with double slash in .npmrc
which returns the Unable to authenticate error
$ NPM_REGISTRY='pahud-112233445566.d.codeartifact.ap-northeast-1.amazonaws.com/npm/aws-repo/' NPM_TOKEN=`aws codeartifact get-authorization-token --domain pahud --domain-owner 903779448426 --query authorizationToken --output text` node_modules/jsii-release/bin/jsii-release-npm
npm notice
npm notice 📦 [email protected]
npm notice === Tarball Contents ===
npm notice 19.3kB .jsii
npm notice 11.4kB LICENSE
npm notice 1.2kB .projenrc.js
npm notice 3.8kB lib/index.js
npm notice 1.7kB test/index.test.js
npm notice 1.0kB test/integ.api.js
npm notice 1.3kB .eslintrc.json
npm notice 2.4kB package.json
npm notice 19B version.json
npm notice 1.6kB README.md
npm notice 346B lib/index.d.ts
npm notice 31B test/index.test.d.ts
npm notice 11B test/integ.api.d.ts
npm notice 397B .mergify.yml
npm notice 339B .github/workflows/build.yml
npm notice 1.3kB .github/workflows/release.yml
npm notice === Tarball Details ===
npm notice name: cdk-codeartifact-demo
npm notice version: 0.0.0
npm notice package size: 12.6 kB
npm notice unpacked size: 46.1 kB
npm notice shasum: eee847e49c128493f884a17e8bf238d500c653d2
npm notice integrity: sha512-XH+b7+/PKCedZ[...]wsLe6ajbdlCVw==
npm notice total files: 16
npm notice
npm ERR! code E401
npm ERR! Unable to authenticate, need: Bearer realm="pahud/aws-repo", Basic realm="pahud/aws-repo"
npm ERR! A complete log of this run can be found in:
npm ERR! /Users/pahud/.npm/_logs/2020-06-12T02_32_35_628Z-debug.log
ERROR
A quick fix is to always remove the trailing slash with bash substitution like this
echo "//${NPM_REGISTRY%%/}/:_authToken=${NPM_TOKEN}" > ~/.npmrc
And it will work with both AWS CodeArtifact as well as npmjs.
See: https://github.com/cdklabs/awscdk-asset-node-proxy-agent/actions/runs/4624066296/jobs/8230436553
Looks like publib-maven
is failing with:
Error:
Error: Nexus Staging Rules Failure Report
Error: ==================================
Error:
Error: Repository "softwareamazon-5342" failures
Error: Rule "RepositoryWritePolicy" failures
Error: * Artifact updating: Repository ='releases:Releases' does not allow updating artifact='/software/amazon/awscdk/cdk-asset-node-proxy-agent-v5/2.0.120/cdk-asset-node-proxy-agent-v5-2.0.120.jar'
Error: * Artifact updating: Repository ='releases:Releases' does not allow updating artifact='/software/amazon/awscdk/cdk-asset-node-proxy-agent-v5/2.0.120/cdk-asset-node-proxy-agent-v5-2.0.120-sources.jar'
Error: * Artifact updating: Repository ='releases:Releases' does not allow updating artifact='/software/amazon/awscdk/cdk-asset-node-proxy-agent-v5/2.0.120/cdk-asset-node-proxy-agent-v5-2.0.120-javadoc.jar'
Error: * Artifact updating: Repository ='releases:Releases' does not allow updating artifact='/software/amazon/awscdk/cdk-asset-node-proxy-agent-v5/2.0.120/cdk-asset-node-proxy-agent-v5-2.0.120.pom'
Error:
Error:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.651 s
[INFO] Finished at: 2023-04-19T01:04:06Z
[INFO] ------------------------------------------------------------------------
Error: Failed to execute goal org.sonatype.plugins:nexus-staging-maven-plugin:1.6.5:release (default-cli) on project dummy: Could not perform action: there are failing staging rules!: Staging rules failure! -> [Help 1]
Error:
Error: To see the full stack trace of the errors, re-run Maven with the -e switch.
Error: Re-run Maven using the -X switch to enable full debug logging.
Error:
Error: For more information about the errors and possible solutions, please read the following articles:
Error: [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
But the github action eventually reports success.
The reason is here:
Lines 262 to 268 in 3a4a114
We may be overindexing on the magic string "does not allow updating artifacts" because it is not true in this case that we are trying to publish the same artifact.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.