Giter Club home page Giter Club logo

cloudsmith-cli's Introduction

Cloudsmith Command Line Interface (CLI)

Latest Version @ Cloudsmith Python Versions PyPI Version CircleCI Maintainability Test Coverage Code style: black

The Cloudsmith Command Line Interface (CLI) is a Python 3 text-based interface to the API. This allows users, machines and other services to access and integrate smoothly with Cloudsmith without requiring explicit plugins or tools. Be awesome. Automate Everything.

The following asciinema video demonstrates some of the CLI commands: asciicast

We also have a demo video on YouTube:

You can also read our blog article that introduced the first version of the CLI and the Cloudsmith RESTful API.

Changelog

Please see the changelog for the list of changes by version. The current version is displayed in the PyPi badge at the top.

Features

The CLI currently supports the following commands (and sub-commands):

  • check: Check rate limits and service status.
  • copy|cp: Copy a package to another repository.
  • delete|rm: Delete a package from a repository.
  • dependencies|deps: List direct (non-transitive) dependencies for a package.
  • docs: Launch the help website in your browser.
  • entitlements|ents: Manage the entitlements for a repository.
    • create|new: Create a new entitlement in a repository.
    • delete|rm: Delete an entitlement from a repository.
    • list|ls: List entitlements for a repository.
    • refresh: Refresh an entitlement in a repository.
    • sync: Sync entitlements from another repository.
    • update|set: Update (patch) a entitlement in a repository.
  • help: Display the delightful help message and exit.
  • list|ls: List distros, packages, repos and entitlements.
    • dependencies|deps List direct (non-transitive) dependencies for a package.
    • distros: List available distributions.
    • entitlements|ents: List entitlements for a repository.
    • packages: List packages for a repository. (Aliases repos list)
    • repos: List repositories for a namespace (owner).
  • login|token: Retrieve your API authentication token/key via login.
  • metrics: Metrics and statistics for a repository.
    • tokens: Retrieve bandwidth usage for entitlement tokens.
    • packages: Retrieve package usage for repository.
  • move|mv: Move (promote) a package to another repo.
  • push|upload: Push (upload) a new package to a repository.
    • alpine: Push (upload) a new Alpine package upstream.
    • cargo: Push (upload) a new Cargo package upstream.
    • composer: Push (upload) a new Composer package upstream.
    • cocoapods: Push (upload) a new CocoaPods package upstream.
    • conan: Push (upload) a new Conan (C++) package upstream.
    • cran: Push (upload) a new R/CRAN package upstream.
    • deb: Push (upload) a new Debian package upstream.
    • docker: Push (upload) a new Docker image upstream.
    • go: Push (upload) a new Go module upstream.
    • helm: Push (upload) a new Helm package upstream.
    • luarocks: Push (upload) a new Lua module upstream.
    • maven: Push (upload) a new Maven package upstream.
    • npm: Push (upload) a new Npm package upstream.
    • nuget: Push (upload) a new NuGet package upstream.
    • python: Push (upload) a new Python package upstream.
    • raw: Push (upload) a new Raw package upstream.
    • rpm: Push (upload) a new RedHat package upstream.
    • ruby: Push (upload) a new Ruby package upstream.
    • terraform: Push (upload) a new Terraform package upstream.
    • vagrant: Push (upload) a new Vagrant package upstream.
  • quarantine|block: Manage quarantined packages in a repository.
    • add: Add a package to quarantine.
    • remove|rm|restore: Add a package to quarantine.
  • quota: Quota limits and history for a organisation.
    • limits: Display the Quota (bandwidth & storage usage/limits) for a specific organisation.
    • history: Display the Quota History (upload, download, and storage usage/limits) for a specific organisation.
  • repositories|repos: Manage repositories.
    • create|new: Create a new repository in a namespace.
    • get|list|ls: List repositories for a user, in a namespace or get details for a specific repository.
    • update: Update a repository in a namespace.
    • delete|rm: Delete a repository from a namespace.
  • resync: Resynchronise a package in a repository.
  • status: Get the synchronisation status for a package.
  • tags: Manage the tags for a package in a repository.
    • add: Add tags to a package in a repository.
    • clear: Clear all existing (non-immutable) tags from a package in a repository.
    • list|ls: List tags for a package in a repository.
    • remove|rm: Remove tags from a package in a repository.
    • replace: Replace all existing (non-immutable) tags on a package in a repository.
  • whoami: Retrieve your current authentication status.

Installation

You can install the latest CLI application from:

The simplest way is to use pip, such as:

pip install --upgrade cloudsmith-cli

Or you can get the latest pre-release version from Cloudsmith:

pip install --upgrade cloudsmith-cli --extra-index-url=https://dl.cloudsmith.io/public/cloudsmith/cli/python/index/

Configuration

There are two configuration files used by the CLI:

  • config.ini: For non-credentials configuration.
  • credentials.ini: For credentials (authentication) configuration.

By default, the CLI will look for these in the following locations:

  • The current working directory.
  • A directory called cloudsmith in the OS-defined application directory. For example:
    • Linux:
      • $HOME/.config/cloudsmith
      • $HOME/.cloudsmith
    • Mac OS:
      • $HOME/Library/Application Support/cloudsmith
      • $HOME/.cloudsmith
    • Windows:
      • C:\Users\<user>\AppData\Local\cloudsmith (Win7+, not roaming)
      • C:\Users\<user>\AppData\Roaming\cloudsmith (Win7+, roaming)
      • C:\Documents and Settings\<user>\Application Data\cloudsmith (WinXP, not roaming)
      • C:\Documents and Settings\<user>\Local Settings\Application Data\cloudsmith (WinXP, roaming)
      • C:\Documents and Settings\<user>\.cloudsmith

Both configuration files use the simple INI format, such as:

[default]
api_key=1234567890abcdef1234567890abcdef

Non-Credentials (config.ini)

See the default config in GitHub:

You can specify the following configuration options:

  • api_host: The API host to connect to.
  • api_proxy: The API proxy to connect through.
  • api_ssl_verify: Whether or not to use SSL verification for requests.
  • api_user_agent: The user agent to use for requests.

Credentials (credentials.ini)

See the default config in GitHub:

You can specify the following configuration options:

  • api_key: The API key for authenticating with the API.

Getting Your API Key

You'll need to provide authentication to Cloudsmith for any CLI actions that result in accessing private data or making changes to resources (such as pushing a new package to a repository)..

With the CLI this is simple to do. You can retrieve your API key using the cloudsmith login command:

cloudsmith login
Login: [email protected]
Password:
Repeat for confirmation:

Note: Please ensure you use your email for the 'Login' prompt and not your user slug/identifier.

The resulting output looks something like:

Retrieving API token for '[email protected]' ... OK
Your API token is: 1234567890abcdef1234567890abcdef

Once you have your API key you can then put this into your credentials.ini, use it as an environment variable export CLOUDSMITH_API_KEY=your_key_here or pass it to the CLI using the -k your_key_here flag.

For convenience the CLI will ask you if you want to install the default configuration files, complete with your API key, if they don't already exist. Say 'y' or 'yes' to create the configuration files.

If the configuration files already exist, you'll have to manually put the API key into the configuration files, but the CLI will print out their locations.

Uploading Packages

Although native uploads, i.e. those supported by the native ecosystem of a package format, are often preferred; it's easy to publish with the Cloudsmith CLI too!

For example, if you wanted to upload a Debian package, you can do it in one-step. Assuming you have a package filename libxml2-2.9.4-2.x86_64.deb, representing libxml 2.9.4, for the Ubuntu 16.04 distribution (which has a cloudsmith identifier of ubuntu/xenial):

cloudsmith push deb your-account/your-repo/ubuntu/xenial libxml2-2.9.4-2.x86_64.deb

Want to know how to do it with another packaging format? Easy, just ask for help:

cloudsmith push rpm --help

Contributing

Yes! Please do contribute, this is why we love open source. Please see CONTRIBUTING for contribution guidelines when making code changes or raising issues for bug reports, ideas, discussions and/or questions (i.e. help required).

License

Copyright 2018 Cloudsmith Ltd

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.

 http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

EOF

This quality product was brought to you by Cloudsmith and the fine folks who have contributed.

cloudsmith-cli's People

Contributors

alancarson avatar apoclyps avatar bartoszblizniak avatar chrisimcevoy avatar da-ar avatar danmckinney avatar davids-cloud avatar dbmurf93 avatar dgonzalez avatar dil-dvinnai avatar dkatavic avatar dmeecs avatar jackgibson1 avatar jesserhoads-pd avatar leeoc avatar lskillen avatar mcclory avatar mrtam avatar paddycarey avatar sanderr avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cloudsmith-cli's Issues

Pushing nuget package raise an error: Failed to upload file! (status: 400 - Bad Request)

Environment:
OS: windows 11
python 3.9.6 is installed

cloudsmith --version

Versions:
CLI Package Version: 1.1.1
API Package Version: 2.0.7

I try to upload nuget package to cloudsmith with the command:

cloudsmith push nuget -k abc123  myorg/o2p mypackage.6.2.0.nupkg

but i get error

Failed to upload file! (status: 400 - Bad Request) 

although key is valid and Uploading is 100%
The output is:

Checking nuget package upload parameters ... OK
Checking mypackage.6.2.0.nupkg file upload parameters ... OK
Requesting file upload for mypackage.6.2.0.nupkg ... OK
Uploading mypackage.6.2.0.nupkg:  [####################################]  100%
ERROR
Failed to upload file! (status: 400 - Bad Request)

What is wrong and how to fix the error?

Update
I traced the request/response using fiddler.
The requests in order are: 204->204->202->400
It seems the error 400 is from AmazonS3 due to : Unsupported Authorization Type error

The details of 400 error:

The request

POST https://cloudsmith-package-uploads-prd.s3-accelerate.amazonaws.com/ HTTP/1.1
Host: cloudsmith-package-uploads-prd.s3-accelerate.amazonaws.com
user-agent: cloudsmith-cli/odata cli:1.1.1 api:2.0.7
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
content-type: multipart/form-data; boundary=3ec17d64043349648469559a823ad667
Content-Length: 1147196
Authorization: Basic abcxyz=

.... rest file binary contents

The response

HTTP/1.1 400 Bad Request
Content-Type: application/xml
Transfer-Encoding: chunked
Connection: keep-alive
x-amz-request-id: THSXS5KPHX6JGZD0
x-amz-id-2: bbdLyfZOA/4szLDNZxNHCDVDisxUA75obiiiLrhWGzfvIw+btmBWzsu87vTvrf0eBgLWd0ClD4Q=
Date: Sat, 04 Nov 2023 18:50:26 GMT
Server: AmazonS3
X-Cache: Error from cloudfront
Via: 1.1 70c565ac15f71f0aa26aecd3763d4108.cloudfront.net (CloudFront)
X-Amz-Cf-Pop: MRS52-P1
X-Amz-Cf-Id: tWb8FYKnrSMCFPu0EXBjxnBu6ZeIzSCd_CW7VTlN7dRKcFnjR5LVOQ==

166
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>InvalidArgument</Code><Message>Unsupported Authorization Type</Message><ArgumentName>Authorization</ArgumentName><ArgumentValue>Basic ??????????=</ArgumentValue><RequestId>THSXS5KPHX6JGZD0</RequestId><HostId>bbdLyfZOA/4szLDNZxNHCDVDisxUA75obiiiLrhWGzfvIw+btmBWzsu87vTvrf0eBgLWd0ClD4Q=</HostId></Error>
0

click.exceptions.BadParameter: is not a valid boolean when incorrect configs are in place

I follow the README.md document step-by-step and once sample configuration files are mentioned I errorneously copied them to their respective directories:

 newman  ~  curl https://raw.githubusercontent.com/cloudsmith-io/cloudsmith-cli/master/cloudsmith_cli/data/credentials.ini > .config/cloudsmith/credentials.ini
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   501  100   501    0     0    533      0 --:--:-- --:--:-- --:--:--   532

 newman  ~  curl https://raw.githubusercontent.com/cloudsmith-io/cloudsmith-cli/master/cloudsmith_cli/data/config.ini > .config/cloudsmith/config.ini
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   737  100   737    0     0    485      0  0:00:01  0:00:01 --:--:--   485^[[A

And I noticed that cloudsmith CLI immediatelly started to crash:

 newman  ~  cloudsmith token
Usage: cloudsmith [OPTIONS] COMMAND [ARGS]...

Traceback (most recent call last):
  File "/home/newman/.local/lib/python3.7/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/newman/.local/lib/python3.7/site-packages/click/core.py", line 1134, in invoke
    Command.invoke(self, ctx)
  File "/home/newman/.local/lib/python3.7/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/newman/.local/lib/python3.7/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/newman/.local/lib/python3.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/newman/.local/lib/python3.7/site-packages/cloudsmith_cli/cli/decorators.py", line 86, in wrapper
    opts.load_config_file(path=config_file, profile=profile)
  File "/home/newman/.local/lib/python3.7/site-packages/cloudsmith_cli/cli/config.py", line 187, in load_config_file
    return config_cls.load_config(self, path, profile=profile)
  File "/home/newman/.local/lib/python3.7/site-packages/cloudsmith_cli/cli/config.py", line 117, in load_config
    config = cls.read_config()
  File "/home/newman/.local/lib/python3.7/site-packages/click_configfile.py", line 401, in read_config
    cls.process_config_section(config_section, storage)
  File "/home/newman/.local/lib/python3.7/site-packages/click_configfile.py", line 445, in process_config_section
    section_data = parse_config_section(config_section, schema)
  File "/home/newman/.local/lib/python3.7/site-packages/click_configfile.py", line 270, in parse_config_section
    value = param.parse(value)
  File "/home/newman/.local/lib/python3.7/site-packages/click_configfile.py", line 197, in parse
    return self.type.convert(text, self, ctx=None)
  File "/home/newman/.local/lib/python3.7/site-packages/click/types.py", line 355, in convert
    self.fail('%s is not a valid boolean' % value, param, ctx)
  File "/home/newman/.local/lib/python3.7/site-packages/click/types.py", line 69, in fail
    raise BadParameter(message, ctx=ctx, param=param)
click.exceptions.BadParameter:  is not a valid boolean

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/newman/.local/bin/cloudsmith", line 10, in <module>
    sys.exit(main())
  File "/home/newman/.local/lib/python3.7/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/newman/.local/lib/python3.7/site-packages/click/core.py", line 734, in main
    e.show()
  File "/home/newman/.local/lib/python3.7/site-packages/click/exceptions.py", line 70, in show
    echo('Error: %s' % self.format_message(), file=file, color=color)
  File "/home/newman/.local/lib/python3.7/site-packages/click/exceptions.py", line 101, in format_message
    param_hint = self.param.get_error_hint(self.ctx)
AttributeError: 'Param' object has no attribute 'get_error_hint'

I suggest two things: (1) catch the exception and make the error message in this case more sensible, (2) perhaps put the paragraph on sample config files after the cloudsmith token procedure.

I fixed my system by removing those sample config files and running cloudsmith token.

I am on Fedora 30 with Python 3.7 and cloudsmith 0.16 from PyPI via pip-3.

`list packages` command fails in version 0.35.0

The following command fails since upgrading to cloudsmith-cli version 0.35.0:

$ cloudsmith list packages --output-format json ORG/REPO
Getting list of packages ... OK
Failed to convert to JSON: Object of type datetime is not JSON serializable

The same command is successful with cloudsmith-cli version 0.34.0.

I re-ran with -v and -d which resulted in the following log line:

2022-12-14 16:20:56,608 DEBUG https://api.cloudsmith.io:443 "GET /v1/packages/ORG/REPO/?page=1&page_size=30 HTTP/1.1" 200 None

So I guess it's still using the v1 API and failing since it now only supports the v2 API.

Issues with list package output

I would like to use list package -q to select a group of packages and pipe that to delete.

I don't know if that's something python and the cloudsmith-cli can do.

Alternatively, I can -F json, parse that, and loop through calling delete for each identifier. Except, the list command outputs Getting list of packages ... OK and "corrupts" the json. Is there a way to suppress the informational message?

cloudsmith docker image please?

Installing cloudsmith to Ubuntu docker container takes a long time.

It'd be great that cloudsmith has its own docker image so that we can make use of it without doing the cloudsmith installation every person every time.

cloudsmith upload raw fails due to unknown architecture

It seems that an upgrade in the cloudsmith-api dependency to 2.0.8 causes an uploading of a raw package via cloudsmith upload raw to fail due to ValueError: Invalid value for `architecture` (dummy) error, which would suggest that the raw package now needs its architecture defined, however the cloudsmith-cli version does not provide an option for specifying this.

The cloudsmith-api was recently released which breaks this due to only be a patch release, which means it was pulled in automatically by cloudsmith-cli.

Below shows two examples of using the previous (working) version of cloudsmith-api 2.0.7 and the newer version 2.0.8.

OK - Using cloudsmith-cli 1.1.1 with cloudsmith-api 2.0.7

Installing cloudsmith-cli:

$ pip3 install --upgrade cloudsmith-cli
Successfully installed certifi-2024.2.2 charset-normalizer-3.3.2 click-8.1.7 click-configfile-0.2.3 click-didyoumean-0.3.0 click-spinner-0.1.10 cloudsmith-api-2.0.7 cloudsmith-cli-1.1.1 configparser-6.0.0 idna-3.6 python-dateutil-2.8.2 requests-2.31.0 requests-toolbelt-1.0.0 semver-3.0.2 six-1.16.0 urllib3-1.26.18

Uploading a raw binary:

cloudsmith upload raw ***/tedge-main tmp/tedge.tar.gz --name tedge-amd64 --version 1.0.1-rc37+g6f6f603 --no-wait-for-sync --api-key ***
Uploading file: target/x86_64-unknown-linux-musl/packages/tedge_1.0.1-rc37+g6f6f603_x86_64-unknown-linux-musl.tar.gz (name=tedge-amd64, version=1.0.1-rc37+g6f6f603, file=tmp/tedge.tar.gz)

FAIL - Using cloudsmith-cli 1.1.1 with cloudsmith-api 2.0.8

Installing cloudsmith-cli:

$ pip3 install --upgrade cloudsmith-cli
Successfully installed certifi-2024.2.2 charset-normalizer-3.3.2 click-8.1.7 click-configfile-0.2.3 click-didyoumean-0.3.0 click-spinner-0.1.10 cloudsmith-api-2.0.8 cloudsmith-cli-1.1.1 configparser-6.0.0 idna-3.6 python-dateutil-2.8.2 requests-2.31.0 requests-toolbelt-1.0.0 semver-3.0.2 six-1.16.0 urllib3-1.26.18

Uploading a raw binary:

cloudsmith upload raw ***/tedge-main tmp/tedge.tar.gz --name tedge-amd64 --version 1.0.1-rc51+g3389918 --no-wait-for-sync --api-key ***
Uploading file: target/x86_64-unknown-linux-musl/packages/tedge_1.0.1-rc51+g3389918_x86_64-unknown-linux-musl.tar.gz (name=tedge-amd64, version=1.0.1-rc51+g3389918, file=tmp/tedge.tar.gz)
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.11.8/x64/bin/cloudsmith", line 5, in <module>
    from cloudsmith_cli.cli.commands.main import main
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/cloudsmith_cli/cli/commands/__init__.py", line 3, in <module>
    from . import (
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/cloudsmith_cli/cli/commands/copy.py", line 10, in <module>
    from .push import wait_for_package_sync
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/cloudsmith_cli/cli/commands/push.py", line 568, in <module>
    create_push_handlers()
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/cloudsmith_cli/cli/commands/push.py", line 420, in create_push_handlers
    context = create_push_handlers.context = get_package_formats()
                                             ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/cloudsmith_cli/core/api/packages.py", line 274, in get_package_formats
    return {
           ^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/cloudsmith_cli/core/api/packages.py", line 275, in <dictcomp>
    key.replace("PackageUploadRequest", "").lower(): get_parameters(cls)
                                                     ^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/cloudsmith_cli/core/api/packages.py", line 251, in get_parameters
    instance = cls(**dummy_kwargs)
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/cloudsmith_api/models/cran_package_upload_request.py", line 65, in __init__
    self.architecture = architecture
    ^^^^^^^^^^^^^^^^^
  File "/opt/hostedtoolcache/Python/3.11.8/x64/lib/python3.11/site-packages/cloudsmith_api/models/cran_package_upload_request.py", line 97, in architecture
    raise ValueError(
ValueError: Invalid value for `architecture` (dummy), must be one of ['arm64', 'x86_64']

Errors/Failures Return Zeroes

Courtesy of a submission from DH:

Also, could the CLI return a failure code on failures? Currently it returns 0 on both success and failures.

So we need to ensure that all error/failure conditions return a non-zero exit code. I suspect that this may possibly be due to some issues with python-click, and I'm not sure this universally happens, but we'll need to investigate it.

Thank you DH!

Unable to login

I am issuing the following command as documented:

cloudsmith login

And it gives me the help menu.

If I add -l it gets a little further:

$ cloudsmith login -l [email protected]
Password: 
Repeat for confirmation: 
Usage: cloudsmith login [OPTIONS] COMMAND [ARGS]...
Try "cloudsmith login -h" for help.

Error: Missing command.

What am I doing wrong?

cloudsmith-cli errors when doing a whoami on a service account

When using a service account with cloudsmith-cli, the whoami fails because it is trying to map fields that don't get returned from the API (name, email).
Stack Trace Snippet:
site-packages/cloudsmith_cli/cli/commands/whoami.py", line 39, in whoami
"email": click.style(email, fg="green"),
File "/usr/lib/python3/dist-packages/click/termui.py", line 449, in style
return ''.join(bits)
TypeError: sequence item 1: expected str instance, NoneType found

I will send a PR where we can explicitly define these as "Null" if they are None, to allow the code to properly complete.

RFE: 'cloudsmith push deb' parse .dsc `Files:` section for sources/changes

Currently to upload a debian source package:

cloudsmith push deb <repo/dist/version> foo_1.0.0-1.dsc --sources-file foo_1.0.0.orig.tar.bz2 --changes-file foo_1.0.0-1.debian.tar.xz

When scripting the CLI this means detecting/determining the sources/changes files (and they can have various suffix, sources: .orig.tar.gz, .orig.tar.bz2, changes: .debian.tar.xz, .diff.gz , etc).

The files that make up the source package are listed in the 'Files:' section of the .dsc file itself. https://manpages.debian.org/testing/dpkg-dev/dsc.5.en.html

It would be useful if 'cloudsmith push deb' when passed only a .dsc file could parse the 'Files:' to determine the additional sources/changes files it should upload.

If it helps, the https://pypi.org/project/pydpkg/ python library can parse .dsc files:

>>> dsc.source_files
['/tmp/testdeb_0.0.0.orig.tar.gz',
 '/tmp/testdeb_0.0.0-1.debian.tar.xz',
 '/tmp/testdeb_0.0.0.dsc' ]
 >>> dsc.all_files_present
True

Error uploading to private repository via cli

Steps to reproduce

  • set up new virtualenv and install dependencies, including cloudsmith-cli
  • set CLOUDSMITH_API_KEY env variable to appropriate key
  • build python (3.5.2) package using setuptools/distutils (successful)
  • run process to upload python package to cloudsmith via cli tools
  • process errors as described below

Expected behavior

Upload occurs, placing package in proper repository per input values at the command line level. Outcome is identical to the process that would occur if uploaded by web UI (I have confirmed this works w/ the same package attempted via cli).

STDERR Output

$ python setup.py bdist_wheel
$ package_filename=$(ls dist)
$ cloudsmith push python private-org/python dist/$package_filename

Checking python package upload parameters ... OK
Checking private-org.models.common-0.1.0-py2.py3-none-any.whl file upload parameters ... OK
Requesting file upload for private-org.models.common-0.1.0-py2.py3-none-any.whl ... OK
Uploading private-org.models.common-0.1.0-py2.py3-none-any.whl:  [------------------------------------]    0%
Traceback (most recent call last):
  File "/home/pmdev/.virtualenvs/models/bin/cloudsmith", line 11, in <module>
    sys.exit(main())
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/cloudsmith_cli/cli/decorators.py", line 40, in wrapper
    return ctx.invoke(f, *args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/cloudsmith_cli/cli/decorators.py", line 65, in wrapper
    return ctx.invoke(f, *args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/cloudsmith_cli/cli/decorators.py", line 81, in wrapper
    return ctx.invoke(f, *args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/cloudsmith_cli/cli/decorators.py", line 119, in wrapper
    return ctx.invoke(f, *args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/cloudsmith_cli/cli/commands/push.py", line 382, in push_handler
    upload_files_and_create_package(ctx, *args, **kwargs)
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/cloudsmith_cli/cli/commands/push.py", line 245, in upload_files_and_create_package
    skip_errors=skip_errors, md5_checksum=md5_checksums[k]
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/cloudsmith_cli/cli/commands/push.py", line 94, in upload_file
    callback=progress_callback
  File "/home/pmdev/.virtualenvs/models/lib/python3.5/site-packages/cloudsmith_cli/core/api/files.py", line 62, in upload_file
    upload_fields.append(
AttributeError: 'dict_items' object has no attribute 'append'

Environment Details

  • python version: Python 3.5.2
  • OS: Elementary OS 0.4.1
  • uname -a: Linux asus13 4.10.0-42-generic #46~16.04.1-Ubuntu SMP Mon Dec 4 15:57:59 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

packages installed in virtualenv

certifi==2017.11.5
chardet==3.0.4
click==6.7
click-configfile==0.2.3
click-didyoumean==0.0.3
click-spinner==0.1.7
cloudsmith-api==0.21.3
cloudsmith-cli==0.3.1
colorama==0.3.9
configparser==3.5.0
idna==2.6
mongoengine==0.15.0
nose==1.3.7
nose-mongoengine==0.2.2
pep8==1.7.1
pycodestyle==2.3.1
pymongo==3.6.0
python-dateutil==2.6.1
requests==2.18.4
requests-toolbelt==0.8.0
semver==2.7.9
simplejson==3.13.2
six==1.11.0
urllib3==1.22

Notes

This looks like just a bad assumption related to the upload_fields object here, but it looks like this is being passed through from the upstream API, so I'm hesitant to fork and fix as I assume this was a list at one point where now a dict is present.

no batch support

for gentoo raw types xpak or bz2 adding *.foo *.bar , is a no go... anycase raw dir tree sync isnt in yet.. as you can have gcc-6.x and cross-gcc/gcc-6.x or etc.. however in repo... on cloudsmith as a flat
the files will collide... hopefully soon fixed..

cloudsmith push raw --tree=/usr/portage/packages/* --batch .tbz2 ,.xapk
for sabayon Linux mainly *.bz2 and repo gpg and entropy db files made of gentoo packages.. in sqlite and metadata... and class dev-util:name-mtimehash-numbers.tbz2

cloudsmith push raw --batch *.tbz2
building file list , loop n cloudsmith push raw reponame $package$name/s

likewise for publishers , *.rpm deb etc..

Got unexpected extra arguments (f2c-20100827.tbz2 go-1.10.1-1.xpak go-1.8.1.tbz2 go-1.9.1-1.xpak lua-5.1.5-r4.tbz2 perl-5.24.1-r2.tbz2 perl-5.24.3.tbz2 perl-5.26.1.tbz2 perl-5.26.1.tbz2.1 python-2.7.12.tbz2 python-3.4.6.tbz2 python-3.5.3.tbz2 python-3.5.3.tbz2.27064 python-3.5.4.tbz2 python-3.6.1-r1.tbz2 python-3.6.3.tbz2 python-exec-2.4.5.tbz2 ruby-2.1.9.tbz2 ruby-2.2.7-r2.tbz2 ruby-2.2.8-1.xpak tcl-8.5.17.tbz2 tcl-8.6.6.tbz2 tcl-8.6.7.tbz2 tk-8.6.7.tbz2 vala-0.34.3.tbz2 vala-0.36.5.tbz2 yasm-1.3.0.tbz2)

ps...
https://cloudsmith.io/package/ns/necrose99/repos/cloudsmith-arm64-eggs/packages/

{gentoo will typically gripe ver eggs or prebuilt  via pip install as it expects virtual en or user 
,  however suitable for any distro....   
mainly since gentoo can swap python defaults on the fly 
and for the time being i don't care to rebuild for iot-arm64... atm.  if the box crashes.. }
however do enjoy... 

Version cloudsmith-cli==1.1.0 is failing when uploading npm packages

Hi,

we noticed that using version cloudsmith-cli==1.1.0 fail with the following error when uploading npm packages:

cloudsmith push npm $org/$repo $some_package.tgz

Checking npm package upload parameters ... Traceback (most recent call last): File "/usr/local/bin/cloudsmith", line 8, in <module> sys.exit(main()) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1157, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1078, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1688, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1434, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/cli/decorators.py", line 100, in wrapper return ctx.invoke(f, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/cli/decorators.py", line 139, in wrapper return ctx.invoke(f, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/cli/decorators.py", line 61, in wrapper return ctx.invoke(f, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/cli/decorators.py", line 191, in wrapper return ctx.invoke(f, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/cli/decorators.py", line 309, in wrapper return ctx.invoke(f, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 783, in invoke return __callback(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/click/decorators.py", line 33, in new_func return f(get_current_context(), *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/cli/commands/push.py", line 517, in push_handler upload_files_and_create_package(ctx, *args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/cli/commands/push.py", line 341, in upload_files_and_create_package validate_create_package( File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/cli/commands/push.py", line 148, in validate_create_package api_validate_create_package( File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/core/api/packages.py", line 47, in validate_create_package client = get_packages_api() File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/core/api/packages.py", line 16, in get_packages_api return get_api_client(cloudsmith_api.PackagesApi) File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/core/api/init.py", line 70, in get_api_client client.api_client.rest_client = RestClient( File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/core/rest.py", line 168, in __init__ self.session = create_requests_session(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/core/rest.py", line 106, in create_requests_session retry = RetryWithCallback( File "/usr/local/lib/python3.8/dist-packages/cloudsmith_cli/core/rest.py", line 25, in __init__ super().__init__(*args, **kwargs) TypeError: __init__() got an unexpected keyword argument 'allowed_methods

python3 --version Python 3.8.10

lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 20.04.4 LTS Release: 20.04 Codename: focal

Downgrading back to version cloudsmith-cli==1.0.0 works as expected.

Kind regards.

400 Bad Request

+ docker save -o ci.docker docker.cloudsmith.io/[owner]/[repo]/[image]:[tag]
+ printf 'Pushing to Cloudsmith…\n'
Pushing to Cloudsmith…
+ cloudsmith push docker [owner]/[repo] ci.docker
Checking docker package upload parameters ... OK
Checking ci.docker file upload parameters ... OK
Requesting file upload for ci.docker ... OK
Uploading ci.docker:
ERROR
Failed to upload file! (status: 400 - Bad Request)

And that's all the info we get! This is not a useful diagnostic: I have absolutely no idea what I've done wrong here.

(Currently attempting to re-run this w/ --debug to see if I can get enough info to piece it together. --debug emits too much data to be useful.)

Authorization error is misleading

If you attempt to cloudsmith push docker with an entitlement token (which, as I've learned the hard way, are read-only), you'll get approximately the following:

Checking docker package upload parameters ... ERROR
Failed to validate upload parameters! (status: 401 - Unauthorized)

Detail: Invalid token.
Hint: Since you have an API key set, this probably means you don't have the permision to perform this action.

First, the error is self-contradictory: the first half, "401 - Unauthorized" is an HTTP status code; 401 is a bit screwed up in HTTP, but it means and should be read as (despite what one might think from the canonical reason phrase) unauthenticated.

The latter half says something completely different: "you don't have the permision (sic)" ; the proper HTTP status for that would be 403 Forbidden.

So, the user is left wondering "which is it?".

(Initially, I had only read the code — 401 — and mistakenly presumed it was an actual authentication problem, and spent considerable time chasing that goose. This turned out to be an authorization problem, and the hint is correct.)

Additionally, "permission" is misspelled in the error message.

Installable Standalone CLI

As a Cloudsmith user, I would like to have an CLI application for interacting with Cloudsmith that doesn't require python. This would make it easier for me to install and update the CLI app without worrying about the version or configuration of Python. Particularly on Windows machines, where Python isn't installed by default.

This would also make it easier to add the CLI to various package managers for simpler installs and hopefully improve adoption.

If it's implemented in Go, the application could take advantage of cobra and viper for a cohesive CLI experience that handles shell completion, in-console help, and compositional values from configuration, environment, and flags/arguments. With the charm libraries, the CLI could also surface a more delightful UX that makes it easier to use.

Credentials File Not Working

Courtesy of a submission from DH:

I think I found a bug in cloudsmith-cli. I'm unable to use the --credentials-file option. The documentation for the option says that it is "The path to your credentials.ini file", but if I pass "--credentials_file ." I get the error
"Error: Invalid value for "--credentials-file": Path "." is a directory."

If I pass "--credentials_file ./credentials.ini", I get
"Checking python package upload parameters ... ERROR
Failed to validate upload parameters! (status: 404 - Not Found)"
meaning that the credentials were not read.

Potential fix also suggested:

I think two places need to be fixed:
https://github.com/cloudsmith-io/cloudsmith-cli/blob/master/cloudsmith_cli/cli/decorators.py#L49
dir_okay should be set to True instead of False.

Also, https://github.com/cloudsmith-io/cloudsmith-cli/blob/master/cloudsmith_cli/cli/config.py#L119
cls.searchpath should be cls.config_searchpath

Thank you DH!

Default credentials file in `~/.cloudsmith/credentials.ini` is ignored

I added a file with the following contents:

# Default configuration
[default]
api_key={redacted}

However, it doesn't work for me if this file is located in ${HOME}/.cloudsmith/credentials.ini, the CLI still sees me as an anonymous user.
If I move that file to ${HOME}/.config/credentials.ini, then everything works fine.

Meta

I am using a server via SSH, my environment is the following:

$ uname -a
Linux veetaha-hetzner 5.4.0-77-generic #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.