Giter Club home page Giter Club logo

artifactory's Introduction

Python interface library for JFrog Artifactory

docs dohq-artifactory build Status dohq-artifactory on PyPI dohq-artifactory license

dohq-artifactory is a live python package for JFrog Artifactory. This module is intended to serve as a logical descendant of pathlib, and it implements everything as closely as possible to the origin with few exceptions. Current module was forked from outdated parallels/artifactory and supports all functionality from the original package.

Tables of Contents

Install

Upgrade/install to the newest available version:

pip install dohq-artifactory --upgrade

Install latest development version (Warning! It may contains some errors!):

pip install dohq-artifactory --upgrade --pre

Or specify version, e.g.:

pip install dohq-artifactory==0.5.dev243

Usage

Authentication

dohq-artifactory supports these ways of authentication:

  • Username and password (or API KEY) to access restricted resources, you can pass auth parameter to ArtifactoryPath.
  • API KEY can pass with apikey parameter.
  • Access Token can pass with token parameter.
from artifactory import ArtifactoryPath

# API_KEY
path = ArtifactoryPath(
    "http://my-artifactory/artifactory/myrepo/restricted-path", apikey="MY_API_KEY"
)

# Access Token
path = ArtifactoryPath(
    "http://my-artifactory/artifactory/myrepo/restricted-path", token="MY_ACCESS_TOKEN"
)

# User and password OR API_KEY
path = ArtifactoryPath(
    "http://my-artifactory/artifactory/myrepo/restricted-path",
    auth=("USERNAME", "PASSWORD or API_KEY"),
)

# Other authentication types
from requests.auth import HTTPDigestAuth

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/myrepo/restricted-path",
    auth=("USERNAME", "PASSWORD"),
    auth_type=HTTPDigestAuth,
)

from requests.auth import HTTPBasicAuth

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/myrepo/restricted-path",
    auth=("USERNAME", "PASSWORD"),
    auth_type=HTTPBasicAuth,
)

# Load username, password from global config if exist:
path = ArtifactoryPath(
    "http://my-artifactory/artifactory/myrepo/restricted-path",
    auth_type=HTTPBasicAuth,
)

path.touch()

Artifactory SaaS

If you use Artifactory SaaS solution - use ArtifactorySaaSPath class.
SaaS supports all methods and authentication types as ArtifactoryPath. We have to use other class, because as a SaaS service, the URL is different from an on-prem installation and the REST API endpoints.

from artifactory import ArtifactorySaaSPath

path = ArtifactorySaaSPath(
    "https://myartifactorysaas.jfrog.io/myartifactorysaas/folder/path.xml",
    apikey="MY_API_KEY",
)

Walking Directory Tree

Get directory listing:

from artifactory import ArtifactoryPath

path = ArtifactoryPath("http://repo.jfrog.org/artifactory/gradle-ivy-local")
for p in path:
    print(p)

Find all .gz files in current dir, recursively:

from artifactory import ArtifactoryPath

path = ArtifactoryPath("http://repo.jfrog.org/artifactory/distributions/org/")

for p in path.glob("**/*.gz"):
    print(p)

Downloading Artifacts

Download artifact to a local filesystem:

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://repo.jfrog.org/artifactory/distributions/org/apache/tomcat/apache-tomcat-7.0.11.tar.gz"
)

with path.open() as fd, open("tomcat.tar.gz", "wb") as out:
    out.write(fd.read())

Downloading Artifacts in chunks

Download artifact to the local filesystem using chunks (in bytes) to prevent loading the entire response into memory at once. This can help with getting big files or resolve known issue

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://repo.jfrog.org/artifactory/distributions/org/apache/tomcat/apache-tomcat-7.0.11.tar.gz"
)

# download by providing path to out file and use default chunk 1024
path.writeto(out="tomcat.tar.gz")

# download and suppress progress messages
path.writeto(out="tomcat2.tar.gz", progress_func=None)

# download by providing out as file object and specify chunk size
with open("tomcat3.tar.gz", "wb") as out:
    path.writeto(out, chunk_size=256)


# download and use custom print function
def custom_print(bytes_now, total, custom):
    """
    Custom function that accepts first two arguments as [int, int] in its signature
    """
    print(bytes_now, total, custom)


# since writeto requires [int, int] in its signature, all custom arguments you have to provide via lambda function or
# similar methods
path.writeto(
    out="tomcat5.tar.gz",
    progress_func=lambda x, y: custom_print(x, y, custom="test"),
)

Downloading Artifacts folder as archive

Download artifact folder to a local filesystem as archive (supports zip/tar/tar.gz/tgz) Allows to specify archive type and request checksum for the folder

Note: Archiving should be enabled on the server!

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my_url:8080/artifactory/my_repo/winx64/aas", auth=("user", "password")
)

with path.archive(archive_type="zip", check_sum=False).open() as archive:
    with open(r"D:\target.zip", "wb") as out:
        out.write(archive.read())

# download folder archive in chunks
path.archive().writeto(out="my.zip", chunk_size=100 * 1024)

Uploading Artifacts

Deploy a regular file myapp-1.0.tar.gz. This method by default will calculate all available checksums and attach them to the file

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/libs-snapshot-local/myapp/1.0"
)
path.mkdir()

path.deploy_file("./myapp-1.0.tar.gz")

Deploy artifacts from archive: this will automatically extract the contents of the archive on the server preserving the archive's paths

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/libs-snapshot-local/myapp/1.0"
)
path.mkdir()

path.deploy_file("./myapp-1.0.tar.gz", explode_archive=True)

Atomically deploy artifacts from archive: this will automatically extract the contents of the archive on the server preserving the archive's paths. This is primarily useful when you want Artifactory to see all the artifacts at once, e.g., for indexing purposes.

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/libs-snapshot-local/myapp/1.0"
)
path.mkdir()

path.deploy_file(
    "./myapp-1.0.tar.gz", explode_archive=True, explode_archive_atomic=True
)

Deploy artifact by checksum: deploy an artifact to the specified destination by checking if the artifact content already exists in Artifactory. If Artifactory already contains a user readable artifact with the same checksum the artifact content is copied over to the new location without requiring content transfer.

from artifactory import ArtifactoryPath

path = ArtifactoryPath("http://my-artifactory/artifactory/my_repo/foo")
sha1 = "1be5d2dbe52ddee96ef2d17d354e2be0a155a951"
sha256 = "00bbf80ccca376893d60183e1a714e707fd929aea3e458f9ffda60f7ae75cc51"

# If you don't know sha value, you can calculate it via
# sha1 = artifactory.sha1sum("local_path_of_your_file")
# or
# sha256 = artifactory.sha256sum("local_path_of_your_file")

# Each of the following 4 methods works fine if the artifact content already
# exists in Artifactory.
path.deploy_by_checksum(sha1=sha1)

# deploy by sha1 via checksum parameter
path.deploy_by_checksum(checksum=sha1)

# deploy by sha256 via sha256 parameter
path.deploy_by_checksum(sha256=sha256)

# deploy by sha256 via checksum parameter
path.deploy_by_checksum(checksum=sha256)

Deploy a debian package myapp-1.0.deb to an existent folder

from artifactory import ArtifactoryPath

path = ArtifactoryPath("http://my-artifactory/artifactory/ubuntu-local/pool/")
path.deploy_deb(
    "./myapp-1.0.deb", distribution="trusty", component="main", architecture="amd64"
)

Deploy a debian package myapp-1.0.deb to a non-existent folder

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/ubuntu-local/pool/myapp-1.0.deb"
)
path.deploy_deb(
    "./myapp-1.0.deb", distribution="trusty", component="main", architecture="amd64"
)

# if you want to set multiple values you can use list to set them
path.deploy_deb(
    "./myapp-1.0.deb",
    distribution=["dist1", "dist2"],
    component="main",
    architecture=["amd64", "i386"],
)

Copy Artifacts

Copy artifact from this path to destination. If files are on the same instance of artifactory, lightweight (local) copying will be attempted.

The suppress_layouts parameter, when set to True, will allow artifacts from one path to be copied directly into another path without enforcing repository layouts. The default behaviour is to copy to the repository root, but remap the [org], [module], [baseVer], etc. structure to the target repository.

For example, we have a builds repository using the default maven2 repository where we publish our builds, and we also have a published repository where a directory for production and a directory for staging environments should hold the current promoted builds. How do we copy the contents of a build over to the production folder?

from artifactory import ArtifactoryPath

source = ArtifactoryPath("http://example.com/artifactory/builds/product/product/1.0.0/")
dest = ArtifactoryPath("http://example.com/artifactory/published/production/")

"""
Using copy with the default, suppress_layouts=False, the artifacts inside
builds/product/product/1.0.0/ will not end up in the published/production
path as we intended, but rather the entire structure product/product/1.0.0
is placed in the destination repo.
"""

source.copy(dest)
for p in dest:
    print(p)
# http://example.com/artifactory/published/production/foo-0.0.1.gz
# http://example.com/artifactory/published/production/foo-0.0.1.pom

for p in ArtifactoryPath(
    "http://example.com/artifactory/published/product/product/1.0.0.tar"
):
    print(p)
# http://example.com/artifactory/published/product/product/1.0.0/product-1.0.0.tar.gz
# http://example.com/artifactory/published/product/product/1.0.0/product-1.0.0.tar.pom

"""
Using copy with suppress_layouts=True, the contents inside our source are copied
directly inside our dest as we intended.
"""

source.copy(dest, suppress_layouts=True)
for p in dest:
    print(p)
"""
http://example.com/artifactory/published/production/foo-0.0.1.gz
http://example.com/artifactory/published/production/foo-0.0.1.pom
http://example.com/artifactory/published/production/product-1.0.0.tar.gz
http://example.com/artifactory/published/production/product-1.0.0.tar.pom
"""

# you can use dry run just to check if command will succeed without real change, adds debug message
source.copy(dest, dry_run=True)

Move Artifacts

Move artifact from this path to destination.

The suppress_layouts parameter, when set to True, will allow artifacts from one path to be copied directly into another path without enforcing repository layouts. The default behaviour is to copy to the repository root, but remap the [org], [module], [baseVer], etc. structure to the target repository.

from artifactory import ArtifactoryPath

source = ArtifactoryPath("http://example.com/artifactory/builds/product/product/1.0.0/")
dest = ArtifactoryPath("http://example.com/artifactory/published/production/")

source.move(dest)

# you can use dry run just to check if command will succeed without real change, adds debug message
source.move(dest, dry_run=True)

Remove Artifacts

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://repo.jfrog.org/artifactory/distributions/org/apache/tomcat/apache-tomcat-7.0.11.tar.gz"
)

if path.exists():
    path.unlink()

Artifact properties

You can get and set (or remove) properties from artifact. Following example shows how to manage properties and property sets

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://repo.jfrog.org/artifactory/distributions/org/apache/tomcat/apache-tomcat-7.0.11.tar.gz"
)

# Get properties
properties = path.properties
print(properties)

# Update a property or add if does not exist
properties["qa"] = "tested"
path.properties = properties

# add/replace set of properties
new_props = {
    "test": ["test_property"],
    "time": ["2018-01-16 12:17:44.135143"],
    "addthis": ["addthis"],
}
path.properties = new_props

# Remove properties
properties.pop("release")
path.properties = properties

Repository Scheduled Replication Status

Returns the status of scheduled cron-based replication jobs define via the Artifactory UI on repositories. Supported by local, local-cached and remote repositories.

Notes: Requires Artifactory Pro

Security: Requires a user with 'read' permission (can be anonymous)

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "https://repo.jfrog.org/artifactory/repo1-cache/archetype-catalog.xml"
)

rep_status = path.replication_status
print("status: ", rep_status["status"])

Artifactory Query Language

You can use Artifactory Query Language in python.

from artifactory import ArtifactoryPath

arti_path = ArtifactoryPath(
    "http://my-artifactory/artifactory"
)  # path to artifactory, NO repo

# dict support
# Send query:
# items.find({"repo": "myrepo"})
artifacts = arti_path.aql("items.find", {"repo": "myrepo"})

# list support.
# Send query:
# items.find().include("name", "repo")
artifacts = arti_path.aql("items.find()", ".include", ["name", "repo"])

#  support complex query
# Example 1
# items.find(
#     {
#         "$and": [
#             {"repo": {"$eq": "repo"}},
#             {"$or": [{"path": {"$match": "*path1"}}, {"path": {"$match": "*path2"}}]},
#         ]
#     }
# )
aqlargs = [
    "items.find",
    {
        "$and": [
            {"repo": {"$eq": "repo"}},
            {
                "$or": [
                    {"path": {"$match": "*path1"}},
                    {"path": {"$match": "*path2"}},
                ]
            },
        ]
    },
]

# artifacts_list contains raw data (list of dict)
# Send query
artifacts_list = arti_path.aql(*aqlargs)

# Example 2
# The query will find all items in repo docker-prod that are of type file and were created after timecode. The
# query will only display the fields "repo", "path" and "name" and will sort the result ascendingly by those fields.
# items.find(
#     {
#         "$or": [{"repo": "docker-prod"}],
#         "type": "file",
#         "created": {"$gt": "2019-07-10T19:20:30.45+01:00"},
#     }
# ).include("repo", "path", "name",).sort({"$asc": ["repo", "path", "name"]})
aqlargs = [
    "items.find",
    {
        "$and": [
            {"repo": "docker-prod"},
            {"type": "file"},
            {"created": {"$gt": "2019-07-10T19:20:30.45+01:00"}},
        ]
    },
    ".include",
    ["repo", "path", "name", "type"],
    ".sort",
    {"$asc": ["repo", "path", "name"]},
]
artifacts_list = arti_path.aql(*aqlargs)

# You can convert to pathlib object:
artifact_pathlib = map(arti_path.from_aql, artifacts_list)
artifact_pathlib_list = list(map(arti_path.from_aql, artifacts_list))

Artifact Stat

File/Folder Statistics

You can get hash (md5, sha1, sha256), creator, create date, and change date:

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://repo.jfrog.org/artifactory/distributions/org/apache/tomcat/apache-tomcat-7.0.11.tar.gz"
)

# Get FileStat
stat = path.stat()
print(stat)
print(stat.ctime)
print(stat.mtime)
print(stat.created_by)
print(stat.modified_by)
print(stat.mime_type)
print(stat.size)
print(stat.sha1)
print(stat.sha256)
print(stat.md5)
print(stat.is_dir)
print(stat.children)
print(stat.repo)

Get Download Statistics

Information about number of downloads, user that last downloaded and date of last download

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://repo.jfrog.org/artifactory/distributions/org/apache/tomcat/apache-tomcat-7.0.11.tar.gz"
)

# Get FileStat
download_stat = path.download_stats()
print(download_stat)
print(download_stat.last_downloaded)
print(download_stat.last_downloaded_by)
print(download_stat.download_count)
print(download_stat.remote_download_count)
print(download_stat.remote_last_downloaded)
print(download_stat.uri)

Promote Docker image

Promotes a Docker image in a registry to another registry.

from artifactory import ArtifactoryPath

path = ArtifactoryPath("http://example.com/artifactory")

path.promote_docker_image("docker-staging", "docker-prod", "my-application", "0.5.1")

Builds

from artifactory import ArtifactoryBuildManager

arti_build = ArtifactoryBuildManager(
    "https://repo.jfrog.org/artifactory", project="proj_name", auth=("admin", "admin")
)

# Get all builds
all_builds = arti_build.builds
print(all_builds)

# Build Runs
build1 = all_builds[0]
all_runs = build1.runs
print(all_runs)

# Build Info
build_number1 = all_runs[0]
print(build_number1.info)

# Builds Diff
"""
  Compare a build artifacts/dependencies/environment with an older build to see what 
  has changed (new artifacts added, old dependencies deleted etc).  
"""
print(build_number1.diff(3))


# Build Promotion
"""
  Change the status of a build, optionally moving or copying the build's artifacts and its dependencies 
  to a target repository and setting properties on promoted artifacts.  
  All artifacts from all scopes are included by default while dependencies are not. Scopes are additive (or)
"""

build_number1.promote(
    ci_user="admin",
    properties={"components": ["c1", "c3", "c14"], "release-name": ["fb3-ga"]},
)

Exception handling

Exceptions in this library are represented by dohq_artifactory.exception.ArtifactoryException or by OSError If exception was caused by HTTPError you can always drill down the root cause by using following example:

from artifactory import ArtifactoryPath
from dohq_artifactory.exception import ArtifactoryException

path = ArtifactoryPath(
    "http://my_arti:8080/artifactory/installer/", auth=("wrong_user", "wrong_pass")
)

try:
    path.stat()
except ArtifactoryException as exc:
    print(exc)  # clean artifactory error message
    # >>> Bad credentials
    print(
        exc.__cause__
    )  # HTTP error that triggered exception, you can use this object for more info
    # >>> 401 Client Error: Unauthorized for url: http://my_arti:8080/artifactory/installer/

Admin area

You can manipulate with user\group\repository and permission. First, create ArtifactoryPath object without a repository

from artifactory import ArtifactoryPath

artifactory_ = ArtifactoryPath(
    "https://artifactory.example.com/artifactory", auth=("user", "password")
)

You can see detailed use of AdminObject in file .\tests\integration\test_admin.py

User

# Find or create first way
from dohq_artifactory import generate_password, User

user = artifactory_.find_user("username")
if user is None:
    # User does not exist
    user = User(
        artifactory_, "username", "[email protected]", password=generate_password()
    )
    user.create()

# Find or create - second way
user = User(artifactory_, "username")
if not user.read():  # Return True if user exist
    # User does not exist
    user = User(
        artifactory_, "username", "[email protected]", password=generate_password()
    )
    user.create()


# Add to group
user.add_to_group("byname")

group = artifactory_.find_group("groupname")
user.add_to_group(group)
user.update()  # Don't forget update :)

enc_pwd = user.encrypted_password

# You can re-read from Artifactory
user.read()

user.delete()

API Keys

from dohq_artifactory import User

user = User(artifactory_, "username")

# create an API key
user.api_key.create()

# get API key
user.api_key.get()
# or using str() method
my_key = str(user.api_key)
# or using repr method
print(user.api_key)

# regenerate API key if one already exists
user.api_key.regenerate()

# remove API key for current user
user.api_key.revoke()

# remove all API keys in system, only if user has admin rights
user.api_key.revoke_for_all_users()

Group

Internal

# Find
from dohq_artifactory import generate_password, Group

group = artifactory_.find_group("groupname")

# Create
if group is None:
    group = Group(artifactory_, "groupname")
    group.create()

# You can re-read from Artifactory
group.read()

# You can add multiple users at once to Group
group.users = ["admin", "anonymous"]
group.create()

# You can remove all users from a Group
group.users = []
group.create()

group.delete()

GroupLDAP

https://www.jfrog.com/confluence/display/RTF/LDAP+Groups#LDAPGroups-UsingtheRESTAPI

# Full DN path in artifactory
dn = "cn=R.DevOps.TestArtifactory,ou=Groups,dc=example,dc=com"
attr = "ldapGroupName=r.devops.testartifactory;groupsStrategy=STATIC;groupDn={}".format(
    dn
)
test_group = GroupLDAP(
    artifactory=artifactory_, name="r.devops.testartifactory", realm_attributes=attr
)
test_group.create()

RepositoryLocal

# Find
from dohq_artifactory import generate_password, RepositoryLocal

repo = artifactory_.find_repository_local("reponame")

# Create
if repo is None:
    # or RepositoryLocal.PYPI, RepositoryLocal.NUGET, etc
    repo = RepositoryLocal(artifactory_, "reponame", packageType=RepositoryLocal.DEBIAN)
    repo.create()

# You can re-read from Artifactory
repo.read()

repo.delete()

RepositoryVirtual

# Find
from dohq_artifactory import RepositoryVirtual

repo = artifactory_.find_repository_virtual("pypi.all")

# Create
if repo is None:
    # or RepositoryVirtual.PYPI, RepositoryLocal.NUGET, etc
    repo = RepositoryVirtual(
        artifactory_,
        "pypi.all",
        repositories=["pypi.snapshot", "pypi.release"],
        packageType=RepositoryVirtual.PYPI,
    )
    repo.create()

# You can re-read from Artifactory
repo.read()

local_repos = repo.repositories  # return List<RepositoryLocal>

repo.delete()

RepositoryRemote

# Find
from dohq_artifactory import RepositoryRemote

repo = artifactory_.find_repository_virtual("pypi.all")

# Create
if repo is None:
    # or RepositoryRemote.PYPI, RepositoryRemote.NUGET, etc
    repo = RepositoryRemote(
        artifactory_,
        "pypi.all",
        url="https://files.pythonhosted.org",
        packageType=RepositoryVirtual.PYPI,
    )
    repo.create()

# You can re-read from Artifactory
repo.read()

repo.delete()

Project

# Find
from artifactory import ArtifactoryPath
from dohq_artifactory import Project

artifactory_ = ArtifactoryPath(
    "http://my-artifactory/artifactory/myrepo/restricted-path", token="MY_TOKEN"
)
project = artifactory_.find_project("t1k1")

# Create
if project is None:
    project = Project(artifactory_, "t1k1", "t1k1_display_name")
    project.create()

# You can re-read from Artifactory
project.read()

project.delete()

Get repository of any type

# Find any repo by name
repo = artifactory_.find_repository("pypi.all")

Iterate over repository artifacts

# Get repo
repo = artifactory_.find_repository("pypi.all")

# Iterate over repo
for artifact in repo:
    print(artifact)
    print(artifact.properties)

# Result:
# http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.0-py3.whl
# {"pypi.name": ["my_package"], "pypy_version": ["1.0.0"]}
# http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.1.dev5-py3.whl
# {"pypi.name": ["my_package"], "pypy_version": ["1.0.1.dev5"]}
# http://my.artifactory.com/artifactory/pypi.all/other_package/other_package-0.0.1-py3.whl
# {"pypi.name": ["other_package"], "pypy_version": ["0.0.1"]}
# ...

Access repository child item

Repo can bee accessed just like any other ArtifactPath:

# Get repo
repo = artifactory_.find_repository("pypi.all")

# Access a folder within the repo
package = repo / "my_package"

# Result:
# ArtifactPath('http://my.artifactory.com/artifactory/pypi.all/my_package')

# Access a file within the repo
package = repo / "my_package" / "my_artifact.tar.gz"

# Result:
# ArtifactPath('http://my.artifactory.com/artifactory/pypi.all/my_package/my_artifact.tar.gz')

Search for certain package artifacts

# Get repo
repo = artifactory_.find_repository("pypi.all")

# Will generate and perform AQL query for getting artifacts by path or name
for artifacts in repo["my_package"]:
    print(artifact)
    print(artifact.properties)

# Result:
# http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.0-py3.whl
# {"pypi.name": ["my_package"], "pypy_version": ["1.0.0"], ...}
# http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.1.dev5-py3.whl
# {"pypi.name": ["my_package"], "pypy_version": ["1.0.1.dev5"], ...}
# http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.1.0-py3.whl
# {"pypi.name": ["my_package"], "pypy_version": ["1.1.0"], ...}
# ...

# Using partial match
for artifacts in repo["my_pack*"]:
    print(artifact)
    print(artifact.properties)

# Result:
# http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.0-py3.whl
# {"pypi.name": ["my_package"], "pypy_version": ["1.0.0"], ...}
# http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.1.dev5-py3.whl
# {"pypi.name": ["my_package"], "pypy_version": ["1.0.1.dev5"], ...}
# http://my.artifactory.com/artifactory/pypi.all/my_package_new_/my_package_new-0.0.1-py3.whl
# {"pypi.name": ["my_package_new"], "pypy_version": ["0.0.1"], ...}
# ...

Some types of repositories support specific ways of searching artifacts.

  • PyPi

    # Get repo
    repo = artifactory_.find_repository("pypi.all")
    
    # Get artifacts by package name
    for artifacts in repo["my_package"]:
        print(artifact)
        print(artifact.properties)
    
    # Result:
    # http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.0-py3.whl
    # {"pypi.name": ["my_package"], "pypy_version": ["1.0.0"], ...}
    # http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.1.dev5-py3.whl
    # {"pypi.name": ["my_package"], "pypy_version": ["1.0.1.dev5"], ...}
    # http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.1.0-py3.whl
    # {"pypi.name": ["my_package"], "pypy_version": ["1.1.0"], ...}
    # ...
    
    # Get artifacts by specific version
    for artifacts in repo["my_package==1.0.0"]:
        print(artifact)
        print(artifact.properties)
    
    # Result:
    # http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.0-py3.whl
    # {"pypi.name": ["my_package"], "pypy_version": ["1.0.0"], ...}
    
    # Using other pip operators (result should be additionaly checked!)
    for artifacts in repo["my_package!=1.0.0"]:
        print(artifact)
        print(artifact.properties)
    
    # Result:
    # http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.1.dev5-py3.whl
    # {"pypi.name": ["my_package"], "pypy_version": ["1.0.1.dev5"], ...}
    # http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.1.0-py3.whl
    # {"pypi.name": ["my_package"], "pypy_version": ["1.1.0"], ...}
    # ...
    
    
    # In case of using > or < operators, the result should be additionaly checked
    # because Artifactory compares strings, not versions
    for artifacts in repo["my_package>=1.0.0"]:
        print(artifact)
        print(artifact.properties)
    
    # Result:
    # http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.0.1.dev5-py3.whl
    # {"pypi.name": ["my_package"], "pypy_version": ["1.0.1.dev5"], ...}
    # http://my.artifactory.com/artifactory/pypi.all/my_package/my_package-1.1.0-py3.whl
    # {"pypi.name": ["my_package"], "pypy_version": ["1.1.0"], ...}
    # ...
  • Docker

    # Get repo
    repo = artifactory_.find_repository("docker.all")
    
    # Get artifacts by image name
    for artifacts in repo["my_image"]:
        print(artifact)
        print(artifact.properties)
    
    # Result:
    # http://my.artifactory.com/artifactory/docker.all/my_image/latest/manifest.json
    # {"docker.repoName": ["my_image"], "docker.manifest": ["latest"], ...}
    # http://my.artifactory.com/artifactory/docker.all/my_image/1.0.0/manifest.json
    # {"docker.repoName": ["my_image"], "docker.manifest": ["1.0.0"], ...}
    # http://my.artifactory.com/artifactory/docker.all/my_image/1.1.0/manifest.json
    # {"docker.repoName": ["my_image"], "docker.manifest": ["1.1.0"], ...}
    # ...
    
    # Get artifacts by specific version
    for artifacts in repo["my_image:1.0.0"]:
        print(artifact)
        print(artifact.properties)
    
    # Result:
    # http://my.artifactory.com/artifactory/docker.all/my_image/1.0.0/manifest.json
    # {"docker.repoName": ["my_image"], "docker.manifest": ["1.0.0"], ...}
    # ...
    
    for artifacts in repo["my_image:latest"]:
        print(artifact)
        print(artifact.properties)
    
    # Result:
    # http://my.artifactory.com/artifactory/docker.all/my_image/latest/manifest.json
    # {"docker.repoName": ["my_image"], "docker.manifest": ["latest"], ...}
    # ...
    
    # Partial search
    for artifacts in repo["my_package:*dev*"]:
        print(artifact)
        print(artifact.properties)
    # http://my.artifactory.com/artifactory/docker.all/my_image/dev/manifest.json
    # {"pypi.name": ["my_package"], "pypy_version": ["dev"]}
    # http://my.artifactory.com/artifactory/docker.all/my_image/1.0.1-dev5/manifest.json
    # {"pypi.name": ["my_package"], "pypy_version": ["1.0.1-dev5"]}
    # ...
  • Maven

    # Get repo
    repo = artifactory_.find_repository("maven.all")
    
    # Get artifacts by group name
    for artifacts in repo["my.group"]:
        print(artifact)
        print(artifact.properties)
    
    # Result:
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/maven-metadata.xml
    # ...
    # http://my.artifactory.com/artifactory/maven.all/my/group/another_package/1.2.3/maven-metadata.xml
    # ...
    
    # Get artifacts by group and package name
    for artifacts in repo["my.group:package"]:
        print(artifact)
        print(artifact.properties)
    
    # Result:
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/maven-metadata.xml
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/package-1.0.0-source.jar
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/package-1.0.0-javadoc.jar
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/package-1.0.0.pom
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/package-1.0.0.jar
    # {"build.number": ["123"], "build.name": ["1.0.0"], ...}
    
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.1/maven-metadata.xml
    # ...
    
    # Get artifacts by group, package name and version
    for artifacts in repo["my.group:package:1.0.0"]:
        print(artifact)
        print(artifact.properties)
    
    # Result:
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/maven-metadata.xml
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/package-1.0.0-source.jar
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/package-1.0.0-javadoc.jar
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/package-1.0.0.pom
    # http://my.artifactory.com/artifactory/maven.all/my/group/package/1.0.0/package-1.0.0.jar
    # {"build.number": ["123"], "build.name": ["1.0.0"], ...}

PermissionTarget

Docs: https://www.jfrog.com/confluence/display/RTF/Managing+Permissions

Supports these roles:

  • PermissionTarget.ROLE_ADMIN = ADMIN + DELETE + DEPLOY + ANNOTATE + READ
  • PermissionTarget.ROLE_DELETE = DELETE + DEPLOY + ANNOTATE + READ
  • PermissionTarget.ROLE_DEPLOY = DEPLOY + ANNOTATE + READ
  • PermissionTarget.ROLE_ANNOTATE = ANNOTATE + READ
  • PermissionTarget.ROLE_READ = READ

And for more modular control:

  • PermissionTarget.ADMIN - Allows changing the permission settings for other users on this permission target
  • PermissionTarget.DELETE - Allows deletion or overwriting of artifacts
  • PermissionTarget.DEPLOY - Allows deploying artifacts and deploying to caches (i.e. populating caches with remote artifacts)
  • PermissionTarget.ANNOTATE - Allows annotating artifacts and folders with metadata and properties
  • PermissionTarget.READ - Allows reading and downloading of artifacts
from dohq_artifactory import PermissionTarget

permission = artifactory_.find_permission_target("rule")

# See repositories, users or groups
permission.repositories
# Result:
# <RepositiryLocal repo1>
# <RepositiryLocal repo2>

permission.users
# Result:
# <User user1>
# <User user2>

permission.groups
# Result:
# <Group group1>
# <Group group2>

# Add repo (string or Repository) object
permission.add_repository("repo3", "repo4")
permission.add_repository(repo5_object)
# Or remove
permission.remove_repository("repo1", "repo2")

# Add user (string or User object) with specific permission
permission.add_user("user3", PermissionTarget.ROLE_ADMIN)
permission.add_user(
    user4_object, PermissionTarget.ROLE_READ + PermissionTarget.ROLE_WRITE
)  # You can add sum of permissions

# Or remove
permission.remove_user("user1", "user2")

# Add group (string or Group object) with permission
permission.add_group("group3", PermissionTarget.ROLE_ADMIN)
permission.add_group(
    group4_object, PermissionTarget.ROLE_READ + PermissionTarget.ROLE_WRITE
)  # You can add sum of permissions

# Or remove
permission.remove_group("group1", "group2")

permission.update()  # Update!!

permission.repositories
# Result:
# <RepositiryLocal repo3>
# <RepositiryLocal repo4>
# <RepositiryLocal repo5>

permission.users
# Result:
# <User user3>
# <User user4>

permission.groups
# Result:
# <Group group3>
# <Group group4>

Token

https://www.jfrog.com/confluence/display/RTF5X/Access+Tokens#AccessTokens-RESTAPI

from requests.auth import HTTPBasicAuth
from artifactory import ArtifactoryPath
from dohq_artifactory import Token

session = ArtifactoryPath(
    "https://artifactory_dns/artifactory",
    auth=("admin", "admin_password"),
    auth_type=HTTPBasicAuth,
    verify=False,
)

# Read token for readers group
group_name = "readers"
scope = "api:* member-of-groups:" + group_name
token = Token(session, scope=scope)
token.read()

# Create token for member of the readers
group_name = "readers"
scope = "api:* member-of-groups:" + group_name
subject = group_name
token = Token(
    session, scope=scope, username=subject, expires_in=31557600, refreshable=True
)
response = token.create()

print("Readonly token:")
print("Username: " + token.username)
print("Token: " + token.token["access_token"])

Common

All AdminObject support:

user = artifactory_.find_user("username")
print(user.raw)  # JSON response from Artifactory

new_repo = RepositoryLocal(artifactory, "reponame")
# If some key you can't find in object, you can use this:
new_repo.additional_params["property_sets"] = ["my", "properties_sets"]
new_repo.create()

# All object support CRUD operations:
obj.read()  # Return True if user exist (and read from Artifactory), else return False
obj.create()
obj.update()
obj.delete()

# ArtifactoryPath have different find_ method:
artifactory_.find_user("name")
artifactory_.find_group("name")
artifactory_.find_repository_local("name")
artifactory_.find_permission_target("name")
artifactory_.find_project("project_key")

Advanced

Session

To re-use the established connection, you can pass session parameter to ArtifactoryPath:

from artifactory import ArtifactoryPath
import requests

ses = requests.Session()
ses.auth = ("username", "password")
path = ArtifactoryPath(
    "http://my-artifactory/artifactory/myrepo/my-path-1", sesssion=ses
)
path.touch()

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/myrepo/my-path-2", sesssion=ses
)
path.touch()

SSL Cert Verification Options

See Requests - SSL verification for more details.

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/libs-snapshot-local/myapp/1.0"
)

... is the same as

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/libs-snapshot-local/myapp/1.0", verify=True
)

Specify a local cert to use as client side certificate

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/libs-snapshot-local/myapp/1.0",
    cert="/path_to_file/server.pem",
)

Disable host cert verification

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/libs-snapshot-local/myapp/1.0", verify=False
)

Note: If host cert verification is disabled, urllib3 will throw a InsecureRequestWarning. To disable these warning, one needs to call urllib3.disable_warnings().

import requests.packages.urllib3 as urllib3

urllib3.disable_warnings()

Timeout on requests

The library supports timeout argument in the same meaner as requests does

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/libs-snapshot-local/myapp/1.0"
)

... is the same as

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/libs-snapshot-local/myapp/1.0", timeout=None
)

Set 5 seconds timeout to your requests after which it will be terminated:

from artifactory import ArtifactoryPath

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/libs-snapshot-local/myapp/1.0", timeout=5
)

Logging

The library can be configured to emit logging that will give you better insight into what it's doing. Just configure logging module in your python script. Simplest example to add debug messages to a console:

import logging
from artifactory import ArtifactoryPath

logging.basicConfig()
# set level only for artifactory module, if omitted, then global log level is used, eg from basicConfig
logging.getLogger("artifactory").setLevel(logging.DEBUG)

path = ArtifactoryPath(
    "http://my-artifactory/artifactory/myrepo/restricted-path", apikey="MY_API_KEY"
)

Global Configuration File

Artifactory Python module also can specify all connection-related settings in a central file, given by environment variable $DOHQ_ARTIFACTORY_PYTHON_CFG (default if not set: ~/.artifactory_python.cfg) that is read upon the creation of first ArtifactoryPath object and is stored globally. For instance, you can specify per-instance settings of authentication tokens, so that you won't need to explicitly pass auth parameter to ArtifactoryPath.

Example:

[DEFAULT]
username = nameforallinstances

[http://artifactory-instance.com/artifactory]
password = ilikerandompasswords
verify = false

[another-artifactory-instance.com/artifactory]
password = @dmin
cert = ~/mycert

Whether or not you specify http:// or https://, the prefix is not essential. The module will first try to locate the best match and then try to match URLs without prefixes. So in the config, if you specify https://my-instance.local and call ArtifactoryPath with http://my-instance.local, it will still do the right thing.

Contribute

About contributing and testing

Advertising

  • artifactory-du - estimate file space usage. Summarize disk usage in JFrog Artifactory of the set of FILEs, recursively for directories.
  • artifactory-cleanup - is an extended and flexible cleanup tool for JFrog Artifactory.

artifactory's People

Contributors

akshaysarraf2016 avatar alanyee avatar allburov avatar arlington1985 avatar axnk avatar beliaev-maksim avatar briantist avatar calvinatnvidia avatar chipjust avatar christiansandberg avatar derrix060 avatar dolfinus avatar donhui avatar flichtenheld avatar fuzzmz avatar gsemet avatar jandvorak-sol60279 avatar jfaith-impinj avatar jravetch avatar jugmac00 avatar nacc avatar nikolasj avatar pre-commit-ci[bot] avatar stealthii avatar tdohany avatar thaligar avatar tim55667757 avatar vasokot avatar yuvalsimon avatar zhan9san avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

artifactory's Issues

ะŸะตั€ะตะธะผะตะฝะพะฒะฐั‚ัŒ pypi-ะฟะฐะบะตั‚ ะฒ dohq-artifactory

dohq_art - ะฝะตะฟะพะฝัั‚ะฝะพ ั‡ั‚ะพ ะทะฐ "ะธัะบัƒััั‚ะฒะพ" dohq, ั…ะพั‡ะตั‚ัั ะฒะธะดะตั‚ัŒ ะฟะพะปะฝะพะต, ะฑะตะท ัะพะบั€ะฐั‰ะตะฝะธะน ะธะผั dohq_artifactory

Documention on Integration Tests is missing and add them to CI

The integration tests aren't run on the Travis CI, @allburov do you run them against your own Artifactory Instance?

I know that the API isn't available without an Artifactory license, I've just sent them a support request asking if they could also add API scope to the free Artifactory version, otherwise I don't think it will be possible to run integration tests against via. Travis.

Seems go-artifactory is getting an Artifactory Cloud instance sponsored for their CI tests.

Artifactory Query Language get the latest artifact in a path

How would one sort and limit results when using the library? Basically, get the latest artifact in path.

Using the API I see that one can do something like this: .sort({"$desc" : ["created"]}).limit(1) with find.

But how to include this when using the library? I'm not sure how to include the sort and limit in the below code.

aql = ArtifactoryPath("https://myartifactory.com/artifactory", auth=('username','password'))

args = ["items.find", {"$and": [
        { "repo": {"$eq": "myrepo"} },
        {
            "$or": [
                {"path": {"$match": "path/in/repo"}}
            ]
        }
        ]
}]

artifacts_list = aql.aql(*args)

Fail in get_stat_json on virtual repository

This is copied from a ticket that was opened in the deprecated repository. The issue is still present in this fork.

Hi,

I am trying to upload via the deploy_file method, and i had issue uploading through a virtual repository with error "404, Not found", what does not make sense because i am trying to upload a new file that he shouldn't be exists any way.

So after debugging the error trace back i found that in the API when i try to reach a file is not exists, i had different behavior if it is virtual repo or not;

in regular repository i get this:

{ "errors" : [ { "status" : 404, "message" : "Unable to find item" } ] }

however in the virtual repository i get a different error message:

{ "errors" : [ { "status" : 404, "message" : "Not Found" } ] }

And the library method artifactory.py in get_stat_json (Line 457),

if code == 404 and "Unable to find item" in text:

Which make this condition not working in virtual repository.

Thank you very much.

Support X-JFrog-Art-Api authentication

Hello,

I started to use your nice library along with the Jfrog CLI, and the CLI does not need the username when authenticating. The API Key already has this information in it, but encrypted.

The Artifactory Rest API provides a way to use it with the X-JFrog-Art-Api header. So in order to make dohq-artifactory apikey-friend, I just implemented my own authentication class:

class XJFrogArtApiAuth(AuthBase):
    """Attaches X-JFrog-Art-Api Authentication to the given Request object."""

    def __init__(self, apikey):
        self.apikey = apikey

    def __eq__(self, other):
        return all([
            self.apikey == getattr(other, 'apikey', None),
        ])

    def __ne__(self, other):
        return not self == other

    def __call__(self, r):
        r.headers['X-JFrog-Art-Api'] = self.apikey
        return r

So user can call it with ArtifactoryPath(path, auth=XJFrogArtApiAuth(apikey)).

[develop] Glob fails when urllib3 > 1.25

This issue is linked to the develop branch
With urllib3 > 1.25, the URL parsing is more constrained.

For instance, with urllib3==1.24.3:

> urllib3.util.parse_url('15.0.0.300')
Url(scheme=None, auth=None, host='15.0.0.300', port=None, path=None, query=None, fragment=None)

With urllib3==1.25, the same call leads to the following exception: urllib3.exceptions.LocationParseError: Failed to parse: //15.0.0.300

This introduction was to explain why when you are walking folder tree, it can occur that a folder is named '15.0.0.300'. In this case, the _ArtifactoryFlavour.splitroot method raise this exception since the function usrllib3.url.parse_url is called on every subfolder name.

File "/home/ubuntu/python_venv/artifactory-cleaner/lib/python3.6/site-packages/artifactory.py", line 360, in splitroot
    url = urllib3.util.parse_url(part)
  File "/home/ubuntu/python_venv/artifactory-cleaner/lib/python3.6/site-packages/urllib3/util/url.py", line 234, in parse_url
    raise LocationParseError(url)
urllib3.exceptions.LocationParseError: Failed to parse: //15.0.0.308

Support for Deploy Artifact by Checksum

What do you guys think about adding support for deploying artifacts by checksum? It uses the checksum to check if the artifact already exists on the server and copies it over to the new location without requiring the upload.

I think it'd be good to use for default behavior because the result is the same as a normal deploy, other than the fact that it doesn't require the client to upload.

No support python 3.7

If i try use this with python 3.7.3 i get this error, i think this could be a issue from main artifatory package

pip install dohq-artifactory
Python 3.7.3 (default, Mar 27 2019, 16:54:48)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import artifactory
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/rafael.sathler/projetos/hadoop-deploy/artifactory.py", line 1, in <module>
    from artifactory import ArtifactoryPath
ImportError: cannot import name 'ArtifactoryPath' from 'artifactory' (/Users/rafael.sathler/projetos/hadoop-deploy/artifactory.py)

but with python 2.7 i can import the module without problems


root@bf72a057cec9:/# python
Python 2.7.16 (default, Mar 27 2019, 09:43:28)
[GCC 6.3.0 20170516] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from artifactory import ArtifactoryPath
>>> 

AttributeError: 'ArtifactoryPath' object has no attribute 'aql'

Steps to reproduce:

  1. Install dohq-artifactory, and then install artifactory
pip3 install dohq-artifactory
pip3 install artifactory
from artifactory import ArtifactoryPath
ArtifactoryPath('https://MY_URL', apikey='MY_API_KEY').aql

This happens because installing artifactory overwrites SITE_PACKAGES/artifactory.py that was also used by dohq-artifactory. Now I NEED to have both packages installed, and dohq-artifactory will always be installed before artifactory.

I'm working on a pipeline that uses framework-plugin architecture.
The "framework" is some python script in a docker image, and dohq-artifactory is already installed in the image.

And the pipeline runs every night, and it goes like this...
Day 1:

  1. Get a list of available plugins (uses AQL)
  2. Install available "plugins" and their requirements (including artifactory)

Day 2:

  1. Get a list of available plugins (uses AQL, but it fails because artifactory got installed yesterday)

I guess the questions are:

Why are there two separate packages in the first place? I'm using dohq-artifactory because it supports AQL (and some other neat enhancements I think). But couldn't these enhancements have happened in the original artifactory repo? And if you wanted to develop an entirely separate package, why keep the same file name? Why not just let users do from dohq_artifactory import ArtifactoryPath? I mean it's not as pretty as from artifactory but I feel like it's even worse to introduce conflicts with other PyPI packages. So basically you're forcing users to use one or the other. And even if you actually want to force users to use only one of these packages, this should be at least documented somewhere, or there should be a warning message during installation.

What do you suggest I do in this situation? I don't want to have to go to every team manager and tell them to start using dohq-artifactory instead. You probably don't want to change the file name because that means all of your users will have to start using the new name.

Documentation

In the docu is a small issue with the installation part:

python3 -mpip install dohq-artifactory

should be

python3 -m pip install dohq-artifactory

Thanks,
Markus

404 when dealing with remote repository

Hello, when trying to get the status or download an artefact that leaves in a remote repository, I get a 404.

>>> url = 'http://[my domain]/artifactory/[dir]/[file]'
>>> path = artifactory.ArtifactoryPath(url, auth=(_AF_USER, _AF_KEY))
>>> path.stat()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mario/miniconda3/envs/tauEnvTmp/lib/python2.7/site-packages/pathlib.py", line 1051, in stat
    return self._accessor.stat(self)
  File "/home/mario/miniconda3/envs/tauEnvTmp/lib/python2.7/site-packages/artifactory.py", line 515, in stat
    jsn = self.get_stat_json(pathobj)
  File "/home/mario/miniconda3/envs/tauEnvTmp/lib/python2.7/site-packages/artifactory.py", line 494, in get_stat_json
    raise RuntimeError(text)
RuntimeError: {
  "errors" : [ {
    "status" : 404,
    "message" : "Not Found"
  } ]
}


>>> with path.open() as fd:
...     with open(path.name, 'wb') as out:
...         out.write(fd.read())
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mario/miniconda3/envs/tauEnvTmp/lib/python2.7/site-packages/artifactory.py", line 1054, in open
    return self._accessor.open(self)
  File "/home/mario/miniconda3/envs/tauEnvTmp/lib/python2.7/site-packages/artifactory.py", line 678, in open
    raise RuntimeError("%d" % code)
RuntimeError: 403

If I copy the link on my browser, I'm able to download the file, and then if I try to run the command again, it succeeds.

is_file() raises when the artifacts' name contains square brackets

The following exception is raised when is_file() method is called for an artifact where its name contains square branckets ('[' or ']'):

if artifact.is_file():

File "C:\Python37-32\lib\site-packages\artifactory.py", line 1035, in is_file
return self._accessor.is_file(self)
File "C:\Python37-32\lib\site-packages\artifactory.py", line 534, in is_file
stat = self.stat(pathobj)
File "C:\Python37-32\lib\site-packages\artifactory.py", line 492, in stat
jsn = self.get_stat_json(pathobj)
File "C:\Python37-32\lib\site-packages\artifactory.py", line 471, in get_stat_json
raise RuntimeError(text)
RuntimeError: {
"errors" : [ {
"status" : 400,
"message" : "Bad Request"
} ]
}

ArtifactoryPath.set_properties should use Update item properties API instead

There is an issue with using the Set item properties API in artifactory. When setting a lot of properties you can run into the issue where the URI becomes too long.

If set_properties use the Update item properties API no such issue exists since it takes json data as input with a PATCH request.

https://www.jfrog.com/confluence/display/RTF/Artifactory+REST+API#ArtifactoryRESTAPI-UpdateItemProperties

The request can add, update and delete properties which means that it can basically do all actions one wants to do with properties.

Should a new method be implemented (update_properties) or should set_properties be rewritten to use the update item properties API? Because I'm willing to do a PR for this as long as I know how you want it implemented.
Using a new method will mirror the API endpoins that exist. But updating set_properties will remove the issue with URI too long.

Add `security/token` class

Currently the AdminObject only allows _create_and_update() by a JSON, since the API also have non-json "endpoints" such as "Create Token I need to figure out a clean way of expanding the AdminObject to an additional class called Token. (Similar to User and Group classes).

Going to add this at work,
if you feel like giving me suggestions how you'd prefer to have it implemented - feel free to reach out here.

PR might follow.

Add some documentation on how to troubleshoot and debug

Please add some documentation on how to troubleshoot and debug

I had to add a custom logger for requests.packages.urllib3 to see the requests to my artifactory server. It would be nice to have a flag that could do this for me.

User.update() fails with Bad request when updating the user

What I'm trying to do is to update a user by adding groups to it. So, the user and groups exist beforehand, but when trying to update I get:

requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://servername/artifactory/api/security/users/sampleuser

Trying to replicate the operation using Postman, I think the problem because when you are updating the user you need to use the method Post, but the library is using Put to create (this is OK), but also to update, hence the bad request.

Upload artifactory and dohq-artifactory to pypi

We have API KEY or we can contact with parallels, get permission to pypi and upload dohq-artifactory to pypi with two names:

  1. artifactory - we can replace it because we didn't do breaking changes
  2. dohq-artifactory for backward compatibility in requirements.in|txt

It will close question like this #76

Artifactory >=5.0 - REST API endpoints which used a pipe character ("|") as a separator have undergone corresponding changes so you can use a semicolon (";")

The version of Tomcat used in Artifactory 5.0 has been upgraded to 8.0.39. This version of Tomcat no longer supports unencoded URLs, so the REST API endpoints which used a pipe character ("|") as a separator have undergone corresponding changes so you can use a semicolon (";") as a separator instead, or use escaping to represent a pipe as %7C. Any scripts that use these endpoints may have to be changed to support the new specification. For details, please refer to Set Item Properties as an example.

https connection seem broken with apl

Hi,

this morning I'm used the API by work and here what I can remember.

Following code was executed:

from artifactory import ArtifactoryPath
aql = ArtifactoryPath( "https://my-artifactory/artifactory", verify=False)
artifacts = aql.aql("items.find", {"repo": "myrepo"})

When the line apl.apl was executed the connection could not verify SSL connection, even if the verification was set to false.

Could somebody take an look?

Thanks,
Markus

Tilde in file name expanded incorrectly on Linux - works on Mac

Linux (Dockerfile at the bottom): upload success

root@a21efe682276:/# touch file~with~tilde
root@a21efe682276:/# ls -la
total 72
drwxr-xr-x   1 root root 4096 Sep  5 01:56 .
drwxr-xr-x   1 root root 4096 Sep  5 01:56 ..
-rw-r--r--   1 root root    0 Sep  5 01:56 file~with~tilde
root@a21efe682276:/# python3
Python 3.6.8 (default, Jan 14 2019, 11:02:34) 
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from artifactory import ArtifactoryPath
>>> target = ArtifactoryPath("https://myrt/my-generic-local/misc/", auth=('calvinp', 'mypwd'))
>>> target.deploy_file("file~with~tilde")

Linux: copy fails

>>> origin = ArtifactoryPath("https://myrt/my-generic-local/misc/file~with~tilde", auth=('calvinp', 'mypwd'))
>>> target = ArtifactoryPath("https://myrt/my-generic-local/misc/file~with~tilde~two", auth=('calvinp', 'mypwd'))
>>> origin.copy(target)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/artifactory.py", line 1299, in copy
    with self.open() as fobj:
  File "/usr/local/lib/python3.6/dist-packages/artifactory.py", line 1099, in open
    return self._accessor.open(self)
  File "/usr/local/lib/python3.6/dist-packages/artifactory.py", line 717, in open
    raise RuntimeError("%d" % code)
RuntimeError: 404

Linux: notice the incorrectly expanded tildes

>>> origin 
ArtifactoryPath('https://myrt/my-generic-local/misc/file~with~tilde/my-generic-local/misc/file%7Ewith%7Etilde')

Mac: upload success

rt $ touch file~with~tilde
rt $ ls -la
total 0
drwxr-xr-x 3 calvin staff  96 Sep  4 18:40 .
drwxr-xr-x 8 calvin staff 256 Sep  4 18:37 ..
-rw-r--r-- 1 calvin staff   0 Sep  4 18:40 file~with~tilde
rt $ python3
Python 3.7.4 (default, Jul  9 2019, 18:13:23) 
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from artifactory import ArtifactoryPath
>>> target = ArtifactoryPath("https://myrt/my-generic-local/misc/", auth=('calvinp', 'mypwd'))
>>> target.deploy_file("file~with~tilde")

Mac: copy success

>>> origin = ArtifactoryPath("https://myrt/my-generic-local/misc/file~with~tilde", auth=('calvinp', 'mypwd'))
>>> target = ArtifactoryPath("https://myrt/my-generic-local/misc/file~with~tilde~two", auth=('calvinp', 'mypwd'))
>>> origin.copy(target)

Mac: tilde correctly not expanded

>>> origin
ArtifactoryPath('https://myrt/my-generic-local/misc/file~with~tilde')

Dockerfile for failed env

FROM ubuntu:bionic

RUN apt-get -qq update && DEBIAN_FRONTEND=noninteractive apt-get -yqq install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

RUN curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
RUN add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

RUN apt-get -qq update && DEBIAN_FRONTEND=noninteractive apt-get -yqq install \
    containerd.io \
    docker-ce \
    docker-ce-cli \
    python3 \
    python3-pip

RUN pip3 install -qr \
    docker
    dohq-artifactory
    requests

ArtifactoryPath returns incorrect path if URL doesn't contain artifactory outside of domain

The ArtifactoryPath method returns an incorrect value if the string passed to it doesn't contain the word artifactory somewhere outside of the domain.

For example, if setting up Artifactory behind NGINX with a custom domain, then you have the following behavior:

path = ArtifactoryPath("http://af.demo.local/gradle-ivy-local")
print(path)

Returns http:/af.demo.local/gradle-ivy-local (notice just the once forwards slash at the scheme level)

Using path = ArtifactoryPath("http://artifactory.demo.local/gradle-ivy-local") returns http:/artifactory.demo.local/gradle-ivy-local (still just one forward slash).

If you add artifactory somewhere in the URL (doesn't really matter where) then the path resolves correctly.

For example using path = ArtifactoryPath("http://artifactory.demo.local/gradle-ivy-local/artifactory") returns http://artifactory.demo.local/gradle-ivy-local/artifactory.

Unlinking a file yields "could not locate artifact"

I tried unlinking a file from artifactory using dohq_artifactory-0.4.102
I got an error 404 "Could not locate artifact"

I then tried uninstalling the aformentioned version of dohq-artifactory and installing artifactory-0.1.17 which seemed to successfully unlink the file.

Uploading to Pypi repository

Hi, this isn't a bug but more a question on how to leverage this tool to interface to a PyPi repository on Artifactory.

If I use the standard setuptools approach to upload a wheel: python setup.py bdist_wheel upload ...
The wheel gets uploaded and is searchable and downloadable via pip. This was done as a test with an initial 0.1 version of a module.

I have tried this tool to upload a new version: 0.2 of the same module. On artifactory, I can see that it has been successfully uploaded and the tree looks correct:

Below are just made up names for illustration:

module_name
     | ----0.1
     |     | module_0.1_wheel.whl
     | ----0.2
           | module_0.2_wheel.whl

However, the 0.2 version is searchable nor is it downloadable via pip.

Is there an additional step that is required, after actually uploading the wheel?

Thanks.

/ operator on ArtifactoryPath fails if string contains "/"

Maybe linked to #31.

>>> arti = Artifactory("http://my.artifactory.ext/artifactory")
>>> arti
ArtifactoryPath('http://my.artifactory.ext/artifactory')
>>> arti / "foo"
ArtifactoryPath('http://my.artifactory.ext/artifactory/foo')
>>> arti / "foo" / "bar"
ArtifactoryPath('http://my.artifactory.ext/artifactory/foo/bar')
>>> arti / "foo/bar"
ArtifactoryPath('foo/bar/')  # expected: http://my.artifactory.ext/artifactory/foo/bar

KeyError if optional fields on LocalRepository are missing

When calling find_local_repositories() and response json object doesnt comprise a repoLayoutRef a KeyError exception is thrown.

...
  File "/home/xxx/Development/misc/python-deployment-helper/artifact.py", line 64, in promote
    local_repos = get_local_repositories(source.repo)
  File "/home/xxx/Development/misc/python-deployment-helper/artifact.py", line 29, in get_local_repositories
    l = artifactory_.find_repository_local(repo)
  File "/home/xxx/.local/share/virtualenvs/python-deployment-helper-T1E8SZRN/lib/python3.6/site-packages/artifactory.py", line 1432, in find_repository_local
    if obj.read():
  File "/home/xxx/.local/share/virtualenvs/python-deployment-helper-T1E8SZRN/lib/python3.6/site-packages/dohq_artifactory/admin.py", line 113, in read
    self._read_response(response)
  File "/home/xxx/.local/share/virtualenvs/python-deployment-helper-T1E8SZRN/lib/python3.6/site-packages/dohq_artifactory/admin.py", line 346, in _read_response
    self.layoutName = response['repoLayoutRef']
KeyError: 'repoLayoutRef'

According to https://www.jfrog.com/confluence/display/RTF/Repository+Configuration+JSON these fields are optional and therefore should not be required.

Add Artifactory Query Language support

I want run AQL:

querty_dict = {
'repo': {
    '$eq': 'myreponame',
}}
artifacts = ArtifactoryAPI("https://server.fqn", 'username', 'password').run_aql('items.find', query_dict)
artifacts_with_properties = ArtifactoryAPI("https://server.fqn", 'username', 'password').run_aql('items.find', query_dict, 'include("*", "property")')

# artifacts* - list of ArtifactoryPath

Add operation with user\repo\group\permission

Add operation with user\repo\group, e.g. CRUD operation

Users operations:

  • Create
  • Search
  • Add User to Group(s)
  • Add User to Permission
  • ? Change
  • ? Remove

Group operations:

  • Create
  • Search
  • LDAP Group supported
  • Add Group to Permission
  • ? Change
  • ? Remove

Repo operations:

  • Create
  • ? Search
  • Add Repo to Permission
  • Remove

Permission operations:

  • Create
  • Search
  • Add Repo, User, Group to Permission

Documentaion:

  • About all operations
  • How tests this features (with test artifactory instance)

Drop support python <=3.4

@georgek has suggest dropping support python < 3.4 in #84
We can drop support 3.4 also, because this release has reached end-of-life.

What is a better python-way to do this?

It will be cool if old versions of library will be available for python <=3.4 and new only for python > 3.4, how we do reach this?

pip3.4 install dohq-artifactory # find 0.4.112

pip3.7 install dohq-artifactory # will find 1.0.1, e.g

Classifier like Programming Language :: Python :: 3.2 it's helpful metadata, but pip don't use it for search.

ask for new fields in ArtifactoryFileStat

Hi there,
For prune purpose, it seems the "Last Downloaded" timestamp is more valuable.
If the artifact has been frequently downloaded and this timestamp got refreshed continuously, we will keep it in spite of its old "ctime" value.
Please help check this.
Thanks.

build promotion

Does this library support build promotion? I know that copying is metioned in the Readme but I'm looking to replace the otherwise flakey jenkins build promotion with a manual script. Just copying isn't enough because I want the artifacts within the build that I'm promoting to be updated to point to the new repository that I've promoted the build to.

Thanks

API Key for authentication

Hello,

Is it possible to enhance this module with the "API Key for authentication" feature?

Thanks,
volkan

minor issue when using / operator

Hello

When using the / operator again a ArtifactoryPath without repository, the url seems wrong.
Example:

>>> ArtifactoryPath(http://myartifactory.server.com/artifactory) / "reponame" / "path"
http://myartifactory.server.com/artifactoryreponame/path

Instead of http://myartifactory.server.com/artifactoryreponame/path.

If the reponame starts with a /, it works great:

>>> ArtifactoryPath(http://myartifactory.server.com/artifactory) / "/reponame" / "path"
http://myartifactory.server.com/artifactory/reponame/path

This does not seem to have any impact on the api however.

Add richer Group API request

Update: Just sent JFrog an E-Mail asking for documentation on this endpoint.

Currently admin.py is using the only documented way of retrieving details of a group by calling the api/security/group/<group id> endpoint.

This endpoint doesn't provide any information on which users are in a group.

If you go into the Artifactory UI to Admin => Security => Groups => Group you'll see that you can indeed get from the backend which members are in a certain group.

Via. web-inspector you'll see that it's sending a request to ui/groups/<group id> which results in a richer JSON. I've also just tried accessing api/groups/<group id> which returns the similar JSON.

I've now checked their application.wadl and the endpoint is also mentioned there:

<resources base="http://localhost:8081/artifactory/api/">
<resource path="groups">
<method id="getGroups" name="GET">
<response><representation mediaType="application/json"/></response>
</method>
<method id="createGroup" name="POST"></method>
<resource path="delete"></resource>
<resource path="{id : [^/]+}">
</resource>
<resource path="{id: [^/]+}">
<method id="getGroup" name="GET">
<response><representation mediaType="application/json"/></response>
</method>
</resource>
</resource>

Example JSON of the undocumented endpoint:

{
  "name": "example-group",
  "description": "My description",
  "autoJoin": false,
  "realm": "internal",
  "adminPrivileges": false,
  "external": false,
  "usersInGroup": [
    "user",
    "admin"
  ],
  "newUserDefault": false,
  "groupName": "example-group"
}

Example JSON of the currently used endpoint:

{
  "name" : "example-group",
  "description" : "My description",
  "autoJoin" : false,
  "realm" : "internal",
  "adminPrivileges" : false
}

I've also just looked at what data the UI is sending, and it appears it's still sending "autoJoin": false, but not doing anything with this data in the UI. I think it is possible that autoJoin still exists, but is dropped in favor of the data that is send when you check Automatically Join New Users to this Group in the setting which is newUserDefault.
I've just send a request with autoJoin: true and newUserDefault: false which resulted in newUserDefault being what was used for storing the preference.

Telling by their old documentation:
- "autoJoin" : false (default, must be false if adminPrivileges is true),

And clicking around in the Group setting I'm super confident that they've renamed the setting.

PUT into /api/groups/example-group

{
  "name": "example-group",
  "description": "Example Description",
  "autoJoin": false,
  "realm": "internal",
  "adminPrivileges": false,
  "external": false,
  "usersInGroup": [],
  "newUserDefault": true,
  "groupName": "example-group"
}

The function splitroot() does not work for Cloud Enterprise artifactory repos

Our Cloud Enterprise repo name path is organized like this:

https://<Company_name>.jfrog.io/<Company_name>/

the function splitroot() in _ArtifactoryFlavour assumes the format:

http://mysite/artifactory

and attempts to parse based on /artifactory/
It was incorrectly parsed in my use case and I needed to hardcode <Company_name> in place of /artifactory/
To support more use cases, an option to pass in the "Drive" and "path" separately into the ArtifactoryPath constructor to bypass the parsing would be very nice

Cannot download latest version

Cannot get the latest version of dohq-artifactory using pip.

python --version
Python 3.7.1

pip freeze | grep dohq-artifactory
dohq-artifactory==0.4.106

python -mpip install dohq-artifactory==0.4.107
Collecting dohq-artifactory==0.4.107
  Could not find a version that satisfies the requirement dohq-artifactory==0.4.107 (from versions: 0.2.10, 0.2.15, 0.2.16, 0.2.26, 0.2.42, 0.2.48, 0.2.49, 0.2.59, 0.2.62, 0.2.65, 0.3.99,
0.3.101, 0.3.102, 0.3.103, 0.4.100, 0.4.101, 0.4.102, 0.4.103, 0.4.104, 0.4.105, 0.4.106)
No matching distribution found for dohq-artifactory==0.4.107

Python 3.6 compatibility; Path.glob()

Parallels/artifactory#49

Hi,

It appears that globbing behavior in Pathlib has changed/optimized to use scandir in 3.6: https://bugs.python.org/issue26032

  File "/usr/lib/python3.6/pathlib.py", line 1081, in glob
    for p in selector.select_from(self):
  File "/usr/lib/python3.6/pathlib.py", line 489, in select_from
    scandir = parent_path._accessor.scandir
AttributeError: '_ArtifactoryAccessor' object has no attribute 'scandir'

Trivially implementing scandir on _ArtifactoryAccessor results in a second failure, so this may be more than just adding a new method that wraps listdir.

'children' does not give list of filenames, only sub-directory names

I understand this seems to be something very basic but when I use this library to get list of files from a certain Artifactory path, I notice that in 'children', I get only subdirectory names and no names of the files which are also present in the directory. I am not sure if this is because I am using an incompatible version of JFrog Artifactory which is 4.4.3

find_group exception if description empty

artifactory_.find_group(group_name_ro) command fails if description of the group is empty:

Traceback (most recent call last):
  File ".\test_artifact.py", line 30, in <module>
    group = artifactory_.find_group(group_name_ro)
  File "C:\Users\xxx\new_artifactory\venv\lib\site-packages\artifactory.py", line 1395, in find_group
    if obj.read():
  File "C:\Users\xxx\new_artifactory\venv\lib\site-packages\dohq_artifactory\admin.py", line 113, in read
    self._read_response(response)
  File "C:\Users\xxx\new_artifactory\venv\lib\site-packages\dohq_artifactory\admin.py", line 247, in _read_response
    self.description = response['description']
KeyError: 'description'

Add complex AQL example to README.md

Hello, I intend to use the following aql. Can you please suggest how can it be formed using the python object?
items.find({"$or":[{"repo": "docker-prod"}], "type":"file","created" : {"$gt" : "2019-07-10T19:20:30.45+01:00"}}).include("repo","path","name",).sort({"$asc" : ["repo", "path", "name"]})

I can get the json created till fin but I am unable to append .include() and .sort()

client is broken at a basic level

I can't even open a connection because the code is mangling the url scheme when it tries to join the API path to the provided URL.

requests.exceptions.InvalidSchema: No connection adapters were found for '/api/storage/https:/hostname.jfrog.io/hostname/repo-folder'

UnboundLocalError for deploy_file when calc_md5 or calc_sha1 is turned off

When calc_md5/calc_sha1 is turned off. md5/sha1 in the function becomes unbound names and will cause UnboundLocalError : local variable 'md5' referenced before assignment

A simple fix is to add md5=None, sha1=None to the beginning.

  def deploy_file(self,
                   file_name,
                   calc_md5=True,
                   calc_sha1=True,
                   parameters={}):
       """
       Upload the given file to this path
       """
       if calc_md5:
           md5 = md5sum(file_name)
       if calc_sha1:
           sha1 = sha1sum(file_name)

       target = self

       if self.is_dir():
           target = self / pathlib.Path(file_name).name

       with open(file_name, 'rb') as fobj:
           target.deploy(fobj, md5, sha1, parameters)

Path.glob to find files not working

we are having issues with this code from you examples

from artifactory import ArtifactoryPath
path = ArtifactoryPath(
    "http://repo.jfrog.org/artifactory/distributions/org/")

for p in path.glob("**/*.gz"):
    print(p)

it throws the following error
NotImplementedError: Non-relative patterns are unsupported

Im having exactly the same i=error in my code please let me know whats wrong here

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.