Giter Club home page Giter Club logo

artifact's Introduction

Artifact

Table of contents

Use-cases

  1. Archiving artifacts - storing final deliverables
  2. Promoting artifacts though pipeline and workflow
  3. Debugging - Store and inspect logs, screenshots and other debug data through UI or CLI

Init

  • Logging is disabled for testing. If some tests are failing and you can't find the reason why, you can add something like
func init() {
  if strings.HasSuffix(os.Args[0], ".test") { // testing
    Init(true)
  }
}

to logging file source.

  • Before committing anything, symlink pre-commit script with ln -s ../../script/pre-commit .git/hooks/

Concepts

Big picture

Artifacts can be stored and accessed on three different layers. This concept makes it easy to use artifacts for different purposes with very simple commands.

Artifact stores:

  • Project - one per project
  • Workflow - nasted under current workflow
  • Job - nasted under current job

You can add verbose logging by the --verbose or -v flag.

Project level

Use-case - Storing final deliverables

It's an artifact store that is per project and it's possible to simply upload/download files and directories from this level from every job from any workflow and pipeline. Here is an example how you can interect with project level store from any job in your pipeline or from your local development environemnt.

From any jobs running on Semaphore:

artifact push project myapp-v3.tar
artifact pull project myapp-v3.tar

From your development environment: TODO: this is not done yet

sem artifact push project --project payment-api myapp-v3.tar
sem artifact pull project -p payment-api myapp-v3.tar

You can also view and download your artifacts from project page in the UI.

Workflow level

Use-case - Promoting build artifacts accross pipelines and block levels. e.g. promoting from Build and test pipeline into Production deployment pipeline.

New store is created for each new workflow. Here are examples for interacting with this store from any pipeline and any job within workflow.

From any jobs running on Semaphore:

artifact push workflow myapp-v3.tar
artifact pull workflow myapp-v3.tar

From your development environment: TODO: this is not done yet

sem artifact push workflow --workflow <WORKFLOW_ID> myapp-v3.tar
sem artifact pull workflow -w <WORKFLOW_ID> myapp-v3.tar

You can also view and download artifacts from workflow page in the UI.

Job level

Use-case - Debugging jobs with easy access to artifacts that job created. e.g. Storing logs, screenshots, core dumps and inspecting them them on the job page.

New store is created for each new job. Here are examples for interacting with this store from job.

From job running on Semaphore:

artifact push job logs/build.log
artifact pull job logs/build.log

From your development environment: TODO: this is not done yet

sem artifact push job --job <JOB_ID> logs/build.log
sem artifact pull job -p <JOB_ID> logs/build.log

You can also view and download artifacts on the job page in the UI.

Configs

$HOME/.artifact.yaml or similar, for more, look at Viper.

Artifact paths expire

ProjectArtifactsExpire

Project level artifacts default expire time in the same format as Alternative forms and flags #3.

WorkflowArtifactsExpire

Workflow level artifacts default expire time in the same format as Alternative forms and flags #3.

JobArtifactsExpire

Job level artifacts default expire time in the same format as Alternative forms and flags #3.

CLI

push

artifact push job x.zip

Description

Uploads file or path into /artifacts/jobs/<SEMAPHORE_JOB_ID>/x.zip.

Example 1: Uploading nested file.

artifact push job logs/webserver/access.log pushes file into /artifacts/jobs/<SEMAPHORE_JOB_ID>/log/webserver/access.log

Example 2: Uploading directory

artifact push job myapp/v1/logs/webserver pushes directory with all sub directories and files into /artifacts/jobs/<SEMAPHORE_JOB_ID>/webserver

artifact push job /var/semaphore/webserver/logs/testxyz pushes directory with all sub directories and files into /artifacts/jobs/<SEMAPHORE_JOB_ID>/testxyz

Alternative forms and flags
  1. --destination or -d sets destination directory or file path

artifact push job x.zip -d y.zip pushes file into /artifacts/jobs/<SEMAPHORE_JOB_ID>/y.zip.

Example for directory: artifact push job logs/webserver --destination debuglogs pushes all sub-dirs and files into /artifacts/jobs/<SEMAPHORE_JOB_ID>/debuglogs.

Example for deeply nested directory as destination: artifact push job logs/webserver --destination path/to/debuglogs pushes all sub-dirs and files into /artifacts/jobs/<SEMAPHORE_JOB_ID>/path/to/debuglogs.

  1. --job <job-id> or -j <job-id>

By default command is looking for SEMAPHORE_JOB_ID env var. If it's not available it fails. If flag --job is specified it takes precedence over SEMAPHORE_JOB_ID.

  1. --expire-in 10d or -e 10d

Expires - deletes the files or directories after amount of time specified.

Supported options are:

  • never or Never means it never expires
  • Nd for N days
  • Nw for N weeks
  • Nm for N months
  • Ny for N years

If expires flag is not set artifact never expires.

  1. --force or -f

artifact push job x.zip if x.zip exists in the bucket this command should fail. To overwrite file or directory user would need to specify "force" flag.

Output

TODO

Requirements
  • SEMAPHORE_JOB_ID (not required if --job flag is specified)
  • Linux, macOS: ~/.artifact/credentials
  • Windows: dir "%UserProfile%\.artifact\credentials"

Putting artifacts into artifact store on different levels

Other supported levels include workflow and project level. These are variations of the command depending on the level:

artifact push workflow x.zip

File is stored into /artifacts/workflows/<SEMAPHORE_WORKFLOW_ID>/x.zip

artifact push project x.zip

File is stored into /artifacts/projects/<SEMAPHORE_PROJECT_ID>/x.zip

pull

artifact pull job x.zip

Description

Artifact stored at /artifacts/jobs/<SEMAPHORE_JOB_ID>/x.zip will be downloaded in current directory as x.zip.

Example with directory: artifact pull job logs, if logs is directory logs it will be created locally in current directory and whole content of logs from bucket will be downloaded into logs directory locally.

Alternative forms and flags
  1. --destination or -d sets destination directory or file path

artifact pull job x.zip -d z.zip pushes file into z.zip.

Example for directory: artifact pull job logs --destination debuglogs pulls all sub-dirs and files into debuglogs locally.

Example for deeply nested directory as destination: artifact pull job logs --destination path/to/debuglogs pulls all sub-dirs and files into path/to/debuglogs in current local directory.

  1. --job <job-id> or -j <job-id>

By default command is looking for SEMAPHORE_JOB_ID env var. If it's not available it fails. If flag --job is specified it takes precedence over SEMAPHORE_JOB_ID.

Requirements
  • SEMAPHORE_JOB_ID (not required if --job flag is specified)
  • Linux, macOS: ~/.artifact/credentials
  • Windows: dir "%UserProfile%\.artifact\credentials"

Putting artifacts into artifact store on different levels

Other supported levels include workflow and project level. These are variations of the command depending on the level:

artifact pull workflow x.zip

File is stored into /artifacts/workflows/<SEMAPHORE_WORKFLOW_ID>/x.zip would be restored at current directory as x.zip.

artifact pull projects x.zip

File is stored into /artifacts/projects/<SEMAPHORE_PROJECT_ID>/x.zip would be restored at current directory as x.zip.

yank

artifact yank

Description

Deletes artifact.

artifact yank job x.zip deletes /artifacts/jobs/<SEMAPHORE_JOB_ID>/x.zip

Example for directory: artifact yank job logs deletes /artifacts/jobs/<SEMAPHORE_JOB_ID>/logs directory and all recursively all the content that is in the logs directory in the bucket.

artifact yank workflow x.zip deletes /artifacts/workflows/<SEMAPHORE_WORKFLOW_ID>/x.zip

artifact yank project x.zip deletes /artifacts/projects/<SEMAPHORE_PROJECT_ID>/x.zip

list

TODO: this is not done yet

artifact list

Description

artifact list job lists root of the job directory /artifacts/jobs/<SEMAPHORE_JOB_ID>/

artifact list workflow lists root of the job directory /artifacts/workflows/<SEMAPHORE_WORKFLOW_ID>/

artifact list project lists root of the job directory /artifacts/projects/<SEMAPHORE_PROJECT_ID>/

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.