semaphoreci / artifact Goto Github PK

View Code? Open in Web Editor NEW

6.0 6.0 0.0 211 KB

License: Apache License 2.0

Makefile 1.11% Go 98.37% Shell 0.52%

artifact's People

Contributors

Stargazers

Watchers

artifact's Issues

Implement `artifact yank job SINGLE_FILE_PATH`

Typo error in `path_resolver` package

artifact/pkg/files/path_resolver.go

Line 52 in 2f90de9

 return nil, fmt.Errorf("project ID is not set. Please use the SEMAPHORE_JOB_ID environment variable or the --job-id parameter to configure it") 

Current:
"project ID is not set. Please use the SEMAPHORE_JOB_ID environment variable or the --job-id parameter to configure it"

Excepted:
"job ID is not set. Please use the SEMAPHORE_JOB_ID environment variable or the --job-id parameter to configure it"

Project, workflow, job level expire values to config file

The default for all three is "never", give the possibility to set these globals.

Regression in latest artifact push workflow

A regression was introduced today which is breaking all of our builds. Pushing nested folders results in a missing middle folder!

Here is an example with the following dir structure

- workdir
  - test2
    - test_file
    -  nested
      - nested_file

semaphore@semaphore-vm:~/workdir$ mkdir test2
semaphore@semaphore-vm:~/workdir$ touch test2/test_file
semaphore@semaphore-vm:~/workdir$ mkdir test2/nested
semaphore@semaphore-vm:~/workdir$ touch test2/nested/nested_file
semaphore@semaphore-vm:~/workdir$ artifact push workflow --force test2
2022/05/19 23:28:09 successful push for current workflow

This results in a workflow artifact directory of:

- test2
  - test_file
  - nested_file

We have downstream blocks depending on a specific file structure, so the fact the artifact push workflow is now flattening them is quite problematic, breaking builds for about 20 of our projects. Can somebody take a 👀 at the changes released in semaphore internally today for regressions?

Push, pull, yank for directory

Is flag --config being used?

@vendelin8 👋 I'm currently writing documentation page for artifact CLI and I'm not sure if this flag is intended or deprecated.

When running artifact pull job (without arguments) I noticed the following lines in the output log


Global Flags:
      --config string   config file (default is $HOME/.artifact.yaml)

Store expire dates for pushed files to the bucket

Expire data is stored in the same Bucket, where the files are pushed.
Home for expire data is /var/expires-in/. When a file is pushed with an expire date other than never, an object is created with

filename: <expire_timestamp_in_seconds>-<random_postfix>
content: the path in the Bucket to delete after the expire date

Implement `artifact push job SINGLE_FILE_PATH`

Flag: --expire-in 10d or -e 10d

Using `/dev/stdin` as an input no longer works

It seems like in a recent release it's no longer possible to pipe data into artifact push /dev/stdin - the artifact is created as a zero-size file. My suspicion is that this is because of this change c86f55d which tries to stat the file (which has no size) and always sends an empty body as a result.

`sem artifact` would be the only cli-way to download the artifacts, when will it ship?

Afaict, sem artifact would be the only "cli-way" to download artifacts. We'd really like that feature.

Its marked as a TODO, is there an estimation when it would ship?
🙏

Restore of dir with a single file restores file to dir name instead

To reproduce:

mkdir savepath
echo "some file content" > savepath/file.txt
artifact push workflow savepath

# change node
rm -r savepath
artifact pull workflow savepath
ls -al savepath
>> -rw-rw-r-- 1 semaphore semaphore 18 Aug 24 11:59 savepath
cat savepath
>> some file content

This is unexpected, and erases the actual filename.
If there are 2 files in the pushed folder the script behaves as expected.

A workaround is to tar the folder first and save the tar, but this is less than ideal for a typically small number of files.

semaphore@semaphore-vm:~/toolbox$ artifact pull job test-results/junit.json
2021/05/04 09:45:18 successful pull for current job
semaphore@semaphore-vm:~/toolbox$ artifact yank job test-results/junit.json
2021/05/04 09:45:26 successful yank for current job
semaphore@semaphore-vm:~/toolbox$

0.4.5

semaphore@semaphore-vm:~/toolbox$ artifact pull job test-results/junit.json
2021/05/04 09:34:57 successful pull for current job
semaphore@semaphore-vm:~/toolbox$ artifact yank job test-results/junit.json
2021/05/04 09:35:11 http status fail
2021/05/04 09:35:11 Artifact not found. Please check if the artifact you are trying to yank exists.

Running yank with -v flag gives more information about that:

semaphore@semaphore-vm:~/toolbox$ artifact yank job test-results/junit.json -v
2021/05/04 10:09:33 initiating artifact...
2021-05-04T10:09:33.542Z	DEBUG	log/log.go:80	initiating artifact...	{"token": "d493f2ff-8350-46b7-a6ae-57918b9a82ee"}
2021/05/04 10:09:33 artifact initiated
2021-05-04T10:09:33.542Z	DEBUG	log/log.go:80	artifact initiated	{"org URL": "https://semaphore.semaphoreci.com", "gatewayAPIBase": "/api/v1/artifacts", "gatewayAPI": "https://semaphore.semaphoreci.com/api/v1/artifacts"}
2021/05/04 10:09:33 YankPath
2021-05-04T10:09:33.542Z	DEBUG	log/log.go:80	YankPath	{"input file": "test-results/junit.json", "output file": "test-results/junit.json"}
2021/05/04 10:09:33 http status fail
2021-05-04T10:09:33.910Z	WARN	log/log.go:67	http status fail	{"while doing": "Upload", "url": "https://storage.googleapis.com/vfb77u567adcgpqcdeframasmygvcp/artifacts/jobs/5a450a1f-8dd0-4382-be7a-f519edb63726/test-results/junit.json?Expires=1620124173&GoogleAccessId=artifacthub%40semaphore2-prod-artifacts.iam.gserviceaccount.com&Signature=KGOAA2sqyV13DsxkgAttqQiAksEuQqnWgNCm%2FYkWUvrrezpDlSNYUNP8WWELGai%2BPHQFCgkJPCg33LMjE20%2FMWD%2FXHSw26hCWbRlIl0VXnFYWVB8aVEyqa2MJIjFNoHZZfL%2FPhsxbhbNJzmQ0J%2Bf25ix61rY4RAlHmT4jNDFTn80MwMzRpIJkde0DQaG7ephCpvw6ypOz60H5ezrk5tkz8VDvXBJhZCb0OQ58x%2FDH8ekuOkkRQgs%2FkBfE1i2ZrjS7OGGhoLqHwz2PqiblN%2BKPg%2Bz%2FHZvtrX0uNT9OvmKkUCRQnhxyOfaej8jPqjEHnQY5F%2FfDFXTPqeKDK4qIsXtPA%3D%3D", "status code": 403, "status": "Forbidden", "content": "<?xml version='1.0' encoding='UTF-8'?><Error><Code>SignatureDoesNotMatch</Code><Message>The request signature we calculated does not match the signature you provided. Check your Google secret key and signing method.</Message><StringToSign>PUT\n\n\n1620124173\n/vfb77u567adcgpqcdeframasmygvcp/artifacts/jobs/5a450a1f-8dd0-4382-be7a-f519edb63726/test-results/junit.json</StringToSign></Error>"}
github.com/semaphoreci/artifact/pkg/util/log.Logger.Warn
	/home/semaphore/artifact/pkg/util/log/log.go:67
github.com/semaphoreci/artifact/pkg/util/log.Warn
	/home/semaphore/artifact/pkg/util/log/log.go:120
github.com/semaphoreci/artifact/pkg/util/http.formatIfErr
	/home/semaphore/artifact/pkg/util/http/httputil.go:31
github.com/semaphoreci/artifact/pkg/util/http.do
	/home/semaphore/artifact/pkg/util/http/httputil.go:47
github.com/semaphoreci/artifact/pkg/util/http.DeleteURL
	/home/semaphore/artifact/pkg/util/http/httputil.go:77
github.com/semaphoreci/artifact/pkg/gcs.doYankGCS
	/home/semaphore/artifact/pkg/gcs/gcs.go:468
github.com/semaphoreci/artifact/pkg/gcs.YankGCS
	/home/semaphore/artifact/pkg/gcs/gcs.go:458
github.com/semaphoreci/artifact/cmd.runYankForCategory
	/home/semaphore/artifact/cmd/yank.go:30
github.com/semaphoreci/artifact/cmd.glob..func8
	/home/semaphore/artifact/cmd/yank.go:46
github.com/spf13/cobra.(*Command).execute
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/semaphoreci/artifact/cmd.Execute
	/home/semaphore/artifact/cmd/root.go:32
main.main
	/home/semaphore/artifact/main.go:6
runtime.main
	/usr/local/golang/1.12/go/src/runtime/proc.go:200
2021/05/04 10:09:33 Artifact not found. Please check if the artifact you are trying to yank exists.
2021-05-04T10:09:33.910Z	WARN	log/log.go:67	Artifact not found. Please check if the artifact you are trying to yank exists.
github.com/semaphoreci/artifact/pkg/util/log.Logger.Warn
	/home/semaphore/artifact/pkg/util/log/log.go:67
github.com/semaphoreci/artifact/pkg/util/log.Warn
	/home/semaphore/artifact/pkg/util/log/log.go:120
github.com/semaphoreci/artifact/pkg/gcs.YankGCS
	/home/semaphore/artifact/pkg/gcs/gcs.go:460
github.com/semaphoreci/artifact/cmd.runYankForCategory
	/home/semaphore/artifact/cmd/yank.go:30
github.com/semaphoreci/artifact/cmd.glob..func8
	/home/semaphore/artifact/cmd/yank.go:46
github.com/spf13/cobra.(*Command).execute
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:830
github.com/spf13/cobra.(*Command).ExecuteC
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
	/home/semaphore/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/semaphoreci/artifact/cmd.Execute
	/home/semaphore/artifact/cmd/root.go:32
main.main
	/home/semaphore/artifact/main.go:6
runtime.main
	/usr/local/golang/1.12/go/src/runtime/proc.go:200

Flag: --job <job-id> or -j <job-id>

Flag `--destination` or `-d sets` destination directory or file path

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.