travis-ci / artifacts Goto Github PK

I want to upload artifact(s) to my s3 bucket with public permission, however it returns error:

$ export ARTIFACTS_PERMISSIONS=public
$ export ARTIFACTS_CACHE_CONTROL=public
$ ./artifacts u
INFO[0000] uploading with settings                       bucket=xxx cache_control=public permissions=public
INFO[0000] uploading: /home/tuna/temp-linux-amd64 (size: 5.3KB)  download_url=https://s3.amazonaws.com/xxx/temp-linux-amd64
INFO[0003] uploading: /home/tuna/temp-linux-amd64 (size: 5.3KB)  download_url=https://s3.amazonaws.com/xxx/temp-linux-amd64
INFO[0006] uploading: /home/tuna/temp-linux-amd64 (size: 5.3KB)  download_url=https://s3.amazonaws.com/xxx/temp-linux-amd64
ERRO[0006] failed to upload: /home/tuna/temp-linux-amd64  err=400 Bad Request

My assets weren't uploading, this is the end of the debug output from Travis:

..
assets/style.css: no such file or directory
  retry: 2
ERROR: failed to upload: /home/travis/build/test/68998280.html
assets/20e28182428903b2619580fd00061422.txt
assets/2ab8a28e5882c593e79561e618a257a0.png
assets/49686bcc48c65a6fc1217fed3873dfdb.png
assets/d708b0fdf7ff39ed464c7d4542459263.txt
assets/style.css
  err: open /home/travis/build/test/68998280.html
assets/20e28182428903b2619580fd00061422.txt
assets/2ab8a28e5882c593e79561e618a257a0.png
assets/49686bcc48c65a6fc1217fed3873dfdb.png
assets/d708b0fdf7ff39ed464c7d4542459263.txt
assets/style.css: no such file or directory
Done uploading artifacts

When I ran locally with the same credentials, the result I got was

ERRO[0007] failed to upload: /tests/assets/style.css  err=Access Denied
ERRO[0007] failed to upload: /tests/123.html  err=Access Denied

So, I'm guessing I've got a permissions issue which is entirely possible.

But it would have been helpful to see the AccessDenied error not no such file or directory

Hi all,

I'm trying to figure out how to change the default prefix that artifacts get uploaded to, currently it looks like {github repo}/{build number}/{build number}.{version}/{file name}.

I've read http://docs.travis-ci.com/user/uploading-artifacts/ but it's not mentioned how to control that.

Cheers!

In job https://travis-ci.org/coletivoEITA/noosfero/jobs/115528708 there is a download URL of one artifact: https://s3.amazonaws.com/noosfero-travis-artifacts/coletivoEITA/noosfero/86/86.5/tmp/artifact-travis-86-create-a-forum.png

But the link only work if the region is suffixed after s3, for instance this is the correct download url:
https://s3-us-west-1.amazonaws.com/noosfero-travis-artifacts/coletivoEITA/noosfero/86/86.5/tmp/artifact-travis-86-create-a-forum.png

Related to #57

When trying to upload artifacts to us-east-1 bucket it fails with following error:

DEBUG: retrying
  artifact: /Users/travis/build/spark/spark-dev/dist/darwin/particle-dev-mac-0.0.25-dev.zip
  err: Put https:/dev.artifacts.particle.io/spark/spark-dev/150/150.1/dist/darwin/particle-dev-mac-0.0.25-dev.zip: dial tcp: i/o timeout
  retry: 1

Next retry is more descriptive:

DEBUG: retrying
  artifact: /Users/travis/build/spark/spark-dev/dist/darwin/particle-dev-mac-0.0.25-dev.zip
  err: Put https:/dev.artifacts.particle.io/spark/spark-dev/150/150.1/dist/darwin/particle-dev-mac-0.0.25-dev.zip: dial tcp: lookup s3.amazonaws.com on 10.182.0.19:53: no answer from server
  retry: 2

Whole log is available here.

I read the article Uploading Artifacts on Travis CI and my understanding is, this artifact upload tool will upload files to a given S3 bucket provided by Travis CI user.

When hosting a Travis CI enterprise for enterprise users, the majority of use cases under this scenario is Travis CI users expect to keep their artifacts in a hosted service (e.g., a bucket managed by Travis CI service) instead of providing their own bucket. So that they can upload/view/download from Travis CI directly, so that they don't need to manage their own object service nor the credentials in .travis.yml.

When a user wants to upload build artifacts into their own S3 bucket, One option is following S3 Deployment to make it.

Is it (the above hosted artifacts service under Travis CI) an expected scenario of this feature for enterprise users?

Dpl can also upload artifacts to S3 & Github Releases.
Maybe those implementations should move here instead?

This is copied over from my SO question: http://stackoverflow.com/questions/35507396/travis-ci-uploading-artifacts-to-s3-results-in-the-bucket-you-are-attempting-t

I have a Travis CI build that is configured to upload the build artifacts to S3. I've followed the Travis artifacts documentation but when the build completes I get the following error (and the S3 container is empty).

ERROR: failed to upload: /home/travis/build/jonburney/KingsgateMediaPlayer-Android/
app/build/outputs/apk/app-release-unsigned.apk

err: The bucket you are attempting to access must be addressed using the specified
endpoint. Please send all future requests to this endpoint.

I have tried to specify the "endpoint" option in the configuration but it was ignored. It appears to be attempting to upload the file to

https://s3.amazonaws.com/kmp-build-output/jonburney/KingsgateMediaPlayer-Android/30/30.1/app/build/outputs/apk/app-release-unsigned.apk.

Here is a copy of the relevant section from my .travis.yml file

addons:
    artifacts: true
    s3_region: "us-west-2"
    artifacts:
        paths:
          - $(git ls-files -o app/build/outputs | tr "\n" ":")

Hi Team

I am trying to raise the current limit of 1 GB for artifacts max size. However, I am getting a 500 error while trying to set it.

I try to set it using the following request:
API ENDPOINT URL: https://api.travis-ci.org/repos/settings/env_vars?repository_id=XXX

BODY: {"env_var": {"name": "ARTIFACTS_MAX_SIZE", "value": 5048576000, "public": false}}

But the response I get is this: Response [500]

Please suggest if I am missing something.

Thank you
Mohd Farid

I'm tried to upload iOS build and when step Uload atrifacts coming i'm get an error

Uploading Artifacts artifacts.setup fatal error: runtime: bsdthread_register error runtime stack: runtime.throw(0x13c0857, 0x21) /home/travis/.gimme/versions/go1.10.2.linux.amd64/src/runtime/panic.go:616 +0x81 fp=0x7ff7bfefda98 sp=0x7ff7bfefda78 pc=0x1028d41 runtime.goenvs() /home/travis/.gimme/versions/go1.10.2.linux.amd64/src/runtime/os_darwin.go:129 +0x83 fp=0x7ff7bfefdac8 sp=0x7ff7bfefda98 pc=0x10268c3 runtime.schedinit() /home/travis/.gimme/versions/go1.10.2.linux.amd64/src/runtime/proc.go:501 +0xd6 fp=0x7ff7bfefdb30 sp=0x7ff7bfefdac8 pc=0x102b636 runtime.rt0_go(0x7ff7bfefdb68, 0x2, 0x7ff7bfefdb68, 0x0, 0x1000000, 0x2, 0x7ff7bfefe050, 0x7ff7bfefe05a, 0x0, 0x7ff7bfefe05d, ...) /home/travis/.gimme/versions/go1.10.2.linux.amd64/src/runtime/asm_amd64.s:252 +0x1f4 fp=0x7ff7bfefdb38 sp=0x7ff7bfefdb30 pc=0x1050fc4 $ export ARTIFACTS_PATHS="$(ls ./build/*.ipa | tr \"\\n\" \":\"):$(ls ./build/*.zip | tr \"\\n\" \":\")" ls: ./build/*.ipa: No such file or directory $ artifacts upload fatal error: runtime: bsdthread_register error

Are you open to merge an artifacts download function that will get files from the s3 bucket?

Context: in GitLab CI we support multiple stages and we want to use this to pass artifacts between runners/workers. /cc @ayufan

I'm working on an open-source project and have several artifacts configured in .travis.yml:

addons:
  artifacts:
    working_dir: app/dist
    target_paths: client/${TRAVIS_COMMIT:0:8}/${TRAVIS_OS_NAME}
    permissions: public-read
    paths:
    - mailsync
    - mailsync.dSYM.zip
    - Merani.dmg
    - Merani.zip
    - $(find . -type f -name *.deb | tr "\n" ":")
    - $(find . -type f -name *.rpm | tr "\n" ":")

In my case, mailsync is an executable, and mailsync.dSYM.zip contains the debug files for the executable. When the Travis build completes, it finds the files and claims to upload them, but chooses an incorrect path for the mailsync file without an extension:

Uploading Artifacts (BETA)
artifacts.setup
artifacts version v0.7.8-26-gd62580b revision=d62580ba6bfffc7245d8ac6a1a384ef60b9302e7
$ export ARTIFACTS_PATHS="mailsync:mailsync.dSYM.zip:Merani.dmg:Merani.zip:$(find . -type f -name *.deb | tr \"\\n\" \":\"):$(find . -type f -name *.rpm | tr \"\\n\" \":\")"
$ artifacts upload
INFO: uploading with settings
  bucket: [secure]
  cache_control: private
  permissions: public-read
INFO: uploading: app/dist/mailsync.dSYM.zip (size: 5.8MB)
  download_url: https://s3.amazonaws.com/[secure]/client/7191c2e8/osx/mailsync.dSYM.zip
INFO: uploading: app/dist/mailsync (size: 5.1MB)
  download_url: https://s3.amazonaws.com/[secure]/client/7191c2e8/osx
....

The second download URL should be https://s3.amazonaws.com/[secure]/client/7191c2e8/osx/mailsync not, https://s3.amazonaws.com/[secure]/client/7191c2e8/osx.

I'm not sure how to fix this in the meantime—anyone have ideas?

Here's a full build log illustrating the problem: https://travis-ci.org/Foundry376/Merani/jobs/264519025.

This issue I think pertains to code in https://github.com/travis-ci/travis-build but issues are disabled so I am coming here.

As far as I can tell from source code I should be able to specify a configuration that looks something like this:

addons:
  artifacts:
    ...
    branch: master

and have artifacts only uploaded on the master branch. However I have been unable to get this to work, artifacts are uploaded on all branches (not on pull requests though). The docs say nothing WRT to filtering on branches.

When a S3 bucket is created in region "eu-central-1" / Frankfurt it is impossible to use artifacts to upload to it.

We use build artifacts to store the binary for an autopilot firmware (http://px4.io) to an AWS bucket to allow flight-testers to grab the binary and fly it.
The current setup is workable, however, the artifacts are buried deep-down in the console log. Would it be possible to add a listing of uploaded artifacts with a download link to the top of the page? For an example, please have a look at:
https://travis-ci.org/PX4/Firmware/builds/44751661

In order to be able to create .deb/.rmp/*.dmg etc. packages a preprocessing stage should be allowed to occur.
As a first step maybe we should allow to run fpm-recipes in order to generate packages.

A few weeks after setting up artifacts with export ARTIFACTS_PATHS="$(git ls-files -o | tr "\n" ":")" doing the work of uploading the assets created during the test run, I decided to tweak my .gitignore.

This had the impact of meaning that no files were returned by git ls-files -o. Instead of no files being uploaded, all files in my test directory were uploaded.

This may be intentional, but I think it should, at least, be documented.

Travis are now recommending removing the sudo tag.

"If you currently specify sudo: false in your .travis.yml, we recommend removing that configuration"

It appears that the URL generated for each artifact is incorrect. The current format is
https://s3.amazonaws.com/%s
(as can be seen here where the supplied parameter is the bucket name) which, for example, generates https://s3.amazonaws.com/my-bucket/my-org/my-project/build/job/thisisthefile.png

However, when visiting a generated URL, I don't get the expected file, but rather this:

<Error>
   <Code>PermanentRedirect</Code>
   <Message>The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.</Message>
   <Bucket>my-bucket</Bucket>
   <Endpoint>my-bucket.s3.amazonaws.com</Endpoint>
   <RequestId>blahblahblah</RequestId>
   <HostId>blahblahblah</HostId>
</Error>

If I then manually modify the generated URL to:

https://my-bucket.s3.amazonaws.com/my-org/my-project/build/job/thisisthefile.png

then it works as expected.

So I'm guessing that the line of code linked to above, should actually be:

downloadHost = fmt.Sprintf("https://%s.s3.amazonaws.com/", b.Name)

...or is there something I'm missing?

Right now when artifacts exits with error ERRO[0006] failed to upload, the process indicates a success (exit code 0), which should not be the case imho.

$ AWS_ACCESS_KEY=fake AWS_SECRET_KEY=fake artifacts upload --bucket fake somefile; echo "Exit code:$?"
INFO[0000] uploading with settings                       bucket=fake cache_control=private permissions=private
ERRO[0006] failed to upload: /path/to/somefile  err=open /path/to/somefile: no such file or directory
Exit code:0

As a comparison, this command exits with code 1:

$ AWS_ACCESS_KEY=fake AWS_SECRET_KEY=fake artifacts upload somefile; echo "Exit code:$?"
FATA[0000] no bucket name given
Exit code:1

I understand that ERRO[0006] concerns a single file upload out of potentially many, and that the error code may be a mix of success and errors, but maybe it would be a good feature to be able to control that with a cli flag (like --fail-on-error or something like that).

Right now, exit code 0 makes it difficult to break the Travis build when the artifact command errors.

I think artifacts upload ignores env vars set in repository settings.

Build #67 fails:

Setting environment variables from repository settings
$ export ARTIFACTS_SECRET=[secure]
$ export ARTIFACTS_KEY=[secure]
...
$ artifacts upload
DEBUG: setting log level to debug
FATAL: no access key given
The command "artifacts upload" failed and exited with 1 during .

I can workaround this by setting them in .travis.yml. Build #69 succeeds:

Setting environment variables from repository settings
$ export ARTIFACTS_SECRET=[secure]
$ export ARTIFACTS_KEY=[secure]
...
Setting environment variables from .travis.yml
$ export ARTIFACTS_KEY=[secure]
$ export ARTIFACTS_SECRET=[secure]
...
$ artifacts upload
DEBUG: setting log level to debug
DEBUG: adding path
  path: &{/Users/travis/build/toolbear/panda hello }
DEBUG: adding path
  path: &{/Users/travis/build/toolbear/panda hola }
DEBUG: starting upload

Long term, I don't really want these credentials bound to the commit.

Having many artifacts to upload (say screenshot for each test) results in a huge artifacts gem output which isn't collapsable, which creates too much noise. It would be good to have some sort of shorter option, maybe only print out errors.

For example I can setup my own S3 API compatible object storage server using https://github.com/minio/minio and would be great to be able to upload artifacts there. As I understand it's currently not possible to configure a custom S3 endpoint, means S3 endpoint is hardcoded somewhere in the code to s3.amazonaws.com.

It would be great to have region and endpoint options. As I discovered with dpl, they are important for uploading to a S3 website

I can't get 'em from the cache and none of the instructions on http://docs.travis-ci.com/user/uploading-artifacts/ nor anything I've found in github issues work.

Latest artifacts binary crashes when run on 10.13.1:

VERSION:
   v0.7.8-26-gd62580b revision=d62580ba6bfffc7245d8ac6a1a384ef60b9302e7

$ artifacts upload
fatal error: unexpected signal during runtime execution
[signal 0xb code=0x1 addr=0xb01dfacedebac1e pc=0xc9d70]

goroutine 1 [running]:
runtime.throw(0x4cbf80, 0x2a)
	/home/travis/.gimme/versions/go1.5.1.linux.amd64/src/runtime/panic.go:527 +0x90 fp=0xc820103088 sp=0xc820103070
runtime.sigpanic()
	/home/travis/.gimme/versions/go1.5.1.linux.amd64/src/runtime/sigpanic_unix.go:12 +0x5a fp=0xc8201030d8 sp=0xc820103088
sync.(*Pool).Get(0x64eab0, 0x0, 0x0)
	/home/travis/.gimme/versions/go1.5.1.linux.amd64/src/sync/pool.go:101 +0x40 fp=0xc820103128 sp=0xc8201030d8
fmt.newPrinter(0xc8200f8bf0)
	/home/travis/.gimme/versions/go1.5.1.linux.amd64/src/fmt/print.go:133 +0x27 fp=0xc820103168 sp=0xc820103128
fmt.Fprintf(0x8d37f8, 0xc8200b0e70, 0x49f4a0, 0x18, 0xc8201033f0, 0x4, 0x4, 0x0, 0x0, 0x0)
	/home/travis/.gimme/versions/go1.5.1.linux.amd64/src/fmt/print.go:187 +0x30 fp=0xc8201031b0 sp=0xc820103168
github.com/Sirupsen/logrus.(*TextFormatter).Format(0xc8200f84c0, 0xc82009b940, 0x0, 0x0, 0x0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/Sirupsen/logrus/text_formatter.go:51 +0xd62 fp=0xc820103498 sp=0xc8201031b0
github.com/Sirupsen/logrus.(*Entry).Reader(0xc82009b940, 0xc800000001, 0x0, 0x0)
	/home/travis/gopath/src/github.com/Sirupsen/logrus/entry.go:43 +0x67 fp=0xc820103530 sp=0xc820103498
github.com/Sirupsen/logrus.(*Entry).log(0xc82009b940, 0x1, 0xc8200afcc0, 0x14, 0x0, 0x0)
	/home/travis/gopath/src/github.com/Sirupsen/logrus/entry.go:84 +0x227 fp=0xc820103688 sp=0xc820103530
github.com/Sirupsen/logrus.(*Entry).Fatal(0xc82009b940, 0xc820103788, 0x1, 0x1)
	/home/travis/gopath/src/github.com/Sirupsen/logrus/entry.go:130 +0x78 fp=0xc8201036d0 sp=0xc820103688
github.com/Sirupsen/logrus.(*Logger).Fatal(0xc82009b840, 0xc820103788, 0x1, 0x1)
	/home/travis/gopath/src/github.com/Sirupsen/logrus/logger.go:124 +0xe0 fp=0xc820103718 sp=0xc8201036d0
main.runUpload(0xc8200f60b0)
	/home/travis/gopath/src/github.com/travis-ci/artifacts/artifacts.go:68 +0x144 fp=0xc8201037b8 sp=0xc820103718
github.com/codegangsta/cli.Command.Run(0x451110, 0x6, 0x451028, 0x1, 0x49cf00, 0x16, 0x5082c0, 0x1db, 0x0, 0x0, ...)
	/home/travis/gopath/src/github.com/codegangsta/cli/command.go:101 +0xdbc fp=0xc820103bd0 sp=0xc8201037b8
github.com/codegangsta/cli.(*App).Run(0xc8200f4000, 0xc82000a1c0, 0x2, 0x2, 0x0, 0x0)
	/home/travis/gopath/src/github.com/codegangsta/cli/app.go:125 +0xa9f fp=0xc820103f08 sp=0xc820103bd0
main.main()
	/home/travis/gopath/src/github.com/travis-ci/artifacts/artifacts.go:22 +0x49 fp=0xc820103f40 sp=0xc820103f08
runtime.main()
	/home/travis/.gimme/versions/go1.5.1.linux.amd64/src/runtime/proc.go:111 +0x2b0 fp=0xc820103f90 sp=0xc820103f40
runtime.goexit()
	/home/travis/.gimme/versions/go1.5.1.linux.amd64/src/runtime/asm_amd64.s:1696 +0x1 fp=0xc820103f98 sp=0xc820103f90

Hello!
I am trying to upload artifacts but I am getting this error:

ERROR: failed to upload: /home/travis/build/RnDome/test_travis_artifacts/build/test_project-0.1.0-Linux.zip

  err: Put /testtravisartifacts2/RnDome/test_travis_artifacts/19/19.1/build/test_project-0.1.0-Linux.zip: unsupported protocol scheme ""

Done uploading artifacts

See this build

List of failed regions:

List of succeeded regions:

• US East (N. Virginia) - requires export ARTIFACTS_REGION=""

Allow the user to upload to a directory of a S3 bucket.

When I upload a single file, it creates a directory named : (colon), which is a bit weird.

❯❯❯ ARTIFACTS_REGION=us-west-2 artifacts upload --key=$redacted --secret=$redacted --bucket=linuxbrew hi
INFO[0000] uploading with settings                       bucket=linuxbrew cache_control=private permissions=private
INFO[0000] uploading: /gsc/btl/linuxbrew/hi (size: 14B)  download_url=https://s3.amazonaws.com/linuxbrew/:/hi

It seems that the option --target-paths is missing from artifacts.go.