drone-plugins / drone-s3-sync Goto Github PK
View Code? Open in Web Editor NEWDrone plugin for syncing artifacts with an Amazon S3 Bucket
Home Page: http://plugins.drone.io/drone-plugins/drone-s3-sync
License: Apache License 2.0
Drone plugin for syncing artifacts with an Amazon S3 Bucket
Home Page: http://plugins.drone.io/drone-plugins/drone-s3-sync
License: Apache License 2.0
i have aws s3 compatible endpoint, provide creadentials and set endpoint in step to my host:
https://s3.selcdn.ru
but drone after try to execute step says:
RequestError: send request failed
--
2 | caused by: Get https://www.s3.selcdn.ru/?prefix=: x509: certificate is valid for *.selcdn.ru, selcdn.ru, not www.s3.selcdn.ru
i'm recheck and i don't add www prefix to host. why it added?
Using Drone image: drone/drone:1.0.0-rc.3
When configuring acl values like so, it does not work:
settings:
acl:
"*": public-read
Bug when configuring it as a JSON string, it seems to work:
settings:
acl: '{ "*": "public-read" }'
This might be more of an issue with drone and it's yaml to env parsing. Let me know if you want me to open it up in a different repo instead.
As per the documentation, we should be able to pass secrets: [ aws_access_key_id, aws_secret_access_key ]
. The secrets are set.
This results in the following error:
Must set access_key, secret_key, and bucket
I'm using the plugin to upload a static Nuxt3 build to S3 in website mode.
I cannot serve the site because the .mjs files that landed on S3 do not have mime type attached.
In the browser:
Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "". Strict MIME type checking is enforced for module scripts per HTML spec.
I am working on a PR for this right now, this is just to keep tracking of effort.
Feedback is welcome on the changes on my cloudfront branch.
Maybe just ignoring the fact that the credentials are empty is enough
(Removing https://github.com/drone-plugins/drone-s3-sync/blob/master/plugin.go#L63)
But basically any plugin communicating with aws should have the ability to work with a role
would like to strip folder prefix for upload to s3 similar to the s3 plugin strip_prefix
Sync to S3: Done without problems (all files was changed)
Invalidation CloudFront: Error 403 - authorization problem.
AWS user (Key+Secret) has all neeeded permissions.
ERROR: failed to invalidateCloudFront to /*: IncompleteSignature: 'AKIAT7AGGEXXXXXXXXXX/20191010/us-east-1/cloudfront/aws4_request' not a valid key=value pair (missing equal-sign) in Authorization header: 'AWS4-HMAC-SHA256 Credential=********/20191010/us-east-1/cloudfront/aws4_request, SignedHeaders=content-length;host;x-amz-date, Signature=c35c46c9d2f6c5aaa0b7357ec0406c823e7cbc1181624e4f3817************'. status code: 403, request id: 66a36506-eb61-11e9-973d-************
After removing white space from credencials Invalidations started to working.
It was hard to find where is the problem becouse credensials are stored in as secrets, but when i opened database and quered for secrets related to repository i find a bug that i have done during copy + paste.
I think that different behavior when authorizing to S3 and to Cloudfront can be fixed using simple Trim().
Is it possible to set a "content-encryption" like "gzip" in s3_sync?
If there be an error or exception throw out to interrupt drone stage when anything unexpected happen during data transfer?
eg. some files transfer successful but left failed.
The docs here mention the exclude and include parameter but these parameters are not implemented.
Either implement them or remove them from the docs.
The Rsync Plugin has them implemented for example
Hi,
We're experiencing an error with Drone artifact uploads to S3 via S3 sync.
The drone build will randomly fail on certain files with the following error:
ERROR: failed to upload /foo/bar/..../baz.svg to foo/img/login.svg: NoSuchKey: The specified key does not exist.
status code: 404, request id: 123456
[info] build failed (exit code 1)
There are no discernible patterns in the filename, filetype or filesize that we causes the error, nor is there a discernible pattern in the number of files before the upload fails.
Please let me know if I can provide any extra info to help contextualize this issue, and thanks in advance.
The documentation shows bucket
parameter provided in two different formats (actually twice for each example):
bucket: my-bucket-name
...
bucket: my-bucket.s3-website-us-east-1.amazonaws.com
Which format is the correct one?
The link to the documentation: http://plugins.drone.io/drone-plugins/drone-s3-sync/
I get
NotImplemented: A header you provided implies functionality that is not implemented - ERROR 501
whenever I try to upload something. Apparently this is a known issue affecting SDK versions 1.6.2 and earlier.
See: aws/aws-sdk-go#1379
Default "*" value for paths is better than nothing, but it'd be nice to have a way to manage it.
plugin is dependent it seems on using the s3 fqdn rather than just referencing the bucket. Using it with a private s3 bucket behind cloudfront behind an auth edge lambda.
Currently cache headers can be set globally or on a per file type basis. There is however use cases that require configuration per path/filename, where e.g.
cache_control:
"*.js": "max-age=31536000"
"entry.js": "no-cache"
"assets/**/*.*": "max-age=86400"
should result in all *.js
files being cached for a year, everything underneath assets/
for a day and entry.js
not at all.
I wanted to document this issue even though I haven't had time to dig into the root cause. I was attempting to publish using the native
branch and run into the following error. http://beta.drone.io/drone/blog/1
Uploading "/drone/src/github.com/drone/blog/public/post/drone-named-rookie-of-the-year/index.html" with Content-Type "text/html; charset=utf-8" and permissions "public-read"
ERROR: failed to upload /drone/src/github.com/drone/blog/public/post/drone-discussion-site/index.html to post/drone-discussion-site/index.html: :
status code: 404, request id:
[info] build failed (exit code 1)
I switched to the master
branch version and the sync executed without error. I'm guessing there is a minor step we're missing in the upload process, but not sure. I'll post back a root cause once I've had the chance to research.
Looks like a bug. The x-amz-meta-cache-control
metadata key is set instead of Cache-Control
.
.drone.yml
:
...
s3_sync:
acl: public-read
region: $$AWS_REGION
bucket: $$AWS_BUCKET
access_key: $$AWS_ACCESS_KEY_ID
secret_key: $$AWS_SECRET_ACCESS_KEY
source: build/
target: /
delete: true
metadata:
"*.js":
Cache-Control: "public, max-age: 10000000"
"*.css":
Cache-Control: "public, max-age: 10000000"
"*.jpg":
Cache-Control: "public, max-age: 10000000"
"*.gif":
Cache-Control: "public, max-age: 10000000"
"*.png":
Cache-Control: "public, max-age: 10000000"
"*.html":
Cache-Control: "public, max-age: 600"
There was an exclude functionality; but it does not appear to exist and is not documented.
It would be good if we could exclude paths like .git
It happens only sometimes that it deletes files that it shouldn't.
For example just today in the logs we had:
Uploading "/drone/src/github.com/matteosuppo/reference/public/en/index.html" with Content-Type "text/html; charset=utf-8" and permissions "public-read"
[...]
Removing remote file "reference/en/index.html"
where the config was
publish:
s3_sync:
acl: public-read
region: "us-east-1"
bucket: $$S3_BUCKET
access_key: $$S3_ACCESS
secret_key: $$S3_SECRET
source: public
delete: true
target: reference
when:
branch: master
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.