I have an app that is deployed to all AWS regions. This is my serverless.yml (simplified)

provider:
  name: aws
  stage: ${opt:stage, "dev"} 
  runtime: nodejs12.x
  stackName: ${self:serviceName}-${self:provider.stage}
  memorySize: 128
  mainRegion: ${opt:main-region, env:AWS_MAIN_REGION}
  accountId: ${opt:aws-account-id, env:AWS_ACCOUNT_ID}
  deploymentBucket:
    name: my-company-deployment-${aws:region}
    maxPreviousDeploymentArtifacts: 5
    blockPublicAccess: true
    serverSideEncryption: AES256
plugins:
  - serverless-deployment-bucket
custom:
  deploymentBucket:
    versioning: false
    accelerate: true
    blockPublicAccess: true 

Previously I didn't include custom.deploymentBucket.blockPublicAccess. After I added it & deployed my service to all regions I saw that almost all deployment buckets (1 per region) were updated to Bucket and objects not public. However, the following regions did not update the deployment buckets to Access: Bucket and object not public:

af-south-1
ap-east-1
ap-northeast-3
ap-southeast-3
eu-north-1
eu-south-1
me-south-1

I've re-run the deployment multiple times and repeated the same procedure in a separate staging and prod account, and the result was the same -> these regions did not update. I had to do it manually via AWS cli.

I've double-checked my config and app code, but I couldn't fin anything particular about the problematic regions.

I'm using the following:

  tags: &tags
    ManagedBy: Serverless Framework
    Service: ${self:service}
    Stage: ${sls:stage}
    Environment: ${sls:stage}
  stackTags:
    <<: *tags

I can not reuse the anchor with the bucket due to the current format.

Having the property

frameworkVersion: '3'
configValidationMode: error
...

in your serverless.yaml while also having

provider:
  deploymentBucket:
    name: ${self:service}-deployment-bucket-${opt:stage}
    serverSideEncryption: aws:kms
    kmsKeyID: ${param:KEY_ALIAS_ARN}

will cause the deployment to fail due to kmsKeyID not being a recognized property. By switching to configValidationMode: warn, you can successfully deploy and have the bucket use the KMS key provided. If I want to keep the configValidationMode property as error, then I cannot use this to set up KMS encryption.

I am using serverless 3.22.0

Serverless seems to have a built-in configuration options for the deploymentBucket.

1. Will the options above work without the serverless-deployment-bucket plugin installed?
2. If so, what is the role of this plugin, it seems to support fewer configuration options than the built-in serverless one (one notable exception may be accelerate).
3. What is the relation between options configured under provider.deploymentBucket and custom.deploymentBucket ones? What if I set versioning: false in provider.deploymentBucket and versioning:true in custom.deploymentBucket?
4. Will accelerate config work if I put it under provider.deploymentBucket?

Thank you upfront for answering, and thank you for your contributions!

I want to specity s3 bucket for deploy. ( firewall is only allow register url pah)
If in serverless.yml without deployBucket part, no issue.
But added deployBucket part,

provider:
deploymentBucket:
name: my-bucket-dev

Timeout Error happen.
Below error message.

Serverless: Packaging service...

Error ---------------------------------------------------

Error: connect ETIMEDOUT 54.222.50.102:443
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16)

 For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

Get Support --------------------------------------------

I'd also like to implement a CD workflow to automate the manual process I currently perform to cut a new release.

Version Info

Serverless: 2.x.x
Plugin: 1.5.2

Reproduction steps:

Run commands:

serverless deploy

Output:

Serverless: Creating deployment bucket 'some-bucket-blahblah'...
Serverless: Applied SSE (AES256) to deployment bucket
Serverless: Enabled acceleration on deployment bucket
Serverless: Applied deployment bucket policy
Serverless: Updated deployment bucket tags
Serverless: Updated deployment bucket public access block
Serverless: Packaging service...
Serverless: Using deployment bucket 'some-bucket-blahblah'
Serverless: Applied deployment bucket policy
Serverless: Updated deployment bucket public access block
Serverless: Ensuring that deployment bucket exists
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Validating template...
Serverless: Creating Stack...
Serverless: Checking Stack create progress...

Explanation

The hook before:aws:common:validate:validate is unnecessary run twice by a serverless deploy (no --package option), first for packaging and then for deploying, which results in extraneous logging of Applied deployment bucket policy and Updated deployment bucket public access block. This bug was introduced in the fix for #20.

Proposed Fix

A few options to investigate:

• Use a different hook for serverless deploy --package vs serverless deploy
• Add smarter diffing logic to determine changes for bucket policy and public access block
• Investigate using the hook before:deploy:deploy

Can you add an example how to correctly add option for policy, e.g.

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>HEAD</AllowedMethod>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>DELETE</AllowedMethod>
    <MaxAgeSeconds>3600</MaxAgeSeconds>
    <ExposeHeader>x-amz-server-side-encryption</ExposeHeader>
    <ExposeHeader>x-amz-request-id</ExposeHeader>
    <ExposeHeader>x-amz-id-2</ExposeHeader>
    <ExposeHeader>ETag</ExposeHeader>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

Hi, thanks for your work putting this project together. Much appreciated

It appears serverless-deployment-bucket doesn't pick up on the provider.deploymentBucket.blockPublicAccess key in serverless.yml. I realize it's possible to set an exact bucket policy, but it seems likely that a common use case is to want to block all public access, and when not using this plugin that can be done with just:

provider:
  deploymentBucket:
    blockPublicAccess: true

For reference, here's how this was implemented in serverless core

Hi, I'm getting this error

-------- Deployment Bucket Error --------
The us-west-2 location constraint is incompatible for the region specific endpoint this request was sent to.

This is my serverless.yml file

service: xxxxx

provider:
  name: aws
  runtime: nodejs12.x
  region: us-west-2
  stage: ${opt:stage, 'dev'}
  versionFunctions: false
  timeout: 15

  deploymentBucket:
    name: test-deployments
    serverSideEncryption: AES256

Now that GitHub Actions have a free usage tier, I'd like to migrate the existing CI workflow to it.

The plugin creates the bucket outside the cloudformation stack.
When removing the project from AWS using serverless remove --stage dev, the bucket is left behind because only the stack and its contents are deleted.
Is it possible to create the bucket inside the cloudformation stack?

Hi, nice plugin. I'm looking to add S3 bucket access logging.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-s3-bucket-loggingconfig.html

Do you know if it's possible to modify the Properties of the deployment bucket? Specifically LoggingConfiguration?

I want to extend your plugin to allow me to specify a destination bucket name as a property.

custom:
  deploymentBucket:
    versioning: true
    accelerate: true
    loggingDestinationBucketName: !Ref MyOwnBucketResource
    blockPublicAccess: true
    tags:
      - Key: Environment
        Value: production

Background
Hey this issue is in relation to #36

Details
Specifically there is a bug in processing tags for a new bucket
The following function which is used to make a boolean check to either add or skip tagging

async hasChangedBucketTags(name, tags) {
    try {
      const response = await this.provider.request('S3', 'getBucketTagging', { Bucket: name })
      return response && JSON.stringify(response.TagSet) !== JSON.stringify(tags)
    } catch (e) {
      return false
    }
  }

The above function will throw an error and return false everytime for a new bucket, and there is no process for adding tags separately. Outside of manually adding a dummy tag to the S3 bucket after creation and then rerunning the serverless deploy step.

Expected Outcome

• when the S3 bucket is created for the first time we should add tags to the creation if any exist

• Bonus Points: error processing for hasChangedBucketTags could also be updated to check to see if the error is specific for no-tags, and then attempt to add tags if any existed in the serverless.yml file.

serverless-deployment-bucket currently only honors the setting for Block Public Access configured by custom/deploymentBucket/blockPublicAccess, and ignore the same value used by the core serverless config at provider/deploymentBucket/blockPublicAccess.

The underlying issue here is that serverless-deployment-bucket requires a confusing configuration with respect to recent versions of serverless. I have not verified this theory by looking at the changelogs for serverless and serverless-deployment-bucket, but I believe that the blockPublicAccess configuration option was added to serverless-deployment-bucket prior to it being added to serverless, and each requires it to be in a different place in the configuration.

Without the plugin, serverless allows the configuration of the name of a pre-existing bucket or other parameters like Block Public Access to be set.

Use this pre-created bucket and do not alter it (e.g., if blockPublicAccess were set, it would be ignored)

provider:
  deploymentBucket:
    name: some-bucket

or, auto-create the bucket and set Block Public Access:

provider:
  deploymentBucket:
    blockPublicAccess: true

Confusingly, when using the serverless-deployment-bucket plugin, the provider/deploymentBucket/name field is used for the bucket name (creating it if it doesn't exist), but the provider/deploymentBucket/blockPublicAccess configuration is ignored and only the custom/deploymentBucket/blockPublicAccess is honored.

A correct configuration for using serverless-deployment-bucket will look like this:

plugins:
  - serverless-deployment-bucket
...
provider:
  deploymentBucket:
    name: slsdeploy-example-app-todos-dev
...
custom:
  deploymentBucket: # used by serverless-deployment-bucket
    blockPublicAccess: true

Currently there is no way to pass config to the KMS key configuration block

ApplyServerSideEncryptionByDefault: {
    SSEAlgorithm: sseAlgorithm,
    KMSMasterKeyID: kmsMasterKeyId
}

The current call only passes 2 of 3 params with no way of getting that 3rd param sent through:
Caller:
https://github.com/MikeSouza/serverless-deployment-bucket/blob/0c9c632170aeb2284b2e5c3f4669f5e81442353a/lib/index.js#L239

Function:
https://github.com/MikeSouza/serverless-deployment-bucket/blob/0c9c632170aeb2284b2e5c3f4669f5e81442353a/lib/index.js#L96

Currently using S3 Transfer Acceleration in our organization for our pipelines.
We deploy with:
serverless deploy --verbose --conceal --aws-s3-accelerate --stage dev

Therefore, we can't use this plugin:

We'd appreciate a custom option to create the s3 with Transfer Acceleration enabled.

https://github.com/mikesouza/serverless-deployment-bucket/blob/258b90c7bc2c94ce8b9a4e9adaf0461d0eee0a89/lib/index.js#LL279C77-L279C77

Aws activates the key (AES256) by default, so I can never change the key to 'aws:kms'.

Would it possible to support for tags?

Version: 1.5.1
Issue:

For existing deployment bucket, after configuring "tags" property with new tags, it is not possible to add new tags. This seem to only work when bucket is either being created or if the bucket has at least one tag.

Steps to reproduce:
Step 1:
Configure vanilla serverless.yml with serverless-deployment-bucket. Bucket configuration in custom section:

  deploymentBucket:
    enabled: true

Then deploy (no tags).

Step 2:
Change deploymentBucket to following:

  deploymentBucket:
    enabled: true
    tags:
      - Key: Environment
        Value: production

Then deploy - observe that bucket tags are empty.

Preliminary issue notes:
Best I can surmise is that issue itself comes from DeploymentBucketPlugin.hasChangedBucketTags in index.json where
const response = await this.provider.request('S3', 'getBucketTagging', { Bucket: name })
throws an exception:
{"code":"AWS_S3_GET_BUCKET_TAGGING_NO_SUCH_TAG_SET","providerError":{"message":"The TagSet does not exist","code":"NoSuchTagSet","region":null,"time":"2021-06-28T17:46:11.101Z","requestId":".....","extendedRequestId":".......","statusCode":404,"retryable":false,"retryDelay":94.99932161936133},"providerErrorCodeExtension":"NO_SUCH_TAG_SET"}
And has DeploymentBucketPlugin.hasChangedBucketTags returns false. This in turn causes applyDeploymentBucket to not apply new tags

Hi 👋 , Serverless Framework core team here!

We're working on the v3 beta version right now. The v3 will include a redesigned CLI output:

You can try the v3 beta with: npm i serverless@pre-3

To help plugins integrate with that new design (and build a great user experience), we've added new APIs for plugins:

• Documentation of the new API
• Video introduction (Youtube)

Any feedback is appreciated! If you want to update the plugin to use these new APIs, feel free to ping me if you have any questions!

If you don't have time to update the plugin, let me know, we can send a pull request!

The update to 1.5.0 is causing failures, rolling back to 1.4.3 has fixed the situation.

Configuration is simple -

provider:
deploymentBucket:
name:

`
Error ---------------------------------------------------

Error: schema is invalid: data.properties['custom'].properties['policy'].type should be equal to one of the allowed values, data.properties['custom'].properties['policy'].type should be array, data.properties['custom'].properties['policy'].type should match some schema in anyOf, data.properties['custom'].properties['tags'].type should be equal to one of the allowed values, data.properties['custom'].properties['tags'].type should be array, data.properties['custom'].properties['tags'].type should match some schema in anyOf

  at Ajv.validateSchema (/usr/local/lib/node_modules/serverless/node_modules/ajv/lib/ajv.js:178:16)

  at Ajv._addSchema (/usr/local/lib/node_modules/serverless/node_modules/ajv/lib/ajv.js:307:10)

  at Ajv.compile (/usr/local/lib/node_modules/serverless/node_modules/ajv/lib/ajv.js:113:24)

  at ConfigSchemaHandler.validateConfig (/usr/local/lib/node_modules/serverless/lib/classes/ConfigSchemaHandler/index.js:106:26)

  at Service.validate (/usr/local/lib/node_modules/serverless/lib/classes/Service.js:225:41)

  at Serverless.run (/usr/local/lib/node_modules/serverless/lib/Serverless.js:288:47)

  at async /usr/local/lib/node_modules/serverless/scripts/serverless.js:613:9



 For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.`

The default behavior of the serverless framework, is to create a deployment bucket inside the stack itself.

Upon destroying the stack, the default behavior is to empty the deployment bucket and then to delete it.

This plugin is creating a bucket outside the stack, which I can live with. But it should also empty and delete the bucket after destroying the stack.

We have our s3 buckets created and managed centrally. Can we deploy to an existing bucket without the bucket properties changing? For example, the tags were removed from the existing bucket since I didn't define any tags in my serverless file.

Could you provide a better example of applying a policy to the deployment bucket?

is it possible to refer to the deployment bucket name in the policy?

can i do a policy on the bucket without renaming the bucket? (if i add this to an existing stack/serverless deployment, what happens? )

Plugin works great in terms of creating new buckets but bucket policy don't seem to be getting applied when passing in as json .

Here is the policy I'm trying to apply

deploymentBucket:
policy:
{
"Id": "DeploymentBucketPolicy",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DenyIncorrectEncryptionHeader",
"Principal": "",
"Action": [
"s3:PutObject"
],
"Effect": "Deny",
"Resource": "arn:aws:s3:::testdeploymentbucket/",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "aws:kms"
}
}
}
]
}

I have a bucket that has already been deloyed

provider:
  name: aws
  runtime: nodejs12.x
  region: ${opt:region}
  stage: ${opt:stage}
  env: ${opt:env}
  logRetentionInDays: 14
  apiGateway:
    disableDefaultEndpoint: true
  deploymentBucket:
    name: my-service-${self:provider.env}-${self:provider.region}-serverless-bucket
    serverSideEncryption: aws:kms
    kmsKeyID: ${${self:provider.env}-kms-s3-internal-key.ExportCMKAlias}

previously I had AES256 for serverSideEncryption and the serverless deploy runs just fine but after running I go check the bucket and it still has AES256.

If I change the bucket name so that it creates another one it works.

Hi Mike,

Thanks for contributing.
Currently I can't see an example of how to pass a policy to the deployment bucket.
I would assume would work smth like?

deploymentBucket:
name: ${self:custom.bucketName}
custom: 
   bucketName: ${someString}-suffix

//this file could be yml or json where S3 bucket would have defined it's properties and policys
//
Resources:
------
Properties:
    BucketName: ${self:custom.bucketName}
-------
Policy etc

Hello,

Great plugin, thank you.

We use it to create our deployment bucket, and also add server side encryption.

I've tried to add tags and also set block public access, but it doesn't work for me.

We are using v1.4.3 , with this config:

deploymentBucket:
        name: 'com.ourproject.${self:provider.region}.${self:provider.stage}.deployments'
        blockPublicAccess: true
        tags:
            - Key: Key
              Value: value
        serverSideEncryption: AES256

When i run sls deploy --stage mystage, the bucket gets created, but with no tags and blockPublicAccess has no effect.

Is there a step i'm missing?

Thanks,

Dave

Sometimes if the deployment bucket does not exist, the plugin will create the bucket, but Serverless will think the bucket still doesn't exist and give this error:

Serverless: Creating deployment bucket 'test-serverless-associate-waf-us-east-1-development'...
Serverless: Applied SSE (AES256) to deployment bucket
Serverless: Packaging service...
Serverless: Excluding development dependencies...
 
  Serverless Error ---------------------------------------
 
  The serverless deployment bucket "test-serverless-associate-waf-us-east-1-development" does not exist. Create it manually if you want to reuse the CloudFormation stack "test-serverless-associate-waf-development", or delete the stack if it is no longer required.

This doesn't always happen and isn't reproducible 100% of the time. I believe this is because the execution of the plugin happens asynchronously and Serverless does not wait for the plugin to finish creating the bucket before it performs its check for whether the deployment bucket exists.

Reproduction steps:

Run commands:
serverless package --package sls-package-output
serverless deploy --package sls-package-output

Output:

Serverless: Invoke aws:common:validate
  Serverless Error ---------------------------------------
 
  Could not locate deployment bucket. Error: The specified bucket does not exist

Explanation

This plugin only creates the bucket on the package step. Logic was added in PR #13 to skip bucket creation on the package step, but the hook used by this plugin is before:package:setupProviderConfiguration.

This creates a problem in the unique situation when deploying with a pre-created package, since that hook is never run.

Proposed Fix

Hook into before:aws:common:validate:validate instead. This is where the bucket's existence is checked.

In Serverless v2 , I cannot deploy from an sls printed yaml.
To recreate - this is an example yaml

service: sls-test
frameworkVersion: 2.30.3
enableLocalInstallationFallback: true
plugins:
  - serverless-deployment-bucket
provider:
  name: aws
  runtime: nodejs14.x
  stage: test
  region: 'us-east-1'
  versionFunctions: false
  deploymentBucket:
    name: 'bucket'

Running sls deploy will try to create the bucket.
Running

> SLS_DEPRECATION_DISABLE='*' SLS_WARNING_DISABLE='*' sls print | grep -v "Serverless:" > printed.yaml
> sls deploy -c printed.yaml

will result in Could not locate deployment bucket. without attempting to create the deployment bucket.

When running sls package, it triggers Creating deployment bucket which actually creates AWS resources. The creation of the bucket should only happen on deploy

It seems with the latest versions of serverless & serverless-deployment-bucket, the provider.deploymentBucket object is ignored. I am getting

 Serverless Error ---------------------------------------

  Could not locate deployment bucket. Error: The specified bucket does not exist

If I change provider.deploymentBucket to provider.deploymentBucketObject it creates the bucket, but also the normal serverless one and doesn't use the one I specified

I'm sure it's me, but this was working a week ago!

Couple of points -

• I would like to be able to control the path and filename of the assets that go into S3. currently it is something like:

s3://{deploymentBucket.name}/{service}/{service}/{stage}/{random-13-digit-number}-{iso-timestamp}/{service}.zip

for traceability I would prefer something like:

s3://{deploymentBucket.name}/{service}/{stage}/{git-commit-sha}.zip

service: serverless

plugins:
  - serverless-python-requirements

custom:
  app_acronym: wibble-foo
  stage: ${opt:branch}
  stack_name: ${self:custom.stage}-${self:custom.app_acronym}
  region: ${opt:region, self:provider.region}
  deletion_policy: Delete
  # Default to using app_acronym as bucket name
  s3_bucket: ${opt:s3_bucket, self:custom.app_acronym}
  # default to using ${stage} as key base path, keeps stages from namespace collisions
  s3_key_base: ${opt:s3_key_base, self:custom.stage}
  s3_bucket_arn: arn:aws:s3:::${self:custom.s3_bucket}
  s3_role_resource: ${self:custom.s3_bucket_arn}/${self:custom.s3_key_base}/*
  # Put this here rather than in code (presigned URL TTL)
  url_default_ttl: 60
  deploymentBucket:
    versioning: true


# The `provider` block defines where your service will be deployed
provider:
  name: aws
  stackName: ${self:custom.stack_name}
  stage: ${self:custom.stage}
  runtime: python3.8
  region: eu-west-1
  environment:
    S3_BUCKET: ${self:custom.s3_bucket}
    S3_KEY_BASE: ${self:custom.s3_key_base}
    URL_DEFAULT_TTL: ${self:custom.url_default_ttl}
  deploymentBucket:
    name: z7of-deployment-bucket


# The `functions` block defines what code to deploy
functions:
  helloWorld:
    name: ${self:custom.stack_name}-hello-world
    handler: handler.run
    # The `events` block defines how to trigger the handler.helloWorld code
    events:
      # Invoke Lambda function every minute
      - schedule: cron(* * * * ? *)
    iamRoleStatementsName: $${self:custom.stack_name}-role-lambda
    iamRoleStatements:
      - Effect: Allow
        Action:
          - s3:*
        Resource: ${self:custom.s3_role_resource}