glassechidna / stackit Goto Github PK

macOS has Homebrew, Debian has apt, Windows has chocolatey, etc.

Should also include man pages.

Both scenarios:

• When deploying a stack with nested stacks already in S3
• Packaging nested stack resources

We should show events from the top-level stack and all nested stacks. Similar in nature to #20 I imagine.

Test e2e functionality, especially stuff like up-ing a zombie stack.

https://github.com/glassechidna/stackit/blob/master/stackit/stackit.go#L212-L215

For stacks in a DELETE_FAILED state

stackit just hangs without any indication of what's happening:

$ stackit up --stack-name s3-buckets --template infra/cloudformation/s3-buckets.yaml --region us-east-1
stackit version 0.0.39

Newly created bucket - maybe versioning takes a while to take effect?

EDIT: A few seconds later, re-running worked

This depends on the region being explicitly specified by the user:

If a user manually cancels a stack update from the web console, those rollback events will have a different client request token to the one created by stackit. This will cause stackit to not show those events and hang indefinitely

This would also serve as nice documentation of the required permissions to use stackit

aws cloudformation package and aws cloudformation deploy behave differently to stackit package and stackit execute.

At first I thought it would be more useful to create and display a changeset as part of stackit package, making it similar to terraform plan. This arguably is more useful in the typical case, but isn't as useful in the CI case.

Specifically, in a CI setup we might want to archive "artifacts" from a Build stage and use them in a later Deploy stage. Right now we are archiving stackit.packaged.yml. The problem is that this isn't replayable, a user can't re-run just the Deploy stage as they might expect.

We could ameliorate this with the following design:

• stackit package acts identically to aws cloudformation package, albeit with sensible defaults -- it will emit a <template name>.packaged.yml file.
• stackit plan is analogous to terraform plan
• stackit apply is analogous to terraform apply
• stackit up --stack-name <stack> --template <template name>.packaged.yml is a wrapper around plan && apply, i.e. it creates and executes a change set.

Right now you get this if you specify a role name:

creating change set: ValidationError: AWSCloudFormationExecutionRole is not a valid RoleARN
 status code: 400, request id: 1982ae99-565e-11e9-a094-2ffc8eaf83c6

If the user provides a name, we can construct the ARN from that for them. This would be useful for when they are looping over accounts etc

rather than failing out

This code assumes that the processed template will be JSON, which is only true if a) the input template was JSON or b) there are transforms in the input template.

It causes this issue:

invalid character 'A' looking for beginning of value
pretty-printing json of template body
github.com/glassechidna/stackit/cmd.savePreparedOutput
	/github/workspace/cmd/load_save.go:18
github.com/glassechidna/stackit/cmd.packageTemplate
	/github/workspace/cmd/package.go:81
github.com/glassechidna/stackit/cmd.init.2.func1
	/github/workspace/cmd/package.go:144
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:766
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:852
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:800
github.com/glassechidna/stackit/cmd.Execute
	/github/workspace/cmd/root.go:44

Refer to this failure: https://github.com/glassechidna/stackit/runs/122926370

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/what-is-cfnstacksets.html

If you run stackit up twice on a template that contains a serverless function and the code doesn't change in between runs, a failure emerges - when it should be a no-op. Output:

[10:12:16] test-stack-packaged - UPDATE_IN_PROGRESS - User Initiated
[10:12:20] Function - UPDATE_IN_PROGRESS
[10:12:21] Function - UPDATE_COMPLETE
[10:12:23] FunctionVersion7b4af - CREATE_IN_PROGRESS
[10:12:23] FunctionVersion7b4af - CREATE_FAILED - A version for this Lambda function exists ( 1 ). Modify the function to create a new version.
[10:12:24] test-stack-packaged - UPDATE_ROLLBACK_IN_PROGRESS - The following resource(s) failed to create: [FunctionVersion7b4af85c00].
[10:12:40] Function - UPDATE_IN_PROGRESS
[10:12:40] Function - UPDATE_COMPLETE
[10:12:42] test-stack-packaged - UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS
[10:12:44] FunctionVersion7b4af - DELETE_SKIPPED
[10:12:44] test-stack-packaged - UPDATE_ROLLBACK_COMPLETE

This (and avoiding unnecessary uploads) is likely why the AWS CLI uses the MD5 of the zip archive as the basename in the S3 key. We should do that (see #30)

e.g. https://github.com/glassechidna/stackit/blob/master/stackit/stackit.go#L208

stackExists := err == nil doesn't necessarily mean the stack doesn't exist if there's an error. The error could be lack of permission to describe stacks or something else. Check the exact error message.

And other stuff

SIGINFO is just Ctrl+T on macOS/BSD.

It seems that both aws cloudformation package and sam deploy have supported YAML anchors for a while now, so we should also support it.

The CloudFormation service itself doesn't support them, so we need to resolve the anchor references locally.

This would allow stacks to be decorated with additional tags set by Jenkins etc

E.g. if two paths are behind an !If or Fn::If, should we upload both? Or just ignore it? What does the AWS CLI do?

Teamcity has this funky ##teamcity output that it will parse - look into how we could usefully support that

• Find references to local paths
• Upload these files to S3 (only if file has changed)
• Return a template with local paths replaced by S3 locations

I think I'd prefer S3 object versions, but MD5 names is also acceptable for now.

Tests to add:

• Bucket without versioning enabled
• Prefix and no prefix
• Local paths nested inside !If [...]
• Tests for each of the packageable types
• Useful error messages when local artifacts don't exist

panic: creating change set: ValidationError: Cannot specify usePreviousTemplate as true for a changeSetType CREATE

CREATE_FAILED - Requires capabilities : [CAPABILITY_AUTO_EXPAND]