glassechidna / stackit Goto Github PK
View Code? Open in Web Editor NEWCross-platform CloudFormation CLI tool for easy synchronous and idempotent stack updates
License: Apache License 2.0
Cross-platform CloudFormation CLI tool for easy synchronous and idempotent stack updates
License: Apache License 2.0
macOS has Homebrew, Debian has apt, Windows has chocolatey, etc.
Should also include man
pages.
Both scenarios:
We should show events from the top-level stack and all nested stacks. Similar in nature to #20 I imagine.
Test e2e functionality, especially stuff like up-ing a zombie stack.
For stacks in a DELETE_FAILED state
stackit just hangs without any indication of what's happening:
$ stackit up --stack-name s3-buckets --template infra/cloudformation/s3-buckets.yaml --region us-east-1
stackit version 0.0.39
Newly created bucket - maybe versioning takes a while to take effect?
stackit/pkg/stackit/packager/upload.go
Line 67 in 45b26cd
EDIT: A few seconds later, re-running worked
This depends on the region being explicitly specified by the user:
Line 46 in 8e23998
2018/03/21 03:36:37 Throttling: Rate exceeded
status code: 400, request id: 0f591be0-2cb9-11e8-b33c-f705a4379654
script returned exit code 1
If a user manually cancels a stack update from the web console, those rollback events will have a different client request token to the one created by stackit
. This will cause stackit to not show those events and hang indefinitely
Line 29 in 8e06a4b
This would also serve as nice documentation of the required permissions to use stackit
aws cloudformation package
and aws cloudformation deploy
behave differently to stackit package
and stackit execute
.
At first I thought it would be more useful to create and display a changeset as part of stackit package
, making it similar to terraform plan
. This arguably is more useful in the typical case, but isn't as useful in the CI case.
Specifically, in a CI setup we might want to archive "artifacts" from a Build stage and use them in a later Deploy stage. Right now we are archiving stackit.packaged.yml
. The problem is that this isn't replayable, a user can't re-run just the Deploy stage as they might expect.
We could ameliorate this with the following design:
stackit package
acts identically to aws cloudformation package
, albeit with sensible defaults -- it will emit a <template name>.packaged.yml
file.stackit plan
is analogous to terraform plan
stackit apply
is analogous to terraform apply
stackit up --stack-name <stack> --template <template name>.packaged.yml
is a wrapper around plan && apply
, i.e. it creates and executes a change set.Right now you get this if you specify a role name:
creating change set: ValidationError: AWSCloudFormationExecutionRole is not a valid RoleARN
status code: 400, request id: 1982ae99-565e-11e9-a094-2ffc8eaf83c6
If the user provides a name, we can construct the ARN from that for them. This would be useful for when they are looping over accounts etc
rather than failing out
Line 16 in 8e23998
This code assumes that the processed template will be JSON, which is only true if a) the input template was JSON or b) there are transforms in the input template.
It causes this issue:
invalid character 'A' looking for beginning of value
pretty-printing json of template body
github.com/glassechidna/stackit/cmd.savePreparedOutput
/github/workspace/cmd/load_save.go:18
github.com/glassechidna/stackit/cmd.packageTemplate
/github/workspace/cmd/package.go:81
github.com/glassechidna/stackit/cmd.init.2.func1
/github/workspace/cmd/package.go:144
github.com/spf13/cobra.(*Command).execute
/go/pkg/mod/github.com/spf13/[email protected]/command.go:766
github.com/spf13/cobra.(*Command).ExecuteC
/go/pkg/mod/github.com/spf13/[email protected]/command.go:852
github.com/spf13/cobra.(*Command).Execute
/go/pkg/mod/github.com/spf13/[email protected]/command.go:800
github.com/glassechidna/stackit/cmd.Execute
/github/workspace/cmd/root.go:44
Refer to this failure: https://github.com/glassechidna/stackit/runs/122926370
If you run stackit up
twice on a template that contains a serverless function and the code doesn't change in between runs, a failure emerges - when it should be a no-op. Output:
[10:12:16] test-stack-packaged - UPDATE_IN_PROGRESS - User Initiated
[10:12:20] Function - UPDATE_IN_PROGRESS
[10:12:21] Function - UPDATE_COMPLETE
[10:12:23] FunctionVersion7b4af - CREATE_IN_PROGRESS
[10:12:23] FunctionVersion7b4af - CREATE_FAILED - A version for this Lambda function exists ( 1 ). Modify the function to create a new version.
[10:12:24] test-stack-packaged - UPDATE_ROLLBACK_IN_PROGRESS - The following resource(s) failed to create: [FunctionVersion7b4af85c00].
[10:12:40] Function - UPDATE_IN_PROGRESS
[10:12:40] Function - UPDATE_COMPLETE
[10:12:42] test-stack-packaged - UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS
[10:12:44] FunctionVersion7b4af - DELETE_SKIPPED
[10:12:44] test-stack-packaged - UPDATE_ROLLBACK_COMPLETE
This (and avoiding unnecessary uploads) is likely why the AWS CLI uses the MD5 of the zip archive as the basename in the S3 key. We should do that (see #30)
e.g. https://github.com/glassechidna/stackit/blob/master/stackit/stackit.go#L208
stackExists := err == nil
doesn't necessarily mean the stack doesn't exist if there's an error. The error could be lack of permission to describe stacks or something else. Check the exact error message.
And other stuff
SIGINFO is just Ctrl+T on macOS/BSD.
stackit version 0.0.40
panic: packaging template: packaging template: uploading zip to s3: getting bucket for object upload: creating s3 bucket: OperationAborted: A conflicting conditional operation is currently in progress against this resource. Please try again.
status code: 409, request id: VP25XTEYVVX82RKS, host id: EmiktUN/9Zk+J6Q8km/2pL2kBl1UaznY8jmM28WU4Z0scl9F1v/iXenOq2KWOMtDBbpUrP2F6yw=
goroutine 1 [running]:
github.com/glassechidna/stackit/cmd.init.6.func1(0xc000103180, 0xc0000712c0, 0x0, 0x4)
/home/runner/work/stackit/stackit/cmd/up.go:166 +0xd2
github.com/spf13/cobra.(*Command).execute(0xc000103180, 0xc000071200, 0x4, 0x4, 0xc000103180, 0xc000071200)
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:830 +0x2aa
github.com/spf13/cobra.(*Command).ExecuteC(0x145ca00, 0xc00009a010, 0xd26831, 0x13)
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:914 +0x2fb
github.com/spf13/cobra.(*Command).Execute(...)
/home/runner/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/glassechidna/stackit/cmd.Execute()
/home/runner/work/stackit/stackit/cmd/root.go:44 +0xb8
main.main()
/home/runner/work/stackit/stackit/main.go:20 +0x20
It seems that both aws cloudformation package
and sam deploy
have supported YAML anchors for a while now, so we should also support it.
The CloudFormation service itself doesn't support them, so we need to resolve the anchor references locally.
This would allow stacks to be decorated with additional tags set by Jenkins etc
E.g. if two paths are behind an !If
or Fn::If
, should we upload both? Or just ignore it? What does the AWS CLI do?
./stackit up --stack-name lkp-lambda-test --template cfn.yml --previous-param-value PstoreCAKeyBytesName
2017/06/30 00:26:56 ValidationError: Either Template URL or Template Body must be specified.
status code: 400, request id: d2a8645f-5d2a-11e7-a986-41b3c23d55b6
panic: ValidationError: Either Template URL or Template Body must be specified.
status code: 400, request id: d2a8645f-5d2a-11e7-a986-41b3c23d55b6
Teamcity has this funky ##teamcity
output that it will parse - look into how we could usefully support that
stackit/pkg/stackit/package.go
Lines 51 to 54 in 614efda
I think I'd prefer S3 object versions, but MD5 names is also acceptable for now.
Tests to add:
!If [...]
panic: creating change set: ValidationError: Cannot specify usePreviousTemplate as true for a changeSetType CREATE
CREATE_FAILED - Requires capabilities : [CAPABILITY_AUTO_EXPAND]
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.