karlderkaefer / cdk-notifier Goto Github PK

Hi,

running the command cdk diff --profile my_profile MyStack returns the differences in the resources, but at the same time, if there are some changes in the IAM Staments, it's showing in the following way:

What would be the best way to include these changes as part of the comment that cdk-notifier does in the PRs?

Sometimes the changed types are known before and it's not necessary to be made aware of this changes. Therefore it would be helpful to suppress these types. Maybe if the overview parameter looks like this:

--show-overview supress=AWS::ECS::TaskDefinition

cdk diff for stacks

Number of stacks with differences: 1

Stack fargate
Resources
[~] AWS::ECS::TaskDefinition DetailFargateService/TaskDef detailFargateServiceTaskDef795131A3 replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -81,7 +81,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
-       [-]   "Fn::Sub": "123456789012.dkr.ecr.eu-central-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-123456789012-eu-central-1:88f53e8e790ee348fe371bd2dd7365d2cc15be096da0c12d4b0d8bf47aff35d3"
+       [+]   "Fn::Sub": "123456789012.dkr.ecr.eu-central-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-123456789012-eu-central-1:64137e051d225c2e197f36cf1156f21c0ec449c2902fa5c8d685e0fbbe822e2d"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
[~] AWS::ECS::TaskDefinition ListFargateService/TaskDef listFargateServiceTaskDef795531A3 replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -81,7 +81,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
-       [-]   "Fn::Sub": "123456789012.dkr.ecr.eu-central-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-123456789012-eu-central-1:88f53e8e790ee348fe371bgt2dd7365d2cc15be096da0c12d4b0d8bf47aff35d3"
+       [+]   "Fn::Sub": "123456789012.dkr.ecr.eu-central-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-123456789012-eu-central-1:64137e051d225c2e197f36cf11456f21c0ec449c2902fa5c8d685e0fbbe822e2d"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
Stack lambda
Resources
[~] AWS::Lambda::Function listHandler/Lambda/lambda listHandlerLambdalambdaC875E395 
 ├─ [~] Code
 │   └─ [~] .S3Key:
-│       ├─ [-] 6a8fb4fcc5f635e40d135b1038a814ab0aca7be1e0d85eabb319af0d323a699b.zip
+│       └─ [+] 57a04aad6ab772d1d155746c5b7f3fad7ec005480af335a673aadc88b1005919.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
-        ├─ [-] asset.6a8fb4fcc5f635e40d135b1038a814ab0aca7be1e0d85eabb319af0d323a699b
+        └─ [+] asset.57a04aad6ab772d1d155746c5b7f3fad7ec005480af335a673aadc88b1005919


✨  Number of stacks with differences: 2

the latest version does not work on my ubuntu

cdk-notifier --version
cdk-notifier: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by cdk-notifier)
cdk-notifier: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by cdk-notifier)

lsb_release -a
Description:    Ubuntu 20.04.6 LTS

I have interest in adding Gitlab support. Would you be amenable to this? Starting the conversation here per your Medium post.

Currently, the diff is shown collapsed or not. Can we add an overview section above to display the number of differences and the required replacement?

Something like that:

cdk diff for stack

Number of stacks with differences: 2
⚠️ Number of resources that require replacement: 2

There were no differences
Stack ddb-stack
Resources
[~] AWS::DynamoDB::Table ddb-table ddbtable7F3F6F3F replace
 └─ [~] TableName (requires replacement)
-    ├─ [-] ddb-table
+    └─ [+] ddb-table2
[~] AWS::DynamoDB::Table ddb-second-table ddbsecondtableAF4C67DA replace
 └─ [~] TableName (requires replacement)
-    ├─ [-] ddb-second-table
+    └─ [+] ddb-second-table2


✨  Number of stacks with differences: 2

In markdown, the <details> tag can be used to fold and unfold content. That would make the comment much cleaner as CDK diff's outputs tend to be very verbose.

Adding the CI Job Link to header of comment will allow the user to see full logs in case it's truncated

when having multiple stacks deploy with cdk deploy --all indivuals diff would not be detected

Stack core-network
There were no differences
Stack corenetwork735961878498apsoutheast21AE73C6D
There were no differences
Stack corenetwork735961878498eucentral1FC47981F
There were no differences
Stack corenetwork735961878498useast1560CA47A
There were no differences
Stack corenetwork857753963368apsoutheast2F744EDCD
There were no differences
Stack corenetwork857753963368eucentral126B4BFD4
There were no differences
Stack corenetwork857753963368useast13A7BF40D
There were no differences

Deletion of comments for stacks without changes is not enabled by default. The parameter delete is string but should be boolean

Hi,

I am not sure if I am making a mistake but I cannot seem to get CDK-Notifier to work on bitbucket.

I wrote the following bitbucket-pipeline.yaml file:

I keep getting the following error for the last line of the pipeline:
time="2023-07-28T00:22:31Z" level=warning msg="could not parse response to *provider.BitbucketComments"
time="2023-07-28T00:22:31Z" level=fatal msg="BitBucket API Error: 401 Unauthorized "

pipelines:
pull-requests:
'**':
- step:
name: Install and use CDK Notifier
image: node:16
script:
- echo "BITBUCKET_REPO_OWNER is $BITBUCKET_REPO_OWNER"
- echo "BITBUCKET_REPO_SLUG is $BITBUCKET_REPO_SLUG"
- echo "BITBUCKET_TOKEN is $BITBUCKET_TOKEN"
- echo "BITBUCKET_PR_ID is $BITBUCKET_PR_ID"
- echo "BITBUCKET_USERNAME is $BITBUCKET_USERNAME"

        # Ensure npm is updated and typescript installed
        - npm install -g npm
        - npm install -g typescript
        # Install AWS CDK globally
        - npm install -g aws-cdk
        # Install project dependencies
        - npm install
        # Compile TypeScript
        - tsc

        - apt-get update && apt-get install -y curl gzip jq
        - curl -L "https://github.com/karlderkaefer/cdk-notifier/releases/latest/download/cdk-notifier_$(uname)_amd64.gz" -o cdk-notifier.gz
        - gunzip cdk-notifier.gz && chmod +x cdk-notifier
        - mv cdk-notifier /usr/local/bin/cdk-notifier

        - cdk diff --progress=events | tee cdk.log
        - ls -al
        - cat cdk.log
        
        - cdk-notifier --owner $BITBUCKET_REPO_OWNER --repo $BITBUCKET_REPO_SLUG --token $BITBUCKET_TOKEN --log-file ./cdk.log --tag-id my-stack --pull-request-id 20 --vcs bitbucket --ci bitbucket

I echoed out all my variables and they were correctly set.

I installed all the required packages and installed CDK-Notifier.
I ran the cdk diff --progress=events | tee cdk.log

I ran ls -al and could see that the log file was at ./cdk.log and have data written to it.

I added this line at the end to test if I could comment on a pull request with my token using the following command in my pipeline and the pipeline successfully uploaded a comment.

#Post the comment to Bitbucket
- |
url="https://api.bitbucket.org/2.0/repositories/$BITBUCKET_REPO_OWNER/$BITBUCKET_REPO_SLUG/pullrequests/$BITBUCKET_PR_ID/comments"
test_message="This is a test message"
data='{"content": {"raw": "'"$test_message"'"}}'
curl -X POST -H "Authorization: Bearer $BITBUCKET_TOKEN" -H "Content-Type: application/json" -d "$data" "$url"

I am not sure what I am doing wrong or if it's an issue. If this is not the right place to post this I am sorry.

It would be nice if the overview section also list the types of changes. Maybe if the overview parameter looks like this:

--show-overview extended

cdk diff for stacks

Number of stacks with differences: 2
⚠️ Number of resources that require replacement: 2

AWS::ECS::TaskDefinition 2 (requires replacement)
AWS::Lambda::Function 1

Stack fargate
Resources
[~] AWS::ECS::TaskDefinition DetailFargateService/TaskDef detailFargateServiceTaskDef795131A3 replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -81,7 +81,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
-       [-]   "Fn::Sub": "123456789012.dkr.ecr.eu-central-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-123456789012-eu-central-1:88f53e8e790ee348fe371bd2dd7365d2cc15be096da0c12d4b0d8bf47aff35d3"
+       [+]   "Fn::Sub": "123456789012.dkr.ecr.eu-central-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-123456789012-eu-central-1:64137e051d225c2e197f36cf1156f21c0ec449c2902fa5c8d685e0fbbe822e2d"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
[~] AWS::ECS::TaskDefinition ListFargateService/TaskDef listFargateServiceTaskDef795531A3 replace
 └─ [~] ContainerDefinitions (requires replacement)
     └─ @@ -81,7 +81,7 @@
        [ ] ],
        [ ] "Essential": true,
        [ ] "Image": {
-       [-]   "Fn::Sub": "123456789012.dkr.ecr.eu-central-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-123456789012-eu-central-1:88f53e8e790ee348fe371bgt2dd7365d2cc15be096da0c12d4b0d8bf47aff35d3"
+       [+]   "Fn::Sub": "123456789012.dkr.ecr.eu-central-1.${AWS::URLSuffix}/cdk-hnb659fds-container-assets-123456789012-eu-central-1:64137e051d225c2e197f36cf11456f21c0ec449c2902fa5c8d685e0fbbe822e2d"
        [ ] },
        [ ] "LogConfiguration": {
        [ ]   "LogDriver": "awslogs",
Stack lambda
Resources
[~] AWS::Lambda::Function listHandler/Lambda/lambda listHandlerLambdalambdaC875E395 
 ├─ [~] Code
 │   └─ [~] .S3Key:
-│       ├─ [-] 6a8fb4fcc5f635e40d135b1038a814ab0aca7be1e0d85eabb319af0d323a699b.zip
+│       └─ [+] 57a04aad6ab772d1d155746c5b7f3fad7ec005480af335a673aadc88b1005919.zip
 └─ [~] Metadata
     └─ [~] .aws:asset:path:
-        ├─ [-] asset.6a8fb4fcc5f635e40d135b1038a814ab0aca7be1e0d85eabb319af0d323a699b
+        └─ [+] asset.57a04aad6ab772d1d155746c5b7f3fad7ec005480af335a673aadc88b1005919


✨  Number of stacks with differences: 2

Can this tool also support Bitbucket?

CircleCI is enabled for fork, but in a required we require a secret. Allowing access to secrets should be disallowed. Instead we gonna remove the context from steps required for PR

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Awaiting Schedule

These updates are awaiting their schedule. Click on a checkbox to get an update now.

• chore(deps): update dependency semantic-release to v23.0.8
• fix(deps): update golang dependencies (github.com/xanzy/go-gitlab, golang.org/x/oauth2)

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update actions/setup-go action to v5
• fix(deps): update module github.com/google/go-github/v53 to v61

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

All my pipelines started to fail with this error: panic: runtime error: invalid memory address or nil pointer dereference

These are the steps in the pipeline:

      - name: cdk diff
        run: cdk diff --progress=events &> >(tee cdk.log)
        working-directory: cdk

      - name: Save diff in PR
        run: >
          cdk-notifier
          --owner             ${{ github.repository_owner }}
          --repo              ${{ github.event.repository.name }}
          --token             ${{ github.token }}
          --log-file          ./cdk.log
          --pull-request-id   ${{ github.event.pull_request.number  }}
          --tag-id "all stacks - ${{github.base_ref}} "
        working-directory: cdk

From any CI/CD tool, it is possible to push to a Pull Request from the Github CLI.
For example from inside GitHub actions, anything we would like to post that has been output as a file from another task could be output like this:

    - name: Comment on Pull Request
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      run: |
        gh pr comment --body-file ${{ steps.diff.outputs.comment_file }}

My suggestion is to add a NO CI option that does not attempt to post the output, but simple supplies the output file with the markdown in it.

Other systems can then easily just take that output and post it themselves by handing it over without having to worry about specific CI system details.

This would make the tool more versatile and allow for support of other systems without implementing a ton of API details.

On the first run of a cdk deployment, the IAM statement changes table in the log is displayed. Not sure if this is a gitlab/github markdown specific issue or not, but documenting this to discuss.

My thought is to suppress the table if it is in the log to reduce noise in the comment posted. Leave the "note" at the end (NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299) with maybe a link to the job log for folks to look closer there?

Would be great to support Workspace Access tokens (App users) as well as Personal Access Tokens, discussed in #99.

func (proxy BitbucketProxy) RoundTrip(req *http.Request) (res *http.Response, e error) {
	msg := fmt.Sprintf("Sending request to %s%s", req.URL.Host, req.URL.Path)
	logrus.Debug(strings.ReplaceAll(msg, "\n", ""))
	if proxy.username != "" {
		req.SetBasicAuth(proxy.username, proxy.password)
	} else {
		req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", proxy.password))
	}
	req.Header.Add("Accept", "application/json")
	return proxy.Proxied.RoundTrip(req)
}

I did some quick tests and the code above works fine with a Workspace Access Token at least. Basically cdk-notifier can be called without the --user flag for Workspace Access Tokens.

cdk-notifier --ci bitbucket -p 1 -r some_repo --vcs bitbucket -l ./output.log --owner some_owner --token "${BITBUCKET_TOKEN}"
INFO[0001] Created comment with id 437530034 and tag id stack https://bitbucket.org/some_owner/some_repo/pull-requests/29/_/diff#comment-437530034

Even though the arg is called --pull-request-id when using github the notifier tries to add a comment to an issue with the same id instead of the PR