follow the instruction https://codefresh.io/steps/step/aws-sts-assume-role-with-web-identity to assume AWS IAM role, received below error:
[2024-01-04T10:43:38.969Z] Running plugin assume_role
[2024-01-04T10:43:48.472Z] Pulling image jland/aws-cli:2.8.12
[2024-01-04T10:43:56.254Z] Pulled layer '33b928eea496'
[2024-01-04T10:43:56.962Z] Pulled layer '43bc19c7da66'
[2024-01-04T10:44:03.576Z] Pulled layer '5e6c927777b7'
[2024-01-04T10:44:03.627Z] Pulled layer '2cb3999af40b'
[2024-01-04T10:44:03.639Z] Pulled layer 'bb2fe80ff7e4'
[2024-01-04T10:44:03.654Z] Pulled layer '112dc306a5df'
[2024-01-04T10:44:03.719Z] Digest: sha256:c0bb71d9f8d6e42f8363c915f023ab4037a6f75c7740cacb2418bc333c14b99f
[2024-01-04T10:44:03.722Z] Status: Downloaded newer image for quay.io/jland/aws-cli:2.8.12
[2024-01-04T10:44:04.468Z] ------------------------------
[2024-01-04T10:44:04.468Z] Executing command: TOKEN=$ID_TOKEN

[2024-01-04T10:44:04.468Z] SESSION_CREDS=$(aws sts assume-role-with-web-identity
[2024-01-04T10:44:04.468Z] --role-arn "$ROLE_ARN"
[2024-01-04T10:44:04.468Z] --role-session-name "$ROLE_SESSION_NAME"
[2024-01-04T10:44:04.468Z] --web-identity-token "$TOKEN"
[2024-01-04T10:44:04.468Z] --output json
[2024-01-04T10:44:04.468Z] --query Credentials)

[2024-01-04T10:44:04.468Z] AWS_ACCESS_KEY_ID=$(echo "$SESSION_CREDS" | jq -r .AccessKeyId)
[2024-01-04T10:44:04.468Z] AWS_SECRET_ACCESS_KEY=$(echo "$SESSION_CREDS" | jq -r .SecretAccessKey)
[2024-01-04T10:44:04.468Z] AWS_SESSION_TOKEN=$(echo "$SESSION_CREDS" | jq -r .SessionToken)

[2024-01-04T10:44:04.468Z] cf_export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID --mask
[2024-01-04T10:44:04.468Z] cf_export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY --mask
[2024-01-04T10:44:04.468Z] cf_export AWS_SESSION_TOKEN=$AWS_SESSION_TOKEN --mask

[2024-01-04T10:44:09.381Z] /bin/sh: line 26: jq: command not found
[2024-01-04T10:44:09.523Z] Reading environment variable exporting file contents.
[2024-01-04T10:44:10.426Z] [SYSTEM]
Message Failed to run freestyle step: main
Caused by Container for step title: main, step type: freestyle, operation: Freestyle step. Failed with exit
code: 127
Documentation Link https://codefresh.io/docs/docs/codefresh-yaml/steps/freestyle/
Exit code 127
Name NonZeroExitCodeError
[2024-01-04T10:44:10.504Z] ------
[2024-01-04T10:44:11.010Z] [SYSTEM]
Message step: assume_role failed
Caused by plugin failed
Documentation Link https://codefresh.io/docs/docs/codefresh-yaml/steps

Please help to check. thanks.

What's the reasoning behind making this field required?

I just want to send SLACK_ATTACHMENTS

Add a mechanism to deny/wait when a conflict is detected.
Possible solution:

• add a parameter to drive the behaviour
  • name: CR_CONFLICT_POLICY
  • value:
    • ignore (default to keep current behaviour)
    • wait: Keep the flow running until the conflict is removed
    • deny: send a deny order to the pipeline

In addition, we will add an annotation to indicate why the pipeline is deny (Change request cancelled or conflict)

hi,

i am new to codefresh , and trying sub modules.

steps:
    get_git_token:
        title: Reading Github token
        stage: prepare
        image: codefresh/cli
        commands:
           - cf_export GITHUB_TOKEN=$(codefresh get context github --decrypt -o yaml | yq -y .spec.data.auth.password)
    main_clone:
        title: Cloning repository...
        stage: prepare
        type: git-clone
        arguments:
            repo: '${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}'
            git: eds_bitbucket
            revision: '${{CF_BRANCH}}'
    init_submodules:
        title: Init Submodules
        stage: prepare
        image: codefreshio/git-image:latest
        working_directory: ${{main_clone}} 
        environment:
              - CF_SUBMODULE_SYNC=true
              - CF_SUBMODULE_UPDATE_RECURSIVE=true

I tried this but does nothing

Getting an error:

HTTP code 404) unexpected - manifest for codefreshplugins/plugin-kompose:0.0.3 not found                                                                        
[SYSTEM]                                                                                                                                                         
 Message             Failed to run freestyle step: main                                                                                                          
 Caused by           Image Not Found Error                                                                                                                       
 Documentation Link  https://codefresh.io/docs/docs/codefresh-yaml/steps/freestyle/                                                                              
 Exit code           44                                                                                                                                          
 Name                NonZeroExitCodeError                                                                                                                        
------                                                                                                                                                           
[SYSTEM]                                                                                                                                                         
 Message             step: delploy failed                                                                                                                        
 Caused by           plugin failed                                                                                                                               
 Documentation Link  https://codefresh.io/docs/docs/codefresh-yaml/steps     

My step

  delploy:
    type: kompose
    arguments:
      KUBE_CONTEXT: "${{KUBE_CONTEXT}}"
      NAMESPACE: "${{NAMESPACE}}"
      FILE: docker-compose.production.yml

This is a suggested enhancement. There can be concurrent jobs running and it is possible that there can some git conflicts when multiple job are updating same repo.

The idea is to add support for rebasing before pushing. An argument 'rebase' can be used to toggle this behaviour.

P.S I can work on this if you think this aligns with the philosophy.

https://github.com/codefresh-io/steps/tree/master/incubating/git-submodules

The git-submodule step assumes that the current working directory is the main_clone repo (as it is on free=style steps).

But in typed-steps, the working directory is always /codefresh/volume.

We need to add an argument to the step, including the path to set as the working directory during the step execution.

Related to ticket 1676

I am using the helm step with action "install" to deploy a package stored in my Codefresh helm private repository.

The deployment is failing because is looking at the local directory for a Chart.yaml.

I expect the command to find the package in the repository.

Below are the commands and output:

helm version --short -c                                                                                                                             
helm repo add cf-helm-default cm://h.cfcr.io/<username>/default/                                                                                       
helm upgrade java-app-test-r java-app-test --install --reset-values --repo cm://h.cfcr.io/<username>/default/ --namespace "ns" --set image.pullPolicy=Always --set image.tag="master-44c3d06"                                                                                                   
----------------------------                                                                                                                        
                                                                                                                                                    
Switched to context "cluster".                                                                                                                
Client: v2.16.1+gbbdfe5e                                                                                                                            
Reading environment variable exporting file contents.                                                                                               
Error: no Chart.yaml exists in directory "/codefresh/volume/java-app-test"                                                                          
"cf-helm-default" has been added to your repositories ```

Add wait option BEFORE sync in argocd-sync step-type

Similar to this:
argocd app wait APPNAME --operation && argocd app sync APPNAME

While trying to copy the variables from source pipe line using "import from file" or "Import from text" option , variables are getting imported as blank key value pair

We have multiple repositories and a single helm chart. Pipeline for each repository only knows the image-version of its own docker image but not others. With regular helm binary we can use '--reuse-values' and only supply the changed image as part of deployment.

Official helm step by codefresh explicitly resets values and thus requires all values. See below code snippet in official plugin

    def build_export_commands(self, google_application_credentials_json):
        lines = super().build_export_commands(google_application_credentials_json)
        return lines

    def build_helm_upgrade_command(self, release_name, chart_ref):
        return 'helm upgrade %s %s --install --reset-values ' % (release_name, chart_ref)

    def build_repo_commands(self):
        lines = []
        lines.append('helm repo add cf-stable https://kubernetes-charts.storage.googleapis.com/')
        return lines

How can we override this behavior? Do codefresh provide steps/feature to maintain a central values file?

PagerDuty's Events API v2 (docs) allows sending two types of events: standard events (docs) and change events (docs). They both use a routing/integration key, while PagerDuty's REST API v2 uses an API key.

Currently, the official PagerDuty step provided by Codefresh implements the REST API for the PAGERDUTY_ALERT_TYPE of incident, and only the change events subset of the Events API for the PAGERDUTY_ALERT_TYPE of change_event. This means that users of the Events API can only send change events, but not trigger incidents by sending a normal event.

The PagerDuty step should support the entire Events API instead of just a subset. This can be done without breaking backwards compatibility. The step uses PagerDuty's pdpyras client to send requests, and implementation is as simple as importing its existing EventsAPISession.

Personally, I've had to implement my own custom PagerDuty step to create alerts from Codefresh, which isn't ideal.

Do rollback to previous rollout if wait command get failed

There may be an issue with the Trivy Scan step. We attempted to use it and received an invalid_blocks message at the point when it attempt to send a message to slack. We are attempting to run the linked entrypoint.sh script locally now to see if we can identify where the slack message body is malformed.

Hi, please would it be possible to add the Helm Secrets plugin to the Helm step as this would enable us to deploy charts with SOPS encrypted values.
Thanks

we wasted 4 hours on debugging why custom variables are automatily changed from '_' to '.' ,
Please fix or add documantaion

Sync to a specific revision. Preserves parameter overrides

Serverless step is documented as type: serverless but expecting type: lambda

I see in the code where TIMEOUT_MINS has a default value of 45 (minutes). But I'm not convinced that actually works. I have a couple pipelines that run about 2 hours but don't time out.

The pipeline definition is very simple:

version: "1.0"

steps:
  test:
    title: "Running All Mid Tier Tests on develop-shadow"
    type: codefresh-run
    arguments:
      PIPELINE_ID: platform/SystemTestAndReport
      VARIABLE: 
        - SUT_NAMESPACE=develop-shadow
        - PYTEST_MARK=all

Currently in https://codefresh.io/docs/docs/new-helm/using-helm-in-codefresh-pipeline/ there is only a 'chart_version'. parameter when the action is "push".

This parameter is added as the --version switch for the "helm package" command executed.

I have a need to be able to set the "--app-version" switch of the "helm package".

Please add this as a parameter of the Codefresh Helm Step

Currently the helm step only takes advantage of kube context that has already been configured.
Can you add an option for this step to run this prior to running helm upgrade?

aws eks --region <region> update-kubeconfig --name <cluster name> --alias <friendly name>

• No Clair image shown in UI. (Showing Gear Icon)
• No pull access to Docker registry.

(HTTP code 404) unexpected - pull access denied for codefreshplugins/paclair, repository does not exist or may require 'docker login'           
[SYSTEM]                                                                                                                                        
 Message             Failed to pull base image: codefreshplugins/paclair:latest       

I recently attempted to use the codefresh-dynamic-run step for a release process that involves a variable number of environments (I realize it is incubating). My plan was to generate the dynamic-run yaml based on existing configuration and spawn N child pipelines to carry out the release process. Technically the step works, I was able to pass the required parameters to a generalized child pipeline, however the visual feedback makes the step very confusing for a broad user base. I have resorted to a more static approach to my process in order to preserve the visual feedback users will expect without searching through logs to determine a dependent pipeline.

I would request that:
-a node is created in the graph for each sub pipeline in the dynamic run step
-the node has a link to the child pipeline execution
-the node indicates the status of the child pipeline execution

I am guessing this may already be under consideration, but I just wanted to reiterate the importance of the dynamic-run step for more complex pipeline design and value of visual feedback for end users.

vault supports secrets with spaces, as long as you store them surrounded in quotes (although highly discouraged).. if a secret contains spaces, the pipeline does not parse it properly.

see: https://g.codefresh.io/build/5e610935e7860798c2604992?step=use_vault_secret&tab=output

When triggering pipeline from BitBucket Pull Request (pullrequest:created or pullrequest:updated events) the step sonar-quality-gates-checker will always fail. The required CF_PULL_REQUEST_NUMBER field for PRs is not set by codefresh. We have CF_PULL_REQUEST_ID.

Is it possible to fail the step when status comes any value other than Healthy?

Can you update Helm version to latest in Helmfile step?

should be equal to one of the allowed values                                                                                                              
keyword:enum                                                                                                                                              
data path:.HELM_VERSION                                                                                                                                   
schema path: #/properties/HELM_VERSION/enum                                                                                                               
params:                                                                                                                                                   
allowedValues:2.14.3,2.17.0,3.0.0,3.0.3,3.1.1  

I've set up a file sonar-project.properties in the root folder of the project. Within it there is the sonar.projectKey= set up with the sonar project key. Although sonar-quality-gates-checker does not use it, and the error is :

sonarqube.utils.exceptions.ClientError: Error in request. Possibly client error [404]: Project '${{SONAR_PROJECT_KEY}}' not found

Although another step that uses sonar-scanner-cli correctly uses the file sonar-project.properties

When using the base url argument, the passed parameter is being ignored because the field in index.js is called baseurl not base-url

In run.sh it gets called using the string base-url
https://github.com/codefresh-io/steps/blob/master/incubating/github-release/run.sh#L94

In index.js it is accepted as a parameter using the string baseurl
https://github.com/codefresh-io/steps/blob/master/incubating/github-release/github-release-cli/src/index.js#L18

There is no way to pass the image tag to the ecs-deploy step, although the underlying tool does accept the --image-tag argument. This means that the only vanilla ECS step available in the Marketplace doesn't actually work for the most common pipeline scenario (build new code, create image, deploy new image to ECS).

It took me several days of communicating with support to work this out.

For anyone who runs into this before the step is fixed, the solution is to use a freestyle step as described in the ECS/Fargate example.

Hi there,

I have a pipeline with parallel steps, in which I need to deploy helm charts in parallel [1]

The problem I am facing is, apparently the helm step always run kubectl config use-context "<MY-CONTEXT>"

And since volumes are shared, it fails with the following error:

error: open /codefresh/volume/sensitive/.kube/config.lock: file exists

I think one way of solving dealing with this is to expose a new field skip_kube_config_init: true|false

Thoughts?

[1]

i'm testing this out since we want a way to make sure the app is healthy.
Is it possible to fail the step when status comes any value other than Healthy?

Hey!

I noticed today that Microsoft is retiring Teams connectors in favor of teams workflows.

See here: https://devblogs.microsoft.com/microsoft365dev/retirement-of-office-365-connectors-within-microsoft-teams/

Can we get https://github.com/codefresh-io/steps/tree/master/incubating/msteams-notifier updated to use a workflow webhook url instead of a connector?

For a helm step, in the examples and documentation a action: push is given without passing a kube_context argument.
But kube_context is marked as 'required' in the docs and in the yaml/json schema.
So the step fails with a validation error.

So either kube_context is made conditional with some subschemas or the examples need to be changed (though a kubernetes cluster isn't needed to only push a chart).

If you have a name with a single space it ignores the second word. If you have two or more spaces it break the build. You should quote the user.name value

In addition to create a new CR,
add an option to close the one created in the Pipeline

Could you please post the YML step example used to produce the notification that is ilustrated in the README.md file as an image?

https://github.com/codefresh-io/steps/blob/master/incubating/git-commit/step.yaml#L216 this pipeline recently changed and now echos the whole REPO_URL instead of REDACTED in place of the access token, as it did previously. It looks like this was introduced with the changes to try to support ssh instead of https.

run-jenkins-jobs - needs to be able to watch Jenkins job and report back status.

When running the webhook step, it now fails with:

Digest: sha256:c7f49c71808c1351afa0396e0e477ee4fd70ad948c23bdb8ac6c9fd35d81c097                                                                                  
Status: Downloaded newer image for codefresh/webhook-plugin:0.0.5                                                                                                
internal/modules/cjs/loader.js:582                                                                                                                               
    throw err;                                                                                                                                                   
    ^                                                                                                                                                            
                                                                                                                                                                 
Error: Cannot find module 'request-promise'                                                                                                                      
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:580:15)                                                                                  
    at Function.Module._load (internal/modules/cjs/loader.js:506:25)                                                                                             
    at Module.require (internal/modules/cjs/loader.js:636:17)                                                                                                    
    at require (internal/modules/cjs/helpers.js:20:18)                                                                                                           
    at Object.<anonymous> (/app/plugin/plugin.logic.js:2:17)                                                                                                     
    at Module._compile (internal/modules/cjs/loader.js:688:30)                                                                                                   
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)                                                                                     
    at Module.load (internal/modules/cjs/loader.js:598:32)                                                                                                       
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)                                                                                                     
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)

It was working 9 hours ago and I'm assuming db06a7d broke it

There seems to be a bug in the codefresh-run-dynamic step, in the case where you don't wait for the pipelines to finish. The pipelines are triggered as they should be, but the step fails with an error:

Executing command: python3 /codefresh-run-dynamic.py                                                                                      
Running pipelines...                                                                                                                      
                                                                                                                                          
codefresh run test/deploy1 --detach                                                                                                       
Started build https://g.codefresh.io/build/613f1202f755fb7679444a2b                                                                       
                                                                                                                                          
codefresh run test/deploy2 --detach                                                                                                       
Started build https://g.codefresh.io/build/613f12057f34fcca9c477aba

Traceback (most recent call last):                                                                                                        
  File "/codefresh-run-dynamic.py", line 315, in <module>                                                                                 
    main()                                                                                                                                
  File "/codefresh-run-dynamic.py", line 311, in main                                                                                     
    write_output_files(builds_started, success)                                                                                           
UnboundLocalError: local variable 'success' referenced before assignment                                                                  
Reading environment variable exporting file contents.

The variable success is only set with the wait flag, but used as a parameter for write_output_files() either way.

for make my release based on commit or custom branch i need this parameter commitish

doc: https://github.com/cheton/github-release-cli#specify-the-commitish-value-for-tag