Giter Club home page Giter Club logo

devsecops-demo's People

Contributors

afouladi7 avatar fduthilleul avatar mooyeg avatar piggyvenus avatar plewyllie avatar rcarrata avatar sebw avatar viniciusfcf avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

devsecops-demo's Issues

Installation failing on OCP 4.9.10 (ROSA)

./install.sh

INFO: Installing Demo
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Install the ACS Demo] ********************************************************************************************************************************************

TASK [Gathering Facts] *************************************************************************************************************************************************
ok: [localhost]

TASK [Install Gitops] **************************************************************************************************************************************************

TASK [ocp4-install-gitops : Create Namespaces] *************************************************************************************************************************
changed: [localhost]

TASK [ocp4-install-gitops : Install GitOps Operator] *******************************************************************************************************************
changed: [localhost]

TASK [ocp4-install-gitops : Wait for GitOps CRD to exist] **************************************************************************************************************
FAILED - RETRYING: Wait for GitOps CRD to exist (30 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (29 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (28 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (27 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (26 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (25 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (24 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (23 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (22 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (21 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (20 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (19 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (18 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (17 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (16 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (15 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (14 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (13 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (12 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (11 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (10 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (9 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (8 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (7 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (6 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (5 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (4 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (3 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (2 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (1 retries left).
failed: [localhost] (item=applications.argoproj.io) => {"ansible_loop_var": "item", "api_found": false, "attempts": 30, "changed": false, "item": "applications.argoproj.io", "msg": "Failed to find API for resource with apiVersion "apiextensions.k8s.io/v1beta1" and kind "CustomResourceDefinition"", "resources": []}
FAILED - RETRYING: Wait for GitOps CRD to exist (30 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (29 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (28 retries left).
FAILED - RETRYING: Wait for GitOps CRD to exist (27 retries left).

CRDS and Pods appear to have been created so the reason for the failure is not clear.

[jwilms@jwilms ~]$ oc get crd | grep gitops
gitopsservices.pipelines.openshift.io 2021-12-15T00:56:50Z

And it also appears that everything is running ok in the openshift-gitops namespace:

[jwilms@jwilms ~]$ oc get pods
NAME READY STATUS RESTARTS AGE
pod/cluster-86f5997f56-jsm7z 1/1 Running 0 6m16s
pod/kam-8579df68c8-9d8pt 1/1 Running 0 6m15s
pod/openshift-gitops-application-controller-0 1/1 Running 0 6m14s
pod/openshift-gitops-applicationset-controller-76dfff754b-4rzxm 1/1 Running 0 6m14s
pod/openshift-gitops-dex-server-6cf4f8d67c-vswn4 1/1 Running 0 6m14s
pod/openshift-gitops-redis-7867d74fb4-shzfg 1/1 Running 0 6m15s
pod/openshift-gitops-repo-server-bb4f985c8-fksj5 1/1 Running 0 6m15s
pod/openshift-gitops-server-6cfc85cbb8-rbpgg 1/1 Running 0 6m14s

Few issues with OCP 4.11

Hello,

I tried to deploy this demo on a 4.11 OCP cluster.

Pre-requisites are installed :

$ pip3 list | grep -e kubernetes -e openshift -e jmespath 
jmespath            1.0.1
kubernetes          24.2.0
openshift           0.13.1

$ ansible --version 
ansible [core 2.13.4]
  config file = None
  configured module search path = ['/Users/slallema/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/Cellar/ansible/6.4.0/libexec/lib/python3.10/site-packages/ansible
  ansible collection location = /Users/slallema/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible
  python version = 3.10.7 (main, Sep 14 2022, 22:38:23) [Clang 14.0.0 (clang-1400.0.29.102)]
  jinja version = 3.1.2
  libyaml = True

I have a first issue with the install.sh phase and the ocp4-post-acs task :

TASK [ocp4-post-acs : Get the secret that contains the token of sa pipeline] ***************************************************************
ok: [localhost] => {"ansible_facts": {"token_sa_pipeline_secret": []}, "changed": false}

TASK [ocp4-post-acs : Get token in the secret for the sa pipeline and decode] **************************************************************
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an undefined variable. The error was: list object has no element 0\n\nThe error appears to be in '/Users/slallema/GIT/github.com/slallemand/devsecops-demo/bootstrap/roles/ocp4-post-acs/tasks/post_ci.yaml': line 68, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- name: Get token in the secret for the sa pipeline and decode\n  ^ here\n"}

PLAY RECAP *********************************************************************************************************************************
localhost                  : ok=70   changed=6    unreachable=0    failed=1    skipped=3    rescued=0    ignored=0   

Anyway, i did try to start the pipeline with the ./demo.sh start but the pipeline is failing at the build-image task.
I have those errors :

STEP-GEN-ENV-FILE

2022/10/05 13:03:28 warning: unsuccessful cred copy: ".docker" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.docker: permission denied
UID uid=1001(1001) gid=0(root) groups=0(root),1000660000
Generated Env file
------------------------------
MAVEN_CLEAR_REPO=false
MAVEN_MIRROR_URL=http://nexus:8081/repository/maven-public/
------------------------------
STEP-GENERATE

2022/10/05 13:03:29 warning: unsuccessful cred copy: ".docker" from "/tekton/creds" to "/": unable to create destination directory: mkdir /.docker: permission denied
UID uid=1001(1001) gid=0(root) groups=0(root),1000660000 s2i build spring-petclinic/target image-registry.openshift-image-registry.svc:5000/openshift/java:11 --image-scripts-url image:///usr/local/s2i --as-dockerfile /gen-source/Dockerfile.gen --environment-file /env-params/env-file
STEP-BUILD

Error: error writing "0 0 4294967295\n" to /proc/29/uid_map: write /proc/29/uid_map: operation not permitted
level=error msg="error writing \"0 0 4294967295\\n\" to /proc/29/uid_map: write /proc/29/uid_map: operation not permitted"
level=error msg="(unable to determine exit status)"
STEP-PUSH-TAG

2022/10/05 13:03:31 Skipping step because a previous step failed
STEP-PUSH-LATEST

2022/10/05 13:03:32 Skipping step because a previous step failed

Could that be relative to the install error ?
Any idea on that ?

OCP 4.11 install

The pipelines operator was not coming up in my OCP 4.11 install. I had to deinstall the operator, and install it from the OperatorHub. Then, relaunching the install.sh worked.

Did not have time to investigate really the cause, but I thought I'd share the issue.

enable keycload / ocp sso instead of dex

Enable OAUTH in ArgoCD 1.2

oc -n openshift-gitops patch argocd openshift-gitops --type='json' -p='[{"op": "add", "path": "/spec/sso", "value": {"provider": "keycloak"} }]'

Enable the admin role in ArgoCD

oc patch cm/argocd-rbac-cm -n openshift-gitops --type=merge -p '{"data":{"policy.default":"role:admin"}}'

Image Scan fails due to roxctl cli output changes

Due to updates and deprecations in roxctl the format output is no longer supporting pretty for the formatting:

## Scanning image image-registry.openshift-image-registry.svc:5000/cicd/spring-petclinic@sha256:48cea137aab4dfad9189f0d733b8c11ef1466ef74f78043d11a9fa8c527fef7c
Flag --format has been deprecated, please use --output/-o to specify the output format. NOTE: The new JSON / CSV format contains breaking changes, make sure you adapt to the new structure before migrating.
ERROR:	invalid arguments: invalid output format "pretty" used. You can only specify json or csv
## Go to https://central-stackrox.apps.cluster-m7mtg.m7mtg.xxx.opentlc.com:443/main/vulnerability-management/image/sha256:48cea137aab4dfad9189f0d733b8c11ef1466ef74f78043d11a9fa8c527fef7c to check more info

Failure uploading the zap proxy report into the upload server

+ ls -lhrt /zap/wrk
total 76K

-rw-r--r--. 1 zap zap 75K Aug 20 10:41 petclinic-build-devm9hqv.html
+ echo 'Uploading the report into the report server'
Uploading the report into the report server

+ curl -u reports:reports -F path=petclinic-build-devm9hqv.html -F file=/zap/wrk/petclinic-build-devm9hqv.html -X POST http://reports-repo:8080/upload
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   335  100    36  100   299   7200  59800 --:--:-- --:--:-- --:--:-- 67000
{"message":"Internal Server Error"}

ArgoCD server is exposing a not valid certificate and Task for Wait Application is failing

The GitOps / ArgoCD Server have not the proper certificate SAN for the openshift-gitops-server.openshift-gitops, and for this reason is failing to login and to do the app sync. Needs to be updated with insecure or use the http instead of https inside of the cluster.

step-login-wait
+ [ -z ]
+ yes
+ argocd login openshift-gitops-server.openshift-gitops:443 --username=admin --password=czX6GbpBg4UaODnM1yKvdlRm8FsYE3fW
WARNING: server certificate had error: x509: certificate is valid for openshift-gitops, openshift-gitops-grpc, openshift-gitops.openshift-gitops.svc.cluster.local, not openshift-gitops-server.openshift-gitops. Proceed insecurely (y/n)? 'admin:login' logged in successfully
Context 'openshift-gitops-server.openshift-gitops:443' updated

step-sync
+ argocd app sync dev-spring-petclinic --revision HEAD --
time="2022-02-25T13:14:53Z" level=fatal msg="Failed to establish connection to openshift-gitops-server.openshift-gitops:443: x509: certificate is valid for openshift-gitops, openshift-gitops-grpc, openshift-gitops.openshift-gitops.svc.cluster.local, not openshift-gitops-server.openshift-gitops"

Failure in the installation of the Openshift GitOps

Due to the newer version of Openshift GitOps 1.3, the CRs changed. Further investigation it's required.

< TASK [ocp4-install-gitops : Wait for GitOps CRD to exist] >
 -----------------------------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

ok: [localhost] => (item=applications.argoproj.io)

FAILED - RETRYING: Wait for GitOps CRD to exist (1 retries left).
failed: [localhost] (item=applicationsets.argoproj.io) => {"ansible_loop_var": "item", "api_found": true, "attempts": 30, "changed": false, "item": "applicationsets.argoproj.io", "resources": []}
ok: [localhost] => (item=appprojects.argoproj.io)
ok: [localhost] => (item=argocds.argoproj.io)

code analysis sometimes randomly fails

when the code analysis runs sometimes are failing raising this type of error:

[�[1;31mERROR�[m] /workspace/source/spring-petclinic/src/test/java/org/springframework/samples/petclinic/service/ClinicServiceTests.java:[30,51] cannot access org.springframework.samples.petclinic.owner.Pet
  bad class file: /workspace/source/spring-petclinic/target/classes/org/springframework/samples/petclinic/owner/Pet.class
    class file contains wrong class: org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest
    Please remove or make sure it appears in the correct subdirectory of the classpath.
[�[1;34mINFO�[m] 1 error
[�[1;34mINFO�[m] -------------------------------------------------------------
[�[1;34mINFO�[m] 
[�[1;34mINFO�[m] �[1m------------------------------------------------------------------------�[m
[�[1;34mINFO�[m] �[1mSkipping petclinic�[m
[�[1;34mINFO�[m] This project has been banned from the build due to previous failures.
[�[1;34mINFO�[m] �[1m------------------------------------------------------------------------�[m
[�[1;34mINFO�[m] �[1m------------------------------------------------------------------------�[m
[�[1;34mINFO�[m] �[1;31mBUILD FAILURE�[m
[�[1;34mINFO�[m] �[1m------------------------------------------------------------------------�[m
[�[1;34mINFO�[m] Total time: 01:08 min
[�[1;34mINFO�[m] Finished at: 2021-07-22T06:09:15Z
[�[1;34mINFO�[m] Final Memory: 118M/1460M
[�[1;34mINFO�[m] �[1m------------------------------------------------------------------------�[m
[�[1;31mERROR�[m] Failed to execute goal �[32morg.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile�[m �[1m(default-testCompile)�[m on project �[36mspring-petclinic�[m: �[1;31mCompilation failure�[m
[�[1;31mERROR�[m] �[1;31m/workspace/source/spring-petclinic/src/test/java/org/springframework/samples/petclinic/service/ClinicServiceTests.java:[30,51] cannot access org.springframework.samples.petclinic.owner.Pet�[m
[�[1;31mERROR�[m] �[1;31m  bad class file: /workspace/source/spring-petclinic/target/classes/org/springframework/samples/petclinic/owner/Pet.class�[m
[�[1;31mERROR�[m] �[1;31m    class file contains wrong class: org.springframework.boot.test.autoconfigure.orm.jpa.DataJpaTest�[m
[�[1;31mERROR�[m] �[1;31m    Please remove or make sure it appears in the correct subdirectory of the classpath.�[m
[�[1;31mERROR�[m] �[1;31m�[m
[�[1;31mERROR�[m] -> �[1m[Help 1]�[m
[�[1;31mERROR�[m] 
[�[1;31mERROR�[m] To see the full stack trace of the errors, re-run Maven with the �[1m-e�[m switch.
[�[1;31mERROR�[m] Re-run Maven using the �[1m-X�[m switch to enable full debug logging.
[�[1;31mERROR�[m] 
[�[1;31mERROR�[m] For more information about the errors and possible solutions, please read the following articles:
[�[1;31mERROR�[m] �[1m[Help 1]�[m http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

Explore GitSecrets of awslabs

Git Secrets could be a nice addon to the pipeline in order to ensure that the git repo have not any exposed secret. Additionally ACS includes out of the box config management for avoid expose any secret/CM in the k8s cluster.

Openshift GitOps for springpetclinic-dev stucks in wait sync up of ArgoCD

GROUP               KIND        NAMESPACE      NAME              STATUS     HEALTH   HOOK  MESSAGE
                    Service     devsecops-dev  spring-petclinic  Synced     Healthy        
apps                Deployment  devsecops-dev  spring-petclinic  OutOfSync  Healthy        
route.openshift.io  Route       devsecops-dev  spring-petclinic  Synced                    
time="2021-11-03T08:34:01Z" level=fatal msg="Operation has completed with phase: Running"

Pipeline petclinic-build-dev - task scan image error

Since I have a free docker.io account, I am getting this error. Is it possible to avoid this?

PodPpetclinic-build-dev-cw8uq2-deploy-check-s7lww-pod-9ts67
NamespaceNScicd
5 minutes ago
Generated from kubelet on ip-10-0-184-15.us-west-1.compute.internal
5 times in the last 13 minutes
Failed to pull image "centos": rpc error: code = Unknown desc = Error reading manifest latest in docker.io/library/centos: toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit

image-check issue for internal registry

Hi,
Getting this in image-check task on OCP 4.10 during pipelinerun.
oc -n cicd logs petclinic-build-dev-75x4fy-image-check-pod -c step-rox-image-check
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 65.2M 100 65.2M 0 0 247M 0 --:--:-- --:--:-- --:--:-- 247M
Getting roxctl
ERROR: Checking image failed: could not check build-time alerts: rpc error: code = Internal desc = image enrichment error: error getting metadata for image: image-registry.openshift-image-registry.svc:5000/cicd/spring-petclinic@sha256:4b6e957cb83f6e5687b71c012343308fb15ac0bce23c1be85d9e2a29d340b29c error: getting metadata from registry: "Autogenerated https://image-registry.openshift-image-registry.svc:5000 for cluster development": Failed to get the manifest digest : Head "https://image-registry.openshift-image-registry.svc:5000/v2/cicd/spring-petclinic/manifests/sha256:4b6e957cb83f6e5687b71c012343308fb15ac0bce23c1be85d9e2a29d340b29c": http: non-successful response (status=401 body=""). Retrying after 3 seconds...

I found this: https://access.redhat.com/solutions/6993372
So wondering how this is working on your side..
Thanks

Upgrade Sonarqube Image to 9.1.0-community

Due to the image of the Sonarqube image have more than 90 days, we need to update towards the 9.1.0-community with tag "sonarqube:9.1.0-community", and test it in the devsecops demo.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.