Comments (5)
rayashworth
commented on July 24, 2024
@jagraj appeantly this could be a fyre issue, open a oneticket here. https://oneticket.devit.ibm.com/OneTicket/frontend/mainpage/index.html#!/login
from community-automation.
jagraj
commented on July 24, 2024
This seems to be intermittent problem but it is failing 80% of the time and passing 20% of the time. I just tried now and got into same problem and I am thinking playbook needs retry logic when it can not ssh to the infra node. After failure immediately I was able to ssh to the infra node and I verified all cephfs storage pods and they all running fine.
TASK [csi-cephfs-fyre : Wait for ceph to finish pod bring-up] ***************************************************************************************************************************************************************************************************************************
fatal: [9.30.209.135]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Shared connection to 9.30.209.135 closed.", "unreachable": true}
PLAY RECAP ******************************************************************************************************************************************************************************************************************************************************************************
9.30.209.135 : ok=13 changed=6 unreachable=1 failed=0 skipped=0 rescued=0 ignored=0
+ '[' 1 -eq 0 ']'
+ log 'ERROR: Ceph storage configuration failed.'
+ local 'msg=ERROR: Ceph storage configuration failed.'
++ date +%Y-%m-%dT%H:%M:%S%z
+ echo '2021-01-11T21:47:15+0000: ERROR: Ceph storage configuration failed.'
2021-01-11T21:47:15+0000: ERROR: Ceph storage configuration failed.
rook-ceph csi-cephfsplugin-5zbbc 3/3 Running 0 6m24s
rook-ceph csi-cephfsplugin-provisioner-5c65b94c8d-4fgq6 6/6 Running 0 6m24s
rook-ceph csi-cephfsplugin-provisioner-5c65b94c8d-k6nqt 6/6 Running 0 5m59s
rook-ceph csi-cephfsplugin-x4skl 3/3 Running 0 6m15s
rook-ceph csi-cephfsplugin-x6qcl 3/3 Running 0 6m24s
rook-ceph csi-rbdplugin-kdk89 3/3 Running 0 6m25s
rook-ceph csi-rbdplugin-m22vc 3/3 Running 0 6m15s
rook-ceph csi-rbdplugin-provisioner-569c75558-9g4dn 6/6 Running 0 6m25s
rook-ceph csi-rbdplugin-provisioner-569c75558-kkp4t 6/6 Running 0 5m59s
rook-ceph csi-rbdplugin-rscfp 3/3 Running 0 6m25s
rook-ceph rook-ceph-crashcollector-worker0.test-pr-1234.cp.fyre.ibm.g9vzd 1/1 Running 0 5m26s
rook-ceph rook-ceph-crashcollector-worker1.test-pr-1234.cp.fyre.ibm.gtk2w 1/1 Running 0 67s
rook-ceph rook-ceph-crashcollector-worker2.test-pr-1234.cp.fyre.ibm.lbk5m 1/1 Running 0 5m26s
rook-ceph rook-ceph-mds-myfs-a-69bfbc5cdb-drjt5 1/1 Running 0 67s
rook-ceph rook-ceph-mds-myfs-b-86b9d59664-t7cqb 1/1 Running 0 66s
rook-ceph rook-ceph-mgr-a-7465df4f6d-wrvth 1/1 Running 0 95s
rook-ceph rook-ceph-mon-a-79f5c6cfb5-9s96x 1/1 Running 0 2m43s
rook-ceph rook-ceph-mon-b-58cfb59746-pzdf9 1/1 Running 0 2m18s
rook-ceph rook-ceph-mon-c-784c4dcb44-9dpqn 1/1 Running 0 2m5s
rook-ceph rook-ceph-operator-59cbfb7c7c-pw5l9 1/1 Running 0 6m
rook-ceph rook-ceph-osd-0-74759cf8fd-kqgv4 1/1 Running 0 85s
rook-ceph rook-ceph-osd-1-586b4c7899-ggv9l 1/1 Running 0 84s
rook-ceph rook-ceph-osd-2-6f959bb755-q6zfq 1/1 Running 0 84s
rook-ceph rook-ceph-osd-prepare-worker0.test-pr-1234.cp.fyre.ibm.com2pdth 0/1 Completed 0 94s
rook-ceph rook-ceph-osd-prepare-worker1.test-pr-1234.cp.fyre.ibm.comvn66c 0/1 Completed 0 93s
rook-ceph rook-ceph-osd-prepare-worker2.test-pr-1234.cp.fyre.ibm.com2v2nw 0/1 Completed 0 93s
rook-ceph rook-discover-58crb 1/1 Running 0 7m23s
rook-ceph rook-discover-w7cnf 1/1 Running 0 7m23s
rook-ceph rook-discover-x7nbj 1/1 Running 0 7m23s
from community-automation.
jagraj
commented on July 24, 2024
FYI...
from community-automation.
rayashworth
commented on July 24, 2024
@jagraj I just updated the cephfs role to allow use of latest cephfs version.
Are you still using this, and can this issue be closed?
from community-automation.
jagraj
commented on July 24, 2024
@rayashworth
This can be closed now. Thanks.
from community-automation.
Related Issues (20)
- Jenkins job to create host with db2 fails to configure VNC HOT 1
- Getting an error while running SetupNewJmeterHost Jenkins job HOT 3
- as a automation developer I want to separate hive role into 2 roles so that provision and delete can be their own role
- Develop Playbook for Fyre Large cluster request HOT 7
- common fyre_vm failures after refactoring HOT 1
- Can't install ceph on OCP 4.9.0-fc.1 HOT 8
- Enable Fyre CephFS play to use more devices
- Fix use of uri module to use force_basic_auth HOT 2
- Fix cs_operator_catalog_image in common-service-play HOT 2
- sensitive info being logged to the console during vm provision
- rook/ceph not installing after v1.8.0
- update default rook/ceph version HOT 1
- as a devops developer I need to update error checking in fyre OCS automation so that fyre OCS installs can work across all OCP+ versions and platforms HOT 4
- recover-expired-certificates-play/ all files are all zero :( HOT 3
- add RedHat ODF storage install option to RedHat OCS automation HOT 2
- roles/request_ocs_local_storage/tasks/request_ocs_local_storage.yml#L4 fails as non-root user
- create a role to example playbook mapping using github actions and pages
- ODF install on larger cluster ( 8 workers ) timed out - 5 mins not enough time
- nfs-client-provisioner: unexpected error getting claim reference: selfLink was empty, can't make reference
- Convert csi-ceph from scripts to proper ansible HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from community-automation.