Comments (7)
lotharschulz
commented on May 28, 2024
#45 merged
from ghe-backup.
lotharschulz
commented on May 28, 2024
trigger new deployment, will free up disk by deleting older backups to have space available for new backup attempt
from ghe-backup.
lotharschulz
commented on May 28, 2024
20170221T171301 backup is started. wait for this to be completed (~80G available on /data should be enough)
from ghe-backup.
lotharschulz
commented on May 28, 2024
redeploy attempt failed:
Feb 21 20:25:46 ip-172-31-138-151 taupage-init: INFO: Attached EBS volume 'ghe-backup-volume2' as '/dev/sdf'
Feb 21 20:25:46 ip-172-31-138-151 taupage-init: INFO: /dev/xvdf: erase_on_boot=False, erase_tag_set=False
Feb 21 20:25:46 ip-172-31-138-151 taupage-init: INFO: Waiting for /dev/xvdf to stabilize..
Feb 21 20:25:47 ip-172-31-138-151 kernel: [ 45.944970] blkfront: xvdf: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled;
Feb 21 20:25:47 ip-172-31-138-151 kernel: [ 45.948950] xvdf: unknown partition table
Feb 21 20:25:51 ip-172-31-138-151 taupage-init: ERROR: Operational error /dev/xvdf.
Feb 21 20:25:51 ip-172-31-138-151 taupage-init: Traceback (most recent call last):
Feb 21 20:25:51 ip-172-31-138-151 taupage-init: File "./init.d/10-prepare-disks.py", line 124, in check_partition
Feb 21 20:25:51 ip-172-31-138-151 taupage-init: call_command(call)
Feb 21 20:25:51 ip-172-31-138-151 taupage-init: File "./init.d/10-prepare-disks.py", line 98, in call_command
Feb 21 20:25:51 ip-172-31-138-151 taupage-init: raise CmdException(proc.returncode, stderr.decode('utf-8'))
Feb 21 20:25:51 ip-172-31-138-151 taupage-init: CmdException: Command returned non-zero exit status 8:
Feb 21 20:25:51 ip-172-31-138-151 taupage-init: e2fsck: Can't allocate 160258200 bytes for directory map
Feb 21 20:25:51 ip-172-31-138-151 taupage-init:
Feb 21 20:25:51 ip-172-31-138-151 taupage-init: e2fsck: aborted
Feb 21 20:25:51 ip-172-31-138-151 taupage-init:
Feb 21 20:25:51 ip-172-31-138-151 taupage-init: ERROR: Failed to start 10-prepare-disks.py
(https://github.com/zalando-stups/taupage/blob/master/runtime/opt/taupage/init.d/10-prepare-disks.py#L135)
Looks like the former full disk caused the issue above.
from ghe-backup.
lotharschulz
commented on May 28, 2024
new volume created, initialised and attached:
root@08af65be79f8:/backup# cd /data/
root@08af65be79f8:/data# ls
lost+found
from ghe-backup.
lotharschulz
commented on May 28, 2024
backup attempt started manually:
root@xxxx:/data# /backup/backup-utils/bin/ghe-backup -v
Creating the backup data directory ...
Starting backup of gh.xus.zalando.de in snapshot 20170221T212602
Connect gh.xus.zalando.de:122 OK (v2.8.7)
Backing up GitHub settings ...
* Transferring settings data ...
* Transferring license data ...
* Transferring management console password ...
* Transferring SAML keys ...
* Transferring UUID ...
Backing up SSH authorized keys ...
Backing up SSH host keys ...
Backing up MySQL database ...
Backing up Redis database ...
Backing up audit log ...
Backing up hookshot logs ...
Backing up Git repositories ...
* Transferring auxiliary files ...
receiving incremental file list
./
0/
0/00/
0/00/99/
0/00/99/c6/
0/00/99/c6/gist/
0/00/99/c6/gist/9a9dd7f3c192dfaf4xxx.git/
...
from ghe-backup.
lotharschulz
commented on May 28, 2024
root@.......:/backup# ll /data/ghe-production-data
....
drwxr-xr-x 6 root root 4096 Feb 21 21:32 20170221T212602/
from ghe-backup.
Related Issues (20)
- current variable expansion throws exception in some cases
- backup schedule overlap HOT 1
- Backup process hangs after a couple of tries due to docker fifo issue HOT 8
- run on Zalando Kubernetes setup HOT 3
- hourly backups HOT 1
- dirty repository
- Docker image creation failed
- create and push docker image to zalando registry HOT 6
- deploy to kubernetes
- ghe-backup-volume could not be mounted again after a failure/termination or stop HOT 1
- update base image HOT 1
- make kms decryption region a parameterised information HOT 1
- add test for kubernetes_labels existance
- be DRY - create ONE function writing the ssh private key for both, Kubernetes & AWS/Taupage HOT 2
- delivery.yaml in k8s-master-like and master branch HOT 2
- follow up with code climate findings
- replace-convert-properties.sh is not added in dockerfile HOT 2
- correct permissions for /kms/convert-kms-private-ssh-key.sh HOT 2
- fixing ssh private key HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ghe-backup.