manheim / backupsss Goto Github PK
View Code? Open in Web Editor NEWA gem for archiving data and pushing it to S3.
License: MIT License
backupsss's People
Watchers
backupsss's Issues
tar gzipped files should have correct extension
when choosing to compress output, the file produced is .tar
it should be either .tgz or .tar.gz
output held onto when running in Docker
We're building backupsss into a docker container like:
FROM ubuntu:latest
RUN apt-get update && apt-get install -y ruby
RUN gem install --no-ri --no-rdoc backupsss -v 0.3.1(which gives ruby 2.3.1)
And running with (simplified): docker run <container name> backupsss.
However, when the cron timer ticks and the backups run, we see output from everything up to Create and Upload Tar: Starting. The rest of the output (the output of the command itself and the debugging around it from Backupsss::Tar.valid_exit? as well as the upload and cleanup) doesn't show up in the logs until the next run.
Note the change in the hour of the timestamp after the "Create and Upload Tar: Starting" line...
20170207T120900+0000 /jenkins-master-backupsss backups/1486242540.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486238940.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486235340.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486231740.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486228140.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486224540.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486220940.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486217340.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486213740.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486210140.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486206540.tar
20170207T120900+0000 /jenkins-master-backupsss Cleaning up backups/1486206540.tar
20170207T120900+0000 /jenkins-master-backupsss Finished cleaning up.
20170207T120900+0000 /jenkins-master-backupsss Create and Upload Tar: Starting
20170207T130900+0000 /jenkins-master-backupsss command.......tar -zcvf
20170207T130900+0000 /jenkins-master-backupsss stderr........tar: Removing leading `/' from member names
20170207T130900+0000 /jenkins-master-backupsss tar: Removing leading `/' from hard link targets
20170207T130900+0000 /jenkins-master-backupsss
20170207T130900+0000 /jenkins-master-backupsss status........pid 4000 exit 0
20170207T130900+0000 /jenkins-master-backupsss exit code.....0
20170207T130900+0000 /jenkins-master-backupsss Checking backup size ...
20170207T130900+0000 /jenkins-master-backupsss Size of backup is greater than 100MB
20170207T130900+0000 /jenkins-master-backupsss Creating a multipart upload
20170207T130900+0000 /jenkins-master-backupsss Starting multipart upload at 2017-02-07 12:09:32 +0000
20170207T130900+0000 /jenkins-master-backupsss Uploading backup as 5 parts
20170207T130900+0000 /jenkins-master-backupsss Uploading part number 1 : <upload ID>
... (more uploading)
20170207T130900+0000 /jenkins-master-backupsss Completed multipart upload at 2017-02-07 12:09:41 +0000
20170207T130900+0000 /jenkins-master-backupsss Completed in 0.16 minutes.
20170207T130900+0000 /jenkins-master-backupsss Create and Upload Tar: Finished
20170207T130900+0000 /jenkins-master-backupsss Found garbage...
20170207T130900+0000 /jenkins-master-backupsss 1486469340.tar
20170207T130900+0000 /jenkins-master-backupsss Cleaning up 1486469340.tar
20170207T130900+0000 /jenkins-master-backupsss Finished cleaning up.
The "Completed in 0.16 minutes" timestamp confirms that this is an output problem.
tar: /var/jenkins_home: file changed as we read it
Handle multipart uploads
Ran a backup with this that generated an 8.4G tar file.
backup.rb push_file failed with: ERROR - backup failed: Your proposed upload exceeds the maximum allowed size
unhandled nil check_tar_result
Create and Upload Tar: Starting
tar command stderr:
tar: Removing leading `/' from member names
tar (child): /backup/1483470540.tar: Cannot open: No space left on device
tar (child): Error is not recoverable: exiting now
ERROR - backup failed: undefined method `nonzero?' for nil:NilClass
backupsss-0.2.0/lib/backupsss/tar.rb:23:in `check_tar_result'
backupsss-0.2.0/lib/backupsss/tar.rb:18:in `make'
Looks like in check_tar_result we're doing if status.exitstatus.nonzero?, but it's possible for a failed run to return a nil exitstatus. We need to check for that.
While we're in that section of the code, it's probably useful (for errors like the above) to log the full tar command we're running before it's run.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.