manheim / backupsss Goto Github PK
View Code? Open in Web Editor NEWA gem for archiving data and pushing it to S3.
License: MIT License
A gem for archiving data and pushing it to S3.
License: MIT License
We're building backupsss into a docker container like:
FROM ubuntu:latest
RUN apt-get update && apt-get install -y ruby
RUN gem install --no-ri --no-rdoc backupsss -v 0.3.1
(which gives ruby 2.3.1)
And running with (simplified): docker run <container name> backupsss
.
However, when the cron timer ticks and the backups run, we see output from everything up to Create and Upload Tar: Starting
. The rest of the output (the output of the command itself and the debugging around it from Backupsss::Tar.valid_exit? as well as the upload and cleanup) doesn't show up in the logs until the next run.
Note the change in the hour of the timestamp after the "Create and Upload Tar: Starting" line...
20170207T120900+0000 /jenkins-master-backupsss backups/1486242540.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486238940.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486235340.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486231740.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486228140.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486224540.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486220940.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486217340.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486213740.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486210140.tar (retaining)
20170207T120900+0000 /jenkins-master-backupsss backups/1486206540.tar
20170207T120900+0000 /jenkins-master-backupsss Cleaning up backups/1486206540.tar
20170207T120900+0000 /jenkins-master-backupsss Finished cleaning up.
20170207T120900+0000 /jenkins-master-backupsss Create and Upload Tar: Starting
20170207T130900+0000 /jenkins-master-backupsss command.......tar -zcvf
20170207T130900+0000 /jenkins-master-backupsss stderr........tar: Removing leading `/' from member names
20170207T130900+0000 /jenkins-master-backupsss tar: Removing leading `/' from hard link targets
20170207T130900+0000 /jenkins-master-backupsss
20170207T130900+0000 /jenkins-master-backupsss status........pid 4000 exit 0
20170207T130900+0000 /jenkins-master-backupsss exit code.....0
20170207T130900+0000 /jenkins-master-backupsss Checking backup size ...
20170207T130900+0000 /jenkins-master-backupsss Size of backup is greater than 100MB
20170207T130900+0000 /jenkins-master-backupsss Creating a multipart upload
20170207T130900+0000 /jenkins-master-backupsss Starting multipart upload at 2017-02-07 12:09:32 +0000
20170207T130900+0000 /jenkins-master-backupsss Uploading backup as 5 parts
20170207T130900+0000 /jenkins-master-backupsss Uploading part number 1 : <upload ID>
... (more uploading)
20170207T130900+0000 /jenkins-master-backupsss Completed multipart upload at 2017-02-07 12:09:41 +0000
20170207T130900+0000 /jenkins-master-backupsss Completed in 0.16 minutes.
20170207T130900+0000 /jenkins-master-backupsss Create and Upload Tar: Finished
20170207T130900+0000 /jenkins-master-backupsss Found garbage...
20170207T130900+0000 /jenkins-master-backupsss 1486469340.tar
20170207T130900+0000 /jenkins-master-backupsss Cleaning up 1486469340.tar
20170207T130900+0000 /jenkins-master-backupsss Finished cleaning up.
The "Completed in 0.16 minutes" timestamp confirms that this is an output problem.
when choosing to compress output, the file produced is .tar
it should be either .tgz or .tar.gz
Ran a backup with this that generated an 8.4G tar file.
backup.rb push_file
failed with: ERROR - backup failed: Your proposed upload exceeds the maximum allowed size
Create and Upload Tar: Starting
tar command stderr:
tar: Removing leading `/' from member names
tar (child): /backup/1483470540.tar: Cannot open: No space left on device
tar (child): Error is not recoverable: exiting now
ERROR - backup failed: undefined method `nonzero?' for nil:NilClass
backupsss-0.2.0/lib/backupsss/tar.rb:23:in `check_tar_result'
backupsss-0.2.0/lib/backupsss/tar.rb:18:in `make'
Looks like in check_tar_result we're doing if status.exitstatus.nonzero?
, but it's possible for a failed run to return a nil exitstatus
. We need to check for that.
While we're in that section of the code, it's probably useful (for errors like the above) to log the full tar command we're running before it's run.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.