Giter Club home page Giter Club logo

zmbkpose's Introduction

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.


############################# Zmbkpose ################################

The zmbkpose tool is a shell script that does hot backup and hot restore of ZCS
Opensource accounts, GPL(OPENSOURCE). It can run from any host in the net, 
which means that it can be set on a backup server already existent.

It was concepted after Zimbra released ZCS 5.0.12, where a new import/export 
feature was exposed to user can be able to do his archiving. From that point 
it was clear for me that I just need to write a code to automatically get all
user data from server, remotely via HTTP SOAP. Plus LDAP data. So, it is 
possible to run full backups and restore even deleted accounts.

Till now, zmbkpose provides

    Full backup of all accounts;
    Full backup of any account;
    Incremental backup of all accounts;
    Incremental backup of any account;
    Lists all backup sessions done;
    Restore all contents (mail, contacts, appointments...) to any account;
    Restore deleted accounts back to directory and all of its contents;
    Restore only account attributes, like password, class of service;
    Configurable full simultaneously backup; 

This tool DOES NOT DO disaster recovery. 


### History
Created
			By Rubens Alonso Filho
				<[email protected]>
				<[email protected]>

Upgrated
26/10/2010 - Version 1.0.5
			By Alan Nikitiuk Milani
				<[email protected]>
				<[email protected]>
			By Bruno Gurgel
				<[email protected]>
				<[email protected]>
Published
			By Bruno Gurgel
				<[email protected]>

Upgraded
24/05/2012 - Version 2.0 Beta 
			By William Felipe Welter
                                <[email protected]>
                                <[email protected]>
	-Added functions:  
	backup_distribution_list()
	restore_distribution_list()
	backup_alias()
	restore_alias()
	backup_accounts_parallel()  (backup specific accounts parallel)

	-Modified: 
	backup_all_accounts_parallel()  (added backup trash and junk folders)
	incremental_all_accounts()  (doesnt create another full directory if new accounts are created after the last full backup)

	-Renamed: 
	DeusMeAjude -> restore_all_accounts()  (Restore all accounts parallel)


	-Bug corrections:  
	Wrong variables names references after translate to english

Upgraded
<<<<<<< HEAD
31/01/2012 - Version 2.1 Beta
			By William Felipe Welter
				<[email protected]>
				<[email protected]>
	-Added functions:
	restore_on_box() - Restore backup on other specified account. 
	notify_email() - Send a report to email address specified on the option EMAIL_NOTIFY in "zmbkpose.conf"
	
	-Bug corrections
	Individual account full backup in paralel mode
=======
25/07/2012 - Version 2.0 Beta 2
			By Diego Alencar Alves de Lima
				<[email protected]>
				<[email protected]>
	- New installer script

	- Modified:
	zmbkpose.conf now uses paralel backup capability by default
>>>>>>> 8dae5174f7f7414b54f1511be456cf74d630f31b

			 
#######################################################################

************For installation instructions please read the INSTALL file.

zmbkpose's People

Contributors

bggo avatar diegolima avatar linuxfabiano avatar rafaelgou avatar wfelipew avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

zmbkpose's Issues

Restauring from FULL curl: option --data-binary: out of memory

Hi Guys,

I have a small problem :) When i try to restore mailbox from my full backup (~3GB ) i receiving this error:

2014-02-23 21:45:55 DEBUG: [email protected] - Restauring from 23/02/2014 21:08:03 FULL
curl: option --data-binary: out of memory
curl: try 'curl --help' or 'curl --manual' for more information

enybody has sollution for this?

p.s. when i try to make restore from another user (mailbox 200mb ) its working without any problems

Old backups remove not working if the file name ends with 60 sec

When running this command: /usr/local/bin/zmbkpose -d 2d in order to delete backups older than 2 days, the command is not working if in the file name, we have 60 for seconds: example 201418081460.tar.gz because the data with 60 sec does not exist.

Could you please fix this issue? Thank you.

zmbkpose restore without sessions.txt

Alguien hizo una copia de todas las cuentas pero el archivos sessions.txt fue borrado, como puedo restaurar manualmente o alguna forma sin este archivo, varias cuentas tiene FULL e INC

zmbkpose --restoreAllAccounts

Hola antes de todo gracias por este scrip que ayuda bastante a gente como yo

Tengo el siguiente problema cuando quiero restaurar el full backup me sale este error por cuenta

HTTP ERROR 500

Problem accessing /service/home/[email protected]/Trash. Reason:

    system failure: tgz formatter failure


Powered by Jetty://

Quisiera saber si estoy haciendo algo mal al tratar de recuperar un backup gracias

Man Page - Pt_BR

Create the man page covering all features in Brazillian portuguese

zmbkpose resaurar

Olá,

estou com problemas para restaurar um aquivo de backup.

o arquivo comprimido tem 9.9G

servidor CentOS 5
zimbra 7.2

em uma maquina com um giga de ram da o erro:

ambkpose -r [email protected] full-20120522174726

curl: (27) out of memory

em uma maquina com 8G de ram da o erro:
zmbkpose -r [email protected] full-20120522174726


com.zimbra.cs.service.formatter.FormatterServiceException: Unexpected end of ZLIB input stream
ExceptionId:btpool0-8://servidor.dominio.org.br:7071/home/[email protected]/?fmt=tgz:1342044883099:d489729fdc4c9d64
Code:formatter.UNKNOWN_ERROR

Obrigado,

accounts restore

aliases and distributions lists are not restored with zmbkpose -restoreAccount

Using issue34 on ZCS 8.0.3

Hello,

I'm using the latest version of Zmbkpose (issue34) but i'm having a problem with it. No matter if I make full or incremental backup constantly the script is hanging with the following reason:

2013-05-16 10:32:22 DEBUG: [email protected] - Last backup from 09/05/2013 10:26:50 ...
2013-05-16 10:32:22 DEBUG: [email protected] - START INC backup ...
2013-05-16 10:32:22 DEBUG: [email protected] - Doing ldap backup ...
2013-05-16 10:32:22 DEBUG: [email protected] - ldap backup complete
2013-05-16 10:32:22 DEBUG: [email protected] - Doing mailbox backup ...
curl: (35) SSL connect error

Anyone knows anything about this issue?

Push the rewrite

Liandros wrote the code from scratch.

I just read it, need's someone to test.

Backup multi mailbox servers

Hi, I'm new using zmbkpose, and i'm having an issue. I have 3 mailbox servers and 2 remote ldap, and when i try to make a backup there is not any errors, just size backup is 0, when the account has 413 mb. In /etc/zmbkpose/zmbkpose.conf, ldapmasterserver is 192.168.X.Z, the mailboxes are 192.168.X.A, X.B, X.C. How can i make the backup. I run the zmbkpose from an external server, 192.168.X.D, with a NAS-NFS on my mnt as WORKDIR. If a run zmbkpose with ALL (LDAP, MTA, MAILBOX) just in the same server, runs fine, but with an ldap apart, and 2 or 3 mailboxes, looks like doesn't makes backups, just size 0. Thanks in advanced...

zmbkpose backups apparently only Inbox Sent Trash and Junk folders

Hello,

i have never used zmbkpose but i took a look at the src code cause i needed some tips for my scripts.

i noticed that yours queries to the zimbra mailbox are those three:

https://$MAILHOST:7071/home/$MAIL/?fmt=tgz
https://$MAILHOST:7071/home/$MAIL/Junk?fmt=tgz
https://$MAILHOST:7071/home/$MAIL/Trash?fmt=tgz

the first one export only Sent and Inbox, the last two Junk and Trash.
so what about Drafts folder? and every custom folder which could be created?

that can be solved with only one query query instead of four or one for each custom folder, here it is:

https://$MAILHOST:7071/home/$MAIL/Trash?fmt=tgz%20is:anywhere

hope that helped.

Script does not detect wrong password

Zmbkpose should capture authentication failure errors. Currently it just goes on creating empty archives. which is dangerous and if one fails to notice it, the backup rotation job will wipe out all valid past backups.

Requirement.

  1. Detect password failure while connecting zimbra server
  2. Send notification to an email-id about failed backup
  3. Log to syslog about the error.

ERROR:You don't have "gawk" command, or it is not in your PATH

Hi,

I just setup new install ubuntu server ver 12.04 , then install zimbra, everything is working good. sending / receiving email. now i want to install zmbkpose backup (i did this already 12 months ago on ubuntu 10, i use this same instruction and backup working OK.)
Now with ubuntu 12.04 ,im go thru readme file from revrite proposal.i think :) all files in on right places
but when i try to run backup manually i have error: ERROR:You don't have "gawk" command, or it is not in your PATH,

what can be wrong?

replace $(wich ldapsearch) with a variable

hi,

in my case i had some troubles with cronjob and $(wich ldapsearch) because it don't found ldapsearch so i replaced it with the variable: LDAPSEARCH=/opt/zimbra/bin/ldapsearch

greetings
gregor

curl: (35) SSL connect error

Hello,

with the latest version of the program, rewrite_proposal_v2.0 after issue #51 , appears the error in object.

My environment has installed the following programs:
Zimbra 8.0.6
libcurl-7.19.7-37.el6_4.x86_64
curl-7.19.7-37.el6_4.x86_64
openssl-1.0.1e-16.el6_5.7.x86_64

Operating System:
CentOS release 6.5 (Final)

During backups, both full and incremental, the error appears.
Never occurs during the backup of the same user, but it happens randomly.

Here the log backup of this night:

[root@mail backup]# tail backup_INC_20140624.log
2014-06-24 02:02:22 DEBUG: user1@domain1 - generated tar file /mnt/backup/user1@domain1/20140624020222:INC.tar
2014-06-24 02:02:22 DEBUG: user1@domain1 - backup size is 0
2014-06-24 02:02:22 DEBUG: user1@domain1 - END in 0secs,
2014-06-24 02:02:22 DEBUG: user2@domain2 - Last backup from 20/06/2014 02:00:38 ...
2014-06-24 02:02:22 DEBUG: user2@domain2 - START INC backup ...
2014-06-24 02:02:22 DEBUG: user2@domain2 - Doing ldap backup ...
2014-06-24 02:02:22 DEBUG: user2@domain2 - ldap backup complete
2014-06-24 02:02:22 DEBUG: user2@domain2 - Doing mailbox backup ...
2014-06-24 02:02:22 DEBUG: user2@domain2 - Doing mailbox backup ...
curl: (35) SSL connect error
[root@mail backup]#

When the error appears in the script exits and missing user are not saved.

In the zmbkpose.conf I disabled the parallel backups.

Launch the script command as follows:

su - zimbra-c "/usr/local/bin/zmbkpose -i" >> $ LOG 2> & 1

or

su - zimbra-c "/usr/local/bin/zmbkpose -f" >> $ LOG 2> & 1

No mailbox.tgz in incremental tar files

Hi,

I have an issue with rewrite_proposal_2.0 of Zmbkpose.

This issue was mentioned in issue #33 by ymarinov but it was not answered as far as I can see.
When incremental files are created, if there are no new mails, calendar or any other entries for the specific user, there will be no mailbox.tgz file in the :INC.tar file. This is not an error per se, but when you try to restore that specific user, an error is thrown and the restore stops. The error is in the form:

ERROR: file mailbox.tgz does not exists in tar file "/opt/zimbra/backup/[email protected]/20140222215236:INC.tar"

NOTE: I changed the domain to mydomain.com for privacy purposes.

The error comes from the code in the restore_account() function, namely this part:

#Restore
        if ! $TAR_cmd -tf "$tar_bkfile" $MAILBOX_FILE_IN_TAR >/dev/null 2>&1;then
            error "file $MAILBOX_FILE_IN_TAR does not exists in tar file \"$tar_bkfile\""
            return 1
        fi
        if [ -z "$SIMULATE" ] ;then
            $TAR_cmd -O -xf "$tar_bkfile" $MAILBOX_FILE_IN_TAR| \
                zimbra_mailbox_restore "$account" "$mailhost" 
        fi 

I changed this code a little bit so that it doesn't stop on error, but just skip. You could also add the check for the -u switch, although this is not an error, this is just the way the incremental files are created. The changed code looks like this:

     #Restore
                if ! $TAR_cmd -tf "$tar_bkfile" $MAILBOX_FILE_IN_TAR >/dev/null 2>&1;then
                        error "file $MAILBOX_FILE_IN_TAR does not exists in tar file \"$tar_bkfile\"
                elif [ -z "$SIMULATE" ] ;then
                                $TAR_cmd -O -xf "$tar_bkfile" $MAILBOX_FILE_IN_TAR| \
                                        zimbra_mailbox_restore "$account" "$mailhost"
                fi        

I am not that good at bash scripting, so I am sure that this can be improved. Hope this helps someone else.

BTW, this is an excellent project.

Thank you for all your efforts and time and keep up the good work!

Create a deb package

Create a Debian package to upload into Debian repositories, so, follow the Debian-rules Debian-Policy

rewrite_proposal_v2.0 branch : curl error

Hello,

during the execution of the program find a number of these errors, you know, tell me if they are to be ignored?


/bin/tar: mailbox.tgz: file changed as we read it
/bin/tar: mailbox.tgz: file changed as we read it
rm: cannot remove `/mnt/backup/1': Directory not empty
rm: cannot remove `/mnt/backup/1/.nfs0000000002f800bf00000069': Device or resource busy
2014-04-29 12:30:29 DEBUG: user1@domain1 - mailbox backup complete 
2014-04-29 12:30:29 DEBUG: user1@domain1 - generated tar file /mnt/backup/user1@domain1/20140429123021:FULL.tar 
2014-04-29 12:30:29 DEBUG: user1@domain1 - backup size is 0 
2014-04-29 12:30:29 DEBUG: user1@domain1 - END in 8secs, 
mkdir: cannot create directory `/mnt/backup/1': File exists
mkdir: cannot create directory `/mnt/backup/1': File exists
2014-04-29 12:30:31 DEBUG: user2@domain2 - START FULL backup ... 
2014-04-29 12:30:31 DEBUG: user2@domain2 - Doing ldap backup ... 
2014-04-29 12:30:31 DEBUG: user3@domain3 - START FULL backup ... 
2014-04-29 12:30:31 DEBUG: user3@domain3 - Doing ldap backup ... 
2014-04-29 12:30:31 DEBUG: user4@domain4 - START FULL backup ... 
2014-04-29 12:30:31 DEBUG: user4@domain4 - Doing ldap backup ... 
2014-04-29 12:30:31 DEBUG: user2@domain2 - ldap backup complete 
2014-04-29 12:30:31 DEBUG: user2@domain2 - Doing mailbox backup ... 
2014-04-29 12:30:31 DEBUG: user2@domain2 - No a mail account. Skipping mailbox backup. 
2014-04-29 12:30:31 DEBUG: user3@domain3 - ldap backup complete 
2014-04-29 12:30:31 DEBUG: user3@domain3 - Doing mailbox backup ... 
2014-04-29 12:30:31 DEBUG: user2@domain2 - generated tar file /mnt/backup/user2@domain2/20140429123031:FULL.tar 
2014-04-29 12:30:31 DEBUG: user4@domain4 - ldap backup complete 
2014-04-29 12:30:31 DEBUG: user2@domain2 - backup size is 0 
2014-04-29 12:30:31 DEBUG: user4@domain4 - Doing mailbox backup ... 
2014-04-29 12:30:31 DEBUG: user3@domain3 - Doing mailbox backup ... 
2014-04-29 12:30:31 DEBUG: user2@domain2 - END in 0secs, 
cat: /mnt/backup/1/acct.ldif: No such file or directory
2014-04-29 12:30:31 DEBUG: user4@domain4 - No zimbraMailHost ldap attribute. 
curl: (23) Failed writing body (0 != 1)
mkdir: cannot create directory `/mnt/backup/1': File exists
mkdir: cannot create directory `/mnt/backup/1': File exists

"zmbkpose -l" don't show anything

hi,

when i try listening the backups from the session file i get nothing

fix: line 295
replace: "iniciada" with "started"
system: centos 5
locale: LANG=en_US.UTF-8

btw: THANKS for this script :-)

greetings from austria
gregor

I would like to collaborate

I made ​​some modifications, and I would like to collaborate with the proyect. How I can to continue?

Cases wiki page

Important page the can show for new users some people using zmbkpose.

I'll start the page, asking 4Linux for some numbers

Out of memory issue

When I try to run zmbkpose -f I keep getting a "DNS SRV: out of memory?" error and i cannot proceed with the backup. Is there any reason I would be getting this? I have 4GB of RAM so I wouldn't think that would be the issue.

Create a rpm package

Needs someone who understant how we can upload this into Fedora/RH/Centos repositories, Suse and Mandriva packages are wellcome too.

Zmbkpose -r incremental problem

Hola tengo el siguiente problema cuando quiero restaurar un incremental tengo el siguiente problema primero detallo como lo estoy haciendo.
Utilizo el comando
zmbkpose -r [email protected] inc-20130102010002
luego de esto me da los siguientes errores.

<title>Error 401 must authenticate</title>

HTTP ERROR 401

Problem accessing /service/home/[email protected]/Junk. Reason:

    must authenticate


Powered by Jetty://

Alguna sugerencia

Suggest installation of gawk in ubuntu

When doing a -l on Ubuntu some errors are thrown for gensub and other awk functions.
Installing gawk solves the problem, so it would be nice to suggest the installation of this package.

I'm using rewrite_proposal

zmbkpose -f backing up all mail exept the last created account

Hi all,

I use zmbkpose -f and it backups all accounts except for the last created account. I deleted the last account but then when i perform again a full backup it lacks again to backup the account created before the last created account. I get an error at the end of the script.

curl: (6) Could not resolve host: ; Name or service not known

Can someone help me please??

Tanx in advance

Man Page - EN

Create the manpage covering all functions in ENGLISH

zmbkpose --RestoreonAccount option

hi,

i am verry pleased with the rewrite pruposal but in the master branche there is a restore option called --RestoreonAccount. This option can be used to restore an account to an other account name. i mis this option in the rewrite proposal.

Will this ever be build inside the rewrite pruposal? Now whenever i restore the acoount i have to clean my hole mailbox again, because i only need 1 mail from my restore?

greetings chris

Translate source code comments

On the zmbkpose source, we have a lot of util comments. Many of the information there need to be translated to english, and maybe the extra information need to be moved to the man pages.

"zmbkpose -d" Do not exclude old full backups.

In a cron job I configure to zmbkpose exclude backups older than 15 days. But I noticed that just the incremental backups were excluded and the full backups still there. This behaviour is normal?

Backup zmbkpose Permission Denied

Quando executo o comando para efetuar o backup ele começa a efetuar o backup, no entanto após alguns minutos lendo a mailbox ele me apresenta o seguinte erro:

postdrop: warning: mail_queue_enter: create file maildrop/39119.13181: Permission denied
postdrop: warning: mail_queue_enter: create file maildrop/39378.13181: Permission denied
postdrop: warning: mail_queue_enter: create file maildrop/39625.13181: Permission denied
postdrop: warning: mail_queue_enter: create file maildrop/40184.13181: Permission denied
postdrop: warning: mail_queue_enter: create file maildrop/40411.13181: Permission denied
postdrop: warning: mail_queue_enter: create file maildrop/40642.13181: Permission denied
postdrop: warning: mail_queue_enter: create file maildrop/40851.13181: Permission denied

E vai crescendo. Alguém pode me ajudar?

modifications in zmbkpose zimbra 8

can any one suggest what are the changes in zmbkpose file for zmbra8.when iam trying to restore the data in zimbra 8 im getiing duplicate mails.what are the changes in zmbkpose file?

zmbkpose rewrite proposal

This is a complete rewrite proposal to zmbkpose.
I present this to the team members for feedback before attempting to upload.
* Full rewrite of source code
* New storage format for backups, incomppatible with older versions
* More flexibility, simplicity backup storage, and performance
* Conditional incremental or full backup(how old is the last backup)
* List information about backups
* Restore the account to a specific time

What do you think?
The code is in https://docs.google.com/open?id=0B9IDNSyRLvcBdVN4cWd6U2ZQUWc

halts on curl error

even if launched with -u switch, the backup is halted on curl error. I got this result 👍 curl: (23) Failed writing body
and the backup stopped...
Using proposed rewrite branch

/usr/local/bin/zmbkpose: line 161: -x: command not found

Hi Guys,

If i try to run cron from zimbra i have error: (Execute cron job aa ZIMBRA user)

Output from command zmbkpose -f ..

Running the parallel backup
/usr/local/bin/zmbkpose: line 161: -x: command not found

p.s. if i use su - zimbra and than i type: zmbkpose -f backup was create successful.

Thx For help

Arek

restoring issues in zimbra8

when iam migrating my server from zimbra7 to zimbra8,im facing problems in restoration.if im restoring the full backup for 3 times,my mailbox size also increased by 3 times(if it is 10MB after restoration for 3 times it is 30MB).can any one suggest what are the changes in zmbkpose file or any zimbra8 compatable zmbkpose?

zmbkpose --backupDistributionList not working

I have Installed rewrite_proposal_v2.0 but in this i can not take backup of zimbra distribution List. I have tried with all command. I have checked zmbkpose file so found there is no any mentioned command like this zmbkpose --backupDistributionList. I have tried using both master and rewrite_proposal_v2.0 so it is working on master but it is not working onrewrite_proposal_v2.0. wo we can not take backup of Zimbra distribution List.

Error while restoring

Hello,

I'm using currently the rewrite_proposal_2.0 of Zmbkpose. I'm replication our main mail server, and I'm having an issue with it. Please find the exact log below, and if you have anything to suggest will be glad to hear from you. Have in mind that no one killed the session - it's saying killed ...

2013-03-05 12:32:50 DEBUG: [email protected] - Restauring from 03/03/2013 05:14:03 FULL
/usr/local/bin/zmbkpose: line 333: 2459 Killed $CURL_cmd $CURL_args --data-binary "@$backup_file" -u $ADMINUSER:$ADMINPASS "https://$mailhost:7071/home/$mailbox/?fmt=tgz&resolve=skip"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.