jarus / imap_copy Goto Github PK

View Code? Open in Web Editor NEW

82.0 14.0 42.0 21 KB

Simple tool to copy folders from one IMAP server to another server.

License: BSD 3-Clause "New" or "Revised" License

Python 100.00%

imap_copy's Introduction

IMAP Copy

This is a very simple tool to copy folders from one IMAP server to another server.

Example:

The example below copies all messages from the INBOX of your other server into the 'OTHER-SERVER/Inbox' folder of Gmail.

python imapcopy.py "imap.otherserver.com.au:993" "username:password" \
"imap.googlemail.com:993" "[email protected]:password" \
"INBOX" "OTHER-SERVER/Inbox" --verbose

Since Gmail terribly throttles uploading and downloading mails over IMAP, you may find the 'skip' and 'limit' options handy. If Gmail disconnected you after copying 123 emails out of your total 1000 emails in the example shown above, you may use the following command to resume copying skipping the first 123 messages.

python imapcopy.py "imap.otherserver.com.au:993" "username:password" \
"imap.googlemail.com:993" "[email protected]:password" \
"INBOX" "OTHER-SERVER/Inbox" --skip 123

Similarly the 'limit' option allows you to copy only the N number of messages excluding the skipped messages. For example, the following command will copy message no. 124 to 223 into Gmail.

python imapcopy.py "imap.otherserver.com.au:993" "username:password" \
"imap.googlemail.com:993" "[email protected]:password" \
"INBOX" "OTHER-SERVER/Inbox" --skip 123 --limit 100

There is also 'recurse' option that copies contents of folders with all of its subfolders. Also if you replace source mailbox with empty string, it will copy all contents of that mailbox:

python imapcopy.py "imap.otherserver.com.au:993" "username:password" \
"imap.googlemail.com:993" "[email protected]:password" \
"" "OTHER-SERVER" --recurse

Usage:

usage: imapcopy.py [-h] [-q] [-v]
               source source-auth destination destination-auth mailboxes
               [mailboxes ...]

positional arguments:
  source            Source host ex. imap.googlemail.com:993
  source-auth       Source host authentication ex. [email protected]:password
  destination       Destination host ex. imap.otherhoster.com:993
  destination-auth  Destination host authentication ex.
                    [email protected]:password
  mailboxes         List of mailboxes alternate between source mailbox and
                    destination mailbox.

optional arguments:
  -h, --help        show this help message and exit
  -c, --create-mailboxes
                    Create the mailboxes on destination
  -r, --recurse     Recurse into submailboxes
  -q, --quiet       ppsssh... be quiet. (no output)
  -v, --verbose     more output please (debug level)
  -s N, --skip N    skip the first N message(s)
  -l N, --limit N   only copy N number of message(s)

Only tested on Python 2.7.

imap_copy's People

Contributors

Stargazers

Watchers

imap_copy's Issues

Doesn't correctly handle servers with different separators

The code has a bug which causes it to use the same mailbox separator on both servers. Some servers use different separators. For example, one server might use '.' so the path would be 'INBOX.Spam' while another might use '/' making the path 'INBOX/Spam'.

The solution is to capture the separator individually for each server. I will create a pull request with a fix.

Error when cloning one yandex mail account to another

Hello!

I've faced with next problem. I want to recursively copy all messages from one mailbox to another. Both mails are located on YANDEX mail server. I use next command and get next output:

# python imapcopy2.py -r -c imap.yandex.ru:993 [email protected]:PASSWORD imap.yandex.ru:993 [email protected]:PASSWORD "" ""
2016-05-31 11:29:14,356 - INFO - Connect to source (imap.yandex.ru)
2016-05-31 11:29:14,965 - INFO - Authenticate at source
2016-05-31 11:29:15,062 - INFO - source connection established
2016-05-31 11:29:15,289 - INFO - Connect to destination (imap.yandex.ru)
2016-05-31 11:29:15,855 - INFO - Authenticate at destination
2016-05-31 11:29:16,005 - INFO - destination connection established
2016-05-31 11:29:16,245 - INFO - Getting list of mailboxes under 
2016-05-31 11:29:16,470 - INFO - Recursing into &BB4EQgQ,BEAEMAQyBDsENQQ9BD0ESwQ1-
2016-05-31 11:29:16,470 - INFO - Getting list of mailboxes under &BB4EQgQ,BEAEMAQyBDsENQQ9BD0ESwQ1-
2016-05-31 11:29:16,862 - INFO - Create destination mailbox |&BB4EQgQ,BEAEMAQyBDsENQQ9BD0ESwQ1-
2016-05-31 11:29:17,231 - INFO - Looking for mails in &BB4EQgQ,BEAEMAQyBDsENQQ9BD0ESwQ1-
2016-05-31 11:29:17,495 - INFO - Start copy &BB4EQgQ,BEAEMAQyBDsENQQ9BD0ESwQ1- => |&BB4EQgQ,BEAEMAQyBDsENQQ9BD0ESwQ1- (180 mails)
2016-05-31 11:29:18,046 - INFO - Copy mail 1 of 180 (copy_count=1, md5(message)=915fa7e6ded9106af3fc906f715a75d9)
2016-05-31 11:29:18,485 - INFO - Copy mail 2 of 180 (copy_count=2, md5(message)=eff927cff1aef0bf8af2f036ad04c968)
2016-05-31 11:29:18,944 - INFO - Copy mail 3 of 180 (copy_count=3, md5(message)=5780dc688ad4ead68e24dc1804885853)
2016-05-31 11:29:19,382 - INFO - Copy mail 4 of 180 (copy_count=4, md5(message)=e948f76ce9bfef6d0ced0f6d0beda919)
2016-05-31 11:29:19,794 - INFO - Copy mail 5 of 180 (copy_count=5, md5(message)=0cd7c9aea9fc3f8c94e4a398d7960cc7)
2016-05-31 11:29:20,249 - INFO - Copy mail 6 of 180 (copy_count=6, md5(message)=43b26808b1b154c403f57d07fc77ae0f)
2016-05-31 11:29:20,732 - INFO - Copy mail 7 of 180 (copy_count=7, md5(message)=d9857d59699b48ae8328ba0ad784e8e2)
2016-05-31 11:29:21,587 - INFO - Copy mail 8 of 180 (copy_count=8, md5(message)=72dd4dd5c85d6ac1fe98c90b93228425)
2016-05-31 11:29:22,046 - INFO - Copy mail 9 of 180 (copy_count=9, md5(message)=dc5f9ba1421f6ca89a2e6de05190ea04)
2016-05-31 11:29:22,464 - INFO - Copy mail 10 of 180 (copy_count=10, md5(message)=248a7e8f5bcbb5ed42868035fcf90b26)
2016-05-31 11:29:22,902 - INFO - Copy mail 11 of 180 (copy_count=11, md5(message)=9cad4c67446d502e8c773c83a2618425)
2016-05-31 11:29:23,322 - INFO - Copy mail 12 of 180 (copy_count=12, md5(message)=c39b1bc89cd95d33b6395ca763f4c49a)
2016-05-31 11:29:23,750 - INFO - Copy mail 13 of 180 (copy_count=13, md5(message)=78ea52b0dcc1cd5a0ccef8aa170e27aa)
2016-05-31 11:29:24,208 - INFO - Copy mail 14 of 180 (copy_count=14, md5(message)=f9b51ea22ee441b392fff1167352cd87)
2016-05-31 11:29:24,637 - INFO - Copy mail 15 of 180 (copy_count=15, md5(message)=2870d5d1aaa606a38d0ad2de8e148dd7)
2016-05-31 11:29:25,083 - INFO - Copy mail 16 of 180 (copy_count=16, md5(message)=f2dd880dcc6012d64b34da8191548aef)
2016-05-31 11:29:26,238 - INFO - Copy mail 17 of 180 (copy_count=17, md5(message)=a34d02cd9a6759136080ff19a9895d23)
2016-05-31 11:29:26,691 - INFO - Copy mail 18 of 180 (copy_count=18, md5(message)=8fe1a29f9507de48a3634369f3773cbc)
2016-05-31 11:29:27,113 - INFO - Copy mail 19 of 180 (copy_count=19, md5(message)=8acf14e49bdf3ce8f3670b101b7ef8b6)
2016-05-31 11:29:27,529 - INFO - Copy mail 20 of 180 (copy_count=20, md5(message)=0f97d518a239ea19608d8d5d2b49af33)
2016-05-31 11:29:27,951 - INFO - Copy mail 21 of 180 (copy_count=21, md5(message)=52976ca5b7e97ae27798b413ac56c553)
2016-05-31 11:29:28,382 - INFO - Copy mail 22 of 180 (copy_count=22, md5(message)=1a46748406a013b236d43a168c9ab64f)
2016-05-31 11:29:28,879 - INFO - Copy mail 23 of 180 (copy_count=23, md5(message)=46f3f5a07092d3e17e70eb8d331b5c15)
2016-05-31 11:29:29,322 - INFO - Copy mail 24 of 180 (copy_count=24, md5(message)=36a01975540787f7e158da41d9c423dc)

2016-05-31 11:30:03,667 - INFO - Copy mail 25 of 180 (copy_count=25, md5(message)=f9d1a379f9389eacc5b7c0df51d540dd)
2016-05-31 11:30:04,262 - INFO - Copy mail 26 of 180 (copy_count=26, md5(message)=277233f3909c9db533322b833513d166)
2016-05-31 11:30:04,814 - INFO - Close mailbox on source
2016-05-31 11:30:04,814 - INFO - Disconnect from source server
2016-05-31 11:30:04,898 - INFO - Disconnect from destination server
Traceback (most recent call last):
  File "imapcopy2.py", line 266, in <module>
    main()
  File "imapcopy2.py", line 260, in main
    imap_copy.run()
  File "imapcopy2.py", line 179, in run
    self.copy(source_mailbox, destination_mailbox, self.skip, self.limit)
  File "imapcopy2.py", line 105, in copy
    skip, limit, recurse_level + 1)
  File "imapcopy2.py", line 158, in copy
    destination_mailbox, flags, msgDate, message
  File "/usr/local/lib/python2.7/imaplib.py", line 337, in append
    return self._simple_command(name, mailbox, flags, date_time)
  File "/usr/local/lib/python2.7/imaplib.py", line 1088, in _simple_command
    return self._command_complete(name, self._command(name, *args))
  File "/usr/local/lib/python2.7/imaplib.py", line 918, in _command_complete
    raise self.error('%s command error: %s %s' % (name, typ, data))
imaplib.error: APPEND command error: BAD ['Command syntax error. sc=HTHlNx5Ix8c8_310929_24o']

GMail > Gmail copy, weird timestamping

When making a copy GMail > Gmail, threads are dated to the moment of a copy; however date of individual messages is copied correctly.

Recursive folder copy.

Is it possible to copy a folder recursively?

Add --all-mailboxes option

With the --all--mailboxes option you should able to copy all mailboxes from the source to the destination.

Needed to change it a bit for me

Needed to change the line 100 to
new_source_mailbox = d.split('"')[2].lstrip()

Not sure why, but I am using a german server.

Python 2.7.6, WSL

crash report

Crashed during copy on destination server.
Your crash message suggests that I should report the problem. Here is what I can report.

Command line (anonymised) :

python imapcopy.py -c "imap.yyy.net:993" "a@b:c" "109.yyy.xxx.zz:993" "a@b:c" "INBOX" "INBOX/anciens" --recurse

Crash message :

2017-03-12 10:27:22,543 - INFO - Copy mail 999 of 1016 (copy_count=999, md5(message)=xxx...xxx)
2017-03-12 10:27:22,586 - INFO - Close mailbox on source
2017-03-12 10:27:22,587 - INFO - Disconnect from source server
2017-03-12 10:27:22,603 - INFO - Disconnect from destination server
Traceback (most recent call last):
  File "imapcopy.py", line 265, in <module>
    main()
  File "imapcopy.py", line 259, in main
    imap_copy.run()
  File "imapcopy.py", line 178, in run
    self.copy(source_mailbox, destination_mailbox, self.skip, self.limit)
  File "imapcopy.py", line 152, in copy
    flags = data[1][8:][:-2]  # Not perfect.. Waiting for bug reports
IndexError: list index out of range

Better handling of flags and timestamps

Here is a patch to better handle flags and timestamps

diff --git a/imapcopy.py b/imapcopy.py
index 3190a57..2c56a95 100644
--- a/imapcopy.py
+++ b/imapcopy.py
@@ -123,12 +123,13 @@ class IMAP_Copy(object):
                     progress_count, mail_count))
                 continue
             else:
-                status, data = self._conn_source.fetch(msg_num, '(RFC822 FLAGS)')
+                status, data = self._conn_source.fetch(msg_num, '(FLAGS INTERNALDATE BODY.PEEK[])')
                 message = data[0][1]
-                flags = data[1][8:][:-2]  # Not perfect.. Waiting for bug reports
+                flags = imaplib.ParseFlags(data[0][0])
+                date = imaplib.Time2Internaldate(imaplib.Internaldate2tuple(data[0][0]))

                 self._conn_destination.append(
-                    destination_mailbox, flags, None, message
+                    destination_mailbox, flags, date, message
                 )

                 copy_count += 1

Faster approach on high latency connections

I live in Australia which means I have a high latency to the US. I was recently migrating from a US server to Australia and the migration was somewhat slow.

It would be much faster on high latency connections to download a batch of messages or all messages.

Note that the list is comma separated, not space separated. When downloading multiple messages imaplib seems to have a bug where for each message there is a ')'. So msg[0] is the proper message and msg[1] contains ')'

msgnums = data[0].split()
In [58]: len(msgnums)
Out[58]: 426

In [17]: %time status, msgs = conn.fetch(",".msgnums), '(RFC822 FLAGS)')
CPU times: user 478 ms, sys: 743 ms, total: 1.22 s
Wall time: 20.1 s

In [18]: len(msgs)
Out[18]: 852

Existing way:
%time single_msgs=[conn.fetch(msgnum, '(RFC822 FLAGS)') for msgnum in msgnums]
CPU times: user 617 ms, sys: 788 ms, total: 1.4 s
Wall time: 2min 51s

In [55]: len(single_msgs)
Out[55]: 426

Fails if source message has a negative hour in timestamp

Some messages have negative values in the hour field of the timestamp (presumably due to an issue with the originating server). This can be handled by checking for and correcting negative values. I will add a pull request to fix this issue.

jarus / imap_copy Goto Github PK

imap_copy's Introduction

IMAP Copy

imap_copy's People

Contributors

Stargazers

Watchers

Forkers

imap_copy's Issues

Recommend Projects

Recommend Topics

Recommend Org