![Thunderbird logo](https://assets-prod.sumo.prod.webservices.mozgcp.net/media/uploads/products/2024-02-12-07-36-53-599028_1.png)
Thunderbird upload over IMAP aborts
I am using TB 78.7.1 (64-bit) on Windows 10 to upload messages from a local folder to a Gmail folder via IMAP. I copy-drag 700+ messages, but only a dozen or so get copied each time before the process aborts. A windows pops up saying "Thunderbird: Connection to server imap.gmail.com timed out". Before the abortion, the TB status line will sometimes say "User.Name@gmail.com: Sending login information..."
It will take forever to copy messages this way, and some messages don't seem to make it to the destination.
I enabled TB logging using the environment variable settings at: https://www.mailenable.com/kb/content/article.asp?ID=ME020591
Since I am using Cygwin's bash, I do the following:
NSPR_LOG_MODULES=IMAP:4 \ NSPR_LOG_FILE="$(cygpath -am ~/tmp/TBirdLog)" \ cygstart thunderbird
This creates a file ~/tmp/TBirdLog.moz_log, whose final lines are ("FamBig" is the name of both the local and destination folder):
[(null) 3836: IMAP]: I/IMAP dC9tZWRpYS9pbWFnZTEwLnBuZ1BLAQIUABQACAgIAOO75FCHrtQYnAsAAJcLAAAVAAAAAAAAAAAA AAAAAPcSJwBwcHQvbWVkaWEvaW1hZ2UxMS5wbmdQSwECFAAUAAgICADju+RQsNYnhMUNAADADQAA FQAAAAAAAAAAAAAAAADWHicAcHB0L21lZGlhL2ltYWdlMTIucG5nUEsBAhQAFAAICAgA47vkULDW J4TFDQAAwA0AABUAAAAAAAAAAAAAAAAA3iwnAHBwdC9tZWRpYS9pbWFnZTEzLnBuZ1BLAQIUABQA CAgIAOO75FCw1ieExQ0AAMANAAAVAAAAAAAAAAAAAAAAAOY6JwBwcHQvbWVkaWEvaW1hZ2UxNC5w [(null) 3836: IMAP]: I/IMAP bmdQSwECFAAUAAgICADju+RQW2mP+lh8JQAAjSUAFgAAAAAAAAAAAAAAAADuSCcAcHB0L21lZGlh L2ltYWdlMTUuanBlZ1BLAQIUABQACAgIAOO75FDzq2EQQJonAFOkJwAWAAAAAAAAAAAAAAAAAIrF TABwcHQvbWVkaWEvaW1hZ2UxNi5qcGVnUEsBAhQAFAAICAgA47vkUIeu1BicCwAAlwsAABUAAAAA AAAAAAAAAAAADmB0AHBwdC9tZWRpYS9pbWFnZTE3LnBuZ1BLAQIUABQACAgIAOO75FCUk6JKQQIA AIEcAAATAAAAAAAAAAAAAAAAAO1rdABbQ29udGVudF9UeXBlc10ueG1sUEsFBgAAAAA8ADwApREA [(null) 3836: IMAP]: I/IMAP AG9udAAAAA==
------=_NextPart_000_015D_01D6523B.3D39BAF0--
[(null) 3836: IMAP]: I/IMAP 000002557C788000:imap.gmail.com:A:SendData: [(null) 3836: IMAP]: D/IMAP ReadNextLine [rv=0x804b000e stream=000002557F494EE0 nb=0 needmore=1] [(null) 3836: IMAP]: I/IMAP 000002557C788000:imap.gmail.com:A:CreateNewLineFromSocket: clearing IMAP_CONNECTION_IS_OPEN - rv = 804b000e [(null) 3836: IMAP]: I/IMAP 000002557C788000:imap.gmail.com:A:TellThreadToDie: close socket connection [(null) 3836: IMAP]: I/IMAP 000002557C788000:imap.gmail.com:A:CreateNewLineFromSocket: (null) [(null) 3836: IMAP]: D/IMAP SetConnectionStatus(0x804b000e) [(null) 3836: IMAP]: D/IMAP URL failed with code 0x804b000e (imap://User%2EName%40gmail%2Ecom@imap.gmail.com:993/appendmsgfromfile%3E/FamBig) [(null) 3836: IMAP]: I/IMAP 000002557C788000:imap.gmail.com:A:ProcessCurrentURL: aborting queued urls [(null) 3836: IMAP]: D/IMAP ImapThreadMainLoop leaving [this=000002557C788000] [(null) 3836: IMAP]: I/IMAP 000002558567C000:imap.gmail.com:S-INBOX:SendData: DONE [(null) 3836: IMAP]: I/IMAP 0000025585683000:imap.gmail.com:S-INBOX:SendData: DONE [(null) 3836: IMAP]: I/IMAP 0000025585689800:imap.gmail.com:S-INBOX:SendData: DONE [(null) 3836: IMAP]: I/IMAP 0000025585683000:imap.gmail.com:S-INBOX:SendData: 14 close [(null) 3836: IMAP]: I/IMAP 0000025585689800:imap.gmail.com:S-INBOX:SendData: 14 close [(null) 3836: IMAP]: I/IMAP 0000025585683000:imap.gmail.com:S-INBOX:SendData: 15 logout [(null) 3836: IMAP]: I/IMAP 0000025585689800:imap.gmail.com:S-INBOX:SendData: 15 logout [(null) 3836: IMAP]: I/IMAP 000002558567C000:imap.gmail.com:S-INBOX:SendData: 14 close [(null) 3836: IMAP]: I/IMAP 000002558567C000:imap.gmail.com:S-INBOX:SendData: 15 logout [(null) 3836: IMAP]: I/IMAP 0000025585689800:imap.gmail.com:S-INBOX:TellThreadToDie: close socket connection [(null) 3836: IMAP]: D/IMAP ImapThreadMainLoop leaving [this=0000025585689800] [(null) 3836: IMAP]: I/IMAP 0000025585683000:imap.gmail.com:S-INBOX:TellThreadToDie: close socket connection [(null) 3836: IMAP]: D/IMAP ImapThreadMainLoop leaving [this=0000025585683000] [(null) 3836: IMAP]: I/IMAP 000002558567C000:imap.gmail.com:S-INBOX:TellThreadToDie: close socket connection [(null) 3836: IMAP]: D/IMAP ImapThreadMainLoop leaving [this=000002558567C000]
Ausgewählte Lösung
Whilst stopping the 'real time protection' will stop a load of scanning which can cause delays and improve upload, there may also be an upload issue with gmail server. I did find this information which may be effecting you, but I'm not sure how many emails/ actual size in MB that you are trying to upload in total. Gmail has it's own limits on amount you can upload to server within a specified time period. So gmail will drop connection if limits are reached. Info here: https://support.google.com/a/answer/1071518?hl=en
quoted info from gmail Migrations or bulk operations using IMAP If you exceed the upload bandwidth limit with IMAP, the following recommendations are important:
- Avoid large copy and paste or drag and drop operations when using IMAP.
- If you're migrating, use a supported migration option and avoid using IMAP to upload messages.
- If you need to download a large amount of data, try throttling the tool you're using or download in parts.
Alle Antworten (11)
The bottom line is IMAP is not designed to be a file system, is is designed around distributed access to a central mail store with an ability to upload a few mails at a time to the imap sever as they are sent or on an adhock basis for other reasons.
The reality is we have folks doing what you are and finding bugs that in some cases have existed for decades. The current one https://bugzilla.mozilla.org/show_bug.cgi?id=1618455 will be fixed in a version of 78 soon. Perhaps a month from now and is probably what you log is showing.
There is also an addon that is in development. There is a link to it in this bug comment https://bugzilla.mozilla.org/show_bug.cgi?id=538375#c169
It might do the job for you as it copies whole folders of mail, not just individual emails.
Thanks, Matt. Not being an IT person, I was hesitant to jump into trying the addon, but what the hey, it never hurts to bloody one's nose. However, I'm not familiar with the ESR versioning concept, so I may just resort to my Outlook Desktop 2019 to do the transfer (meaning I download what I already downloaded into TB, then upload it to the desired destination).
ESR is Thunderbird.... back about 10 years ago when Mozilla wandered off, Thunderbird stopped the mad dash for the highest version number that could be found with major releases every 6 weeks. Instead opting for a once a year update in line with the Firefox ESR release cycle with regular security and stability fixes ( and outright bug fixes have crept in there over the past few years.
You could use the daily build, but it will not be compatible with your already created profile. So may create more issues than you want trying to go backwards. Or just wait a few weeks. Or as you say use outlook now. But do not know anything about outlook these days. But years ago when I last used it IMAP was a mess. It might have improved in the intervening years, but as Microsoft wants everyone on Exchange sync, I have my doubts that IMAP will have seen much love from the guys at Redmont.
Having paid about $160 local currency for Outlook 2019, I can say that it seems to work less well than 2013. It's hard to know what things will connect you with the cloud and slurp up all the info in Contacts and Notes. It seems to stall during email syncing over IMAP and leave zombie processes that prevent any further syncing, but with no indication that it is in this state. Restarting Outlook doesn't help unless you know enough to remove the zombie process via Task Manager. The Unread Mail indicator is always confused, i.e., shows unread messages even though there are none. Deleting messages from All Mail leaves messages in Inbox, even though All Mail is supposed to be a superset of Inbox. (This might be a problem on the Google side because folders are actually tags, so I'm not sure if problem exists with TB).
In other words, you're right about Outlook. That's why I started with TB for mail migration. But hey, now much damage can be done by trying? If I have time this weekend, I might give it a try.
Thanks. I will keep my eye on the link about the addon. If it gets folded into the main TB available to common folk, I'll give it a spin.
If you have Anti-virus scanning files it may contribute to a slow down. Suggest stop AV scanning opened files, so make your thunderbird profile folders exempt from scanning. Maybe your AV has real time protection scanning, suggest you try switching off whilst trying to upload and also upload smaller batches.
Thanks, Toad-Hall. I will Google that. Currently, I only use the native Windows AV. But hearing you refer to the cause as a slowdown, I should add that I'm migrating email conversation threads with big attachments. I assumed that large files get broken up into pieces that can be managed over IMAP without confusion about delays, but as I mentioned in an earlier reply, my expertise isn't really in IT, so I could very well be wrong about that.
TRy this info about real time protection - scroll to the bottom.
Switch it off for eg: an hour and try doing some uploading in that time to see if improves.
Geändert am
Thanks for the suggestion, Toad-Hall. I still got the same problem. Now it seems that I consistently get the message about lost connection to the server. I mentioned above that I am only migrating the conversation threads containing big messages, and the one that seems to repeatedly problematic at this point contains a 28.8MB attachment. I now cast my fate into the arms of Outlook 2019 desktop (shudder).
P.S. I took what speculative measures I could think of. Well, only one measure. I ensured that I only had one tab open rather than two tabs. For the latter, I had one tab for the local source folder and another for the destination folder on the Gmail side. I was concerned, however, that the TB view of the remote folder might be trying to update as the upload was occurring, leading to competition for bandwidth. I realize that the upload hogs mostly outgoing bandwidth and the updating of the Gmail folder mostly uses incoming bandwidth. The handshaking for each one, however, uses both incoming and outgoing signalling, which might get impeded by the high bandwidth email payload.
Geändert am
Interim report on use of Outlook:
As I am rediscovering for the Nth time in the past 2 years, setting up the IMAP connection from Outlook is just the opposite of the hands-free Thunderbird setup. Lots and lots and lots of trial and retrial. For some reason, setting things like allowing less secure app access and the various ports of SSL don't seem to "stick" until the Nth attempt. Once it working, there is no progress indication for uploads.....Sigh.
Having said that, the process *does* seem to be working. The Task Manager's WiFi usage plot shows the upload dropping to zero after 1-3 minutes, but instead of quitting like TB, it will start up again after 1-3 minutes. The Process monitoring tab shows that every so often, significant bandwidth is allotted to Outlook. The motion bar on Outlook continues to indicate some sort of activity. It has been like this for about 1.5 hours. I suspect that whenever the connection is lost, Outlook waits a while before trying again, then resuming the upload.
Ausgewählte Lösung
Whilst stopping the 'real time protection' will stop a load of scanning which can cause delays and improve upload, there may also be an upload issue with gmail server. I did find this information which may be effecting you, but I'm not sure how many emails/ actual size in MB that you are trying to upload in total. Gmail has it's own limits on amount you can upload to server within a specified time period. So gmail will drop connection if limits are reached. Info here: https://support.google.com/a/answer/1071518?hl=en
quoted info from gmail Migrations or bulk operations using IMAP If you exceed the upload bandwidth limit with IMAP, the following recommendations are important:
- Avoid large copy and paste or drag and drop operations when using IMAP.
- If you're migrating, use a supported migration option and avoid using IMAP to upload messages.
- If you need to download a large amount of data, try throttling the tool you're using or download in parts.
Geändert am
Thank you, Toad-Hall. In a way, this limit explain the dropped connection: "Upload with IMAP" is limited to 500MB/day. The emails vary in size from few dozen KB to a *few* dozen MB (less than a handful). There are 725 messages, and the TB folder is 1.1GB.
It's odd, however, that Outlook is able to resume uploading within very few minutes. It seems to have completed the entire 1.1GB in under 12 hours.
Not that I have the need at this point, but one solution would be if an email client showed the number of KB/MB taken up by the messages selected. One can than upload messages a bunch per day, totaling less than 500MB/day. Both TB and Outlook only shows the number of selected messages, not their collective size.
Geändert am