[SOLVED] Switching IMAP Providers

[CaliZ]

First thing first, I searched the forums and even saw a couple of threads with ones that I replied to a couple years back, but ultimately no answer.

The situation is that my customer is switching from their existing e-mail provider (Using IMAP on all systems) to another e-mail provider. If they were using POP3, the easiest way to prevent e-mail loss would be to:
1. Be sure MX records are the same, "mail.domainname.com"
2. Create the same accounts with the same password, "user@domainname.com" with "samepassword"
3. Make the switch and just wait for the root servers to populate.

That way if the records take place half way through a busy day, or late at night that it is as seamless as possible without having to make any changes to the systems configurations.

However, with IMAP, my understanding is that the customer's e-mail client (Outlook) will update after the record change takes place, with no existing e-mails in the new mail server and remove those that were on the customer's e-mail client. (Using an .OST file)


My question is, what is the best way to ensure that no e-mails are lost?
I could backup the current IMAP accounts through their e-mail client (Outlook) to a .PST file. However, without knowing when the switch will precisely occur, the backup will only have the e-mails and folders up to the point of when the backup is done. (Or creating a rule to make a copy to the .PST, but this is a lot more work for 15+ systems and multi e-mail addresses.)

Have I lost anyone yet?
Thank you for any help.

 

[timeshifter]

This tool will take care of all that for you. It will sync the old mailboxes to the new ones a few days before cut over. Then it will sweep them for changes daily before and after the MX switch. Very nice tool that I learned about here and used on a migration. However, that was from IMAP to Office 365. Don't know if they do IMAP to IMAP.

https://www.skykick.com/migrate/pop3-imap-to-office-365-migration

 

[CaliZ]

Hi timeshifter,

Thanks for the reply. I checked out the link you referred to, however I was only able to find information relating to Office365 hosted migrations.

The other alternative option is to setup all systems with POP3 to download all messages continually until the switch, but then they would lose their folder structures. (And doing a mixture with a .PST backup of the IMAP with the folders and .PST of the POP3 will be completely chaotic to get reorganized.)

 

[Markverhyden]

How big are the stores?

At any rate, after making a backup of course, create a new email profile in Outlook. This preserves the old one which you can flip back to, offline line of course. Just did this on a OS X customer with Outlook 2011.

Made the MX record changes, made sure the old profile was not receiving any emails, created the new profile, setup, and then imported the backup. The only thing that popped up, new to me, is for some reason the all of dates and time in the sent folder were from the migration date. So figuring that one out.

 

[YeOldeStonecat]

Safest way is to use a middle man migration service, such as skykick, or we more often use BitTitan also (previously known as Migration Wiz).

This way you can "pre-seed" the new mail host...without interrupting the old host. I often do this a week ahead of time, since...some users might have large mailboxes that can take a while to upload to the new mail host. Takes the pressure off of actual cutover night. Also dial down the TTL for the MX record at least a week ahead of time.

Cutover night..flip the MX record to the new mail host.
Repoint the Outlook and mobile clients. Adjust any on-prem devices like MFPs and other 3rd party devices or services that might tie into e-mail (such as contact forms from a website).

Leave the migration tool still parsing the old mail host for new arrivals (called Delta Passes)..so it will snag any straggler e-mails that still arrive at the old host and upload to new host..so it really minimizes chances of missing any e-mail sent during cutover weekend.

 

[CaliZ]

Thanks again for both the replies, to answer a bit more:
-The mailbox stores range from 500 MB to 2 GB.
-Backups are nice for basic structure, but with the volume of incoming new mail, I'd have to try and backup each account as close to the switchover as possible before it syncs and says there is no mail on the new server. (The switch being done sometime in 12-24 Hours from now)
-I've setup two test systems, one system with IMAP accounts from the previous hosting provider and new provider, along with a system gathering just the POP3 of all the accounts. This way at worst I have a complete POP3 continuous backup of all mail until the switch is complete, and the IMAP accounts for their folder structures until the previous one goes away. (Rules set for each account to make a copy to the new IMAP already configured)

The new arising issue I found is that their new e-mail provider only has 250 MB e-mail box limits. Will have to try and get a hold of the provider before that time to expand their limits, or as I would prefer, get them switched over to POP3. IMAP is nice for checking e-mails on multi-devices, but POP3 just works fine. (Always one primary system making copies, and the rest are throw away's.) Those saying sent items are missed, each device/program is set to send a cc'd copy back to themselves and organized with a simple Outlook rule.

YeOldeStonecat, if I had more time, I would consider that route or the ability to make a complete account backup from the old mail provider server to the new one. (The previous provider is not willing to hold e-mails past the final date that the owner has agreed to switchover with the new provider. Late one with these two!)
Will keep everyone updated on progress.

 

[YeOldeStonecat]

YeOldeStonecat, if I had more time, I would consider that route or the ability to make a complete account backup from the old mail provider server to the new one. (The previous provider is not willing to hold e-mails past the final date that the owner has agreed to switchover with the new provider. Late one with these two!)
Will keep everyone updated on progress.

So I would just pull the trigger for the migration several days BEFORE the day the prior host deletes his mailboxes. During a cutover..straggler e-mail often still ends up in the old mailboxes...some DNS servers just don't get that update, even though textbooks teach to reel in the TTL values ahead of time. The middle man migration tools keep sweeping the old mailboxes for a while to get and retrieve any straggler emails. Else..once you download (back up) to a local PST...how do you get email that arrives in the old mailbox after that?

 

[CaliZ]

So I would just pull the trigger for the migration several days BEFORE the day the prior host deletes his mailboxes. During a cutover..straggler e-mail often still ends up in the old mailboxes...some DNS servers just don't get that update, even though textbooks teach to reel in the TTL values ahead of time. The middle man migration tools keep sweeping the old mailboxes for a while to get and retrieve any straggler emails. Else..once you download (back up) to a local PST...how do you get email that arrives in the old mailbox after that?

Exactly, I was just hired on just yesterday and informed of this switch and the deadlines. *doh* And yes, the system with the POP3 I have set with all the mailboxes will continue to pull from the previous mail server to the absolutely latest times just in case. Otherwise as mentioned, once the mailbox sizes are adjusted to accommodate the previous storage capacity, it will continue to sync from my test end and be pretty seamless once the switch is made. Then I can manually sync their inbox folders with the POP3 I have on to the new mail server if needed.

 

[Diggs]

This tool will take care of all that for you. It will sync the old mailboxes to the new ones a few days before cut over. Then it will sweep them for changes daily before and after the MX switch. Very nice tool that I learned about here and used on a migration. However, that was from IMAP to Office 365. Don't know if they do IMAP to IMAP.

https://www.skykick.com/migrate/pop3-imap-to-office-365-migration

I thought this was very cool but they've put all their eggs in the Office 365 basket so it's not the golden tool I was hoping for.

 

[ComputerRepairTech]

He could always just setup an email forward to an address on a different service like gmail or something on his old host that way he doesn't miss emails during the dns propagation.

 

[CaliZ]

He could always just setup an email forward to an address on a different service like gmail or something on his old host that way he doesn't miss emails during the dns propagation.

I'm not sure that would work with IMAP folders though. For POP, yes. Thus having set up all their POP accounts on one of my systems as a continual backup until the DNS props.

The customer decided to follow our recommendation for us to have the IMAP backed up to a .PST at its current state (Folder structure), and expect for us to restore the missing e-mails from this point using the POP config and merging them together. (Final cleanup with duplicate remover). Done and done for now.

 

[ComputerRepairTech]

I'm not sure that would work with IMAP folders though. For POP, yes. Thus having set up all their POP accounts on one of my systems as a continual backup until the DNS props.

The customer decided to follow our recommendation for us to have the IMAP backed up to a .PST at its current state (Folder structure), and expect for us to restore the missing e-mails from this point using the POP config and merging them together. (Final cleanup with duplicate remover). Done and done for now.

When you say pop you mean retrieving it from the old web host using old web host specific host names right? just double checking.

 

[CaliZ]

POP being Post Office Protocol 3. End result due to time constraint was:
1. Run export of IMAP accounts to .PST file.
2. Setup Outlook with accounts using POP and let it collect further e-mails until propagation to new records take place (Different provider, different MX records)
3. Once e-mail stops working on old provider, remove old IMAP accounts on customers Outlook applications, load up backup of IMAP to PST files, load up POP PST file, configure accounts using new hosting provider MX details and merge.

If the new hosting provider allowed for IMAP - > IMAP (From old host to new with both being available.) then it would have been easier, but I was told their total storage was 1/4 of what it was before. (One of the big reasons for switching to POP)

Eliminates issues with merging straight from IMAP (.OST) to POP (.PST) (Outlook doesn't seem to like transferring directly from one to another, but exports fine to a .PST. (Needed to change view settings to show "IMAP Messages" otherwise messages have filter applied and do not show in .PST file.

 

[timeshifter]

I kind of got lost with your jumping back and forth to POP3 and the like. Seems like the easiest concept would have been to set up two Outlook profiles for each user. One for the old host. One for the new. All IMAP. After the cutover you could go to the old profile periodically and sync the mail. Once all the leftovers are scooped up after a couple of days you could export out to PST.

This would require that a) you still had a relationship with the old host and b) you could still access the old host with their server names like imap.secureserver.net as opposed to imap.yourdomain.com.

 

[CaliZ]

I kind of got lost with your jumping back and forth to POP3 and the like. Seems like the easiest concept would have been to set up two Outlook profiles for each user. One for the old host. One for the new. All IMAP. After the cutover you could go to the old profile periodically and sync the mail. Once all the leftovers are scooped up after a couple of days you could export out to PST.

This would require that a) you still had a relationship with the old host and b) you could still access the old host with their server names like imap.secureserver.net as opposed to imap.yourdomain.com.

Unfortunately not, there was no further time to work with the old host as the old host would be deleting all old mail at the same time as the switchover to the new one, and the customer needed a solution with a finalized transfer in one shot in order to resume business and us not interfering any more than absolutely needed. Little bit of a rare case.

 

[ComputerRepairTech]

but without accessing the old host how can you be sure you didnt miss emails? dns propagation does not happen all at once for everyone, it may looked done for you but others it may not be and so you almost always have emails that arrive at the old host.

 

[CaliZ]

The old host had deleted the account by then, any *lost* e-mails would likely either bounce back, or eventually query the DNS to be directed to the new host's MX records.

I understand what you are saying, and this is a rare case where most would still have access to the old host after records are changed to be sure no mail is missing, but the account was terminated out of mine and the customers control.

 

[timeshifter]

What's the story of the host terminating the account?

 

[CaliZ]

End of contract. Customer did not want to continue, nor understood the importance of getting everything switched over even a week sooner. We just happened to be hired on within the 1-2 days before it needed to be completed.

 

[ComputerRepairTech]

The old host had deleted the account by then, any *lost* e-mails would likely either bounce back, or eventually query the DNS to be directed to the new host's MX records.

I understand what you are saying, and this is a rare case where most would still have access to the old host after records are changed to be sure no mail is missing, but the account was terminated out of mine and the customers control.

Depends right? many hosts just suspend the account, i think they can still get mail at that point they just can't see it. Email forwards will still work with cpanel...well that was years ago i'm not sure if they ever got around to fixing that or having an option to suspend email xD