ICG Link News

02.10.11

Mail System Improvements

We had to perform an unscheduled major retooling of our mail on 2/9/11. We were planning on doing this project during a scheduled window, but circumstances caused us to do it during business hours on that day. The end result for users who kept mail on the servers would have been the same regardless, but at least there would have been a warning. The reason the rebuild was unscheduled was that POP problems compounded when multiple users with many thousands of messages checked mail all at once on that day, overloading the POP servers. In an emergency response, we rebuilt the entire mail architecture. This caused all messages to get a new ID which meant that they were all downloaded again. Folks who had that problem needed to either uncheck the box in advanced settings that says "Leave a copy of mail on the server." or if they need it checked, to also check the one that says "Delete after x days" and set it to something reasonable like one or two or at max, maybe 10.

Also, on another note, when sending mail, most folks have their outgoing mail set to authenticate which is required. However, some may not have had it set properly and could have had problems sending mail. For those with problems sending, the outgoing (SMTP) mail settings must be set to require authentication, using the same settings as incoming. Be sure the username is the full email address such as john@johndoe.com rather than john or john/johndoe.com.

During the first week of March, we uncovered what we believe is a final related problem and resolved it. During the initial debugging, we failed to anticipate a rare bottleneck in data logging that caused the incoming mail servers to stop processing mail every once in a while. That bottleneck has been uncovered and was corrected.

The end result of the rebuild is much faster and more reliable mail and performance was successful beyond our expectations. The load on the mail servers has dropped to nothing and the systems are purring along beautifully. I am very confident we will be much more stable and reliable in the future as a result of the effort.