IMAP

UIDs, synchronization, and IDLE

Use stable identifiers and mailbox state to synchronize safely while receiving near-real-time updates.

IMAP gives every message two numbers, and mixing them up is the classic IMAP bug.

The sequence number is the message’s position in the mailbox right now. Expunge message 3 and every message after it shifts down by one.

The UID is assigned once and never reused while the mailbox keeps the same UIDVALIDITY. It’s the number you store. Always use the UID variants of commands (UID FETCH, UID STORE, UID SEARCH) in anything that survives longer than one session.

The cache key

A UID alone isn’t unique. UID 104 in INBOX and UID 104 in Archive are different messages. And if the server rebuilds INBOX, it changes UIDVALIDITY and may hand out UID 104 again to a different message.

So a local cache entry needs four parts: account, mailbox, UIDVALIDITY, and UID. Miss any one and you’ll eventually show the wrong message under the wrong subject.

The sync loop

A synchronizing client does the same thing every time it connects:

  1. SELECT the mailbox and read UIDVALIDITY
  2. If it matches the saved value, fetch flags for known UIDs and fetch new UIDs above the last one seen
  3. If it changed, throw the local cache for this mailbox away and rebuild from scratch

Step 3 feels drastic. The alternative is trusting IDs the server told you are invalid.

IDLE: let the server call you

Polling every minute wastes battery and still feels slow. The IDLE extension fixes that. The client says it’s idle, and the server pushes updates as they happen:

C: A010 IDLE
S: + Idling
S: * 43 EXISTS
C: DONE
S: A010 OK IDLE terminated

The + is the continuation request: the server accepted and is now watching. When a message arrives it sends * 43 EXISTS. The client ends the idle period by sending DONE, which is not a tagged command. It’s a raw line that belongs to A010, and the tagged OK arrives after it.

Servers time idle connections out, often around 30 minutes, so clients send DONE and re-enter IDLE periodically. TLS-protected IMAP runs on port 993, and with IDLE that connection stays open for a long time.

EXISTS is a notification, not a message

* 43 EXISTS means the mailbox now has 43 messages. It doesn’t include the new message. The client still has to UID FETCH to learn what arrived. It’s a doorbell, not a delivery.

When the network drops

Phones lose connectivity constantly, and an idling connection dies silently. When you reconnect: SELECT again, compare UIDVALIDITY, fetch what changed since your saved state. Don’t assume you received every push while you were gone.

Try designing a cache record for one message. Then write the recovery steps for a client whose network disappeared while it was idling.

Lesson completed