nnmlexodus - nnml to IMAP migration

Sat, 2012-09-01 22:35 by quest

Gnus has the ability to migrate groups of messages from one backend to another (via "B c"). However, this quickly becomes impractical when you have dozens (not to mention hundreds) of folders. In addition, it appears that Gnus is having problem retaining the various flags on messages. In order to migrate 15 years worth of mail from nnml, I wrote nnmlexodus.

Get nnmlexodus from bitbucket.

nnmlexodus reads messages from an nnml tree and injects them into a standards-compliant IMAP4 server. In order to achieve decent performance, it can use multiple simultaneous IMAP clients. It also preserves message marks (read marks, replies, forwards, et cetera) as documented by .marks files inside the nnml tree.

PYTHONPATH=. python ./main.py --ssl \
  --host imap.gmail.com --user foo --pass bar \
  --root ~/Mail/nnml

(Because the tool is organized into multiple modules, you need to supply current directory as PYTHONPATH. Future versions may do away with this limitation.)

Using --exclude, it is possible to exclude part of the nnml tree; perhaps those 200000 spam messages can stay behind, no? Similarily, using --transform, you can point to a file that control how the nnml path is converted into a folder name. Each row in this file has the following format:

<regex> <tab> <substitution>

The regex and the substitution pattern are handed directly to Python's re.sub. Consult for further details.

nnmlexodus can be stopped and restarted because it checks if the message-id from nnml is already present in the target IMAP folder before importing it. If so, that message will be skipped.

Running nnmlexodus produces fairly detailed output. You may want to redirect this output to a file in order to verify that the import went as you expected.

nnmlexodus is a work-in-progress and patches are welcome. It has been tested with python2.7 on Linux and should also work with python 3.2. It is unlikely to work fully on Windows in its current state.

TODO

  • Make sure we can run on python 2.5 (using argparse ties us to 2.7)
  • Add shell script that sets PYTHONPATH