Larch

Larch is a tool to copy messages from one IMAP server to another quickly and safely. It’s smart enough not to copy messages that already exist on the destination and robust enough to deal with interruptions caused by flaky connections or misbehaving servers.

Larch is particularly well-suited for copying email to, from, or between Gmail accounts.

Author

Ryan Grove ([email protected])

Version

1.1.0 (git)

Copyright

Copyright © 2009 Ryan Grove. All rights reserved.

License

GPL 2.0 (opensource.org/licenses/gpl-2.0.php)

Website

github.com/rgrove/larch

Installation

Install the latest stable version of Larch via RubyGems:

gem install larch

Or you can install the latest development version from GitHub:

gem sources -a http://gems.github.com
gem install rgrove-larch

Usage

larch [config section] [options]
larch --from <uri> --to <uri> [options]

Server Options:
       --from, -f <s>:   URI of the source IMAP server.
--from-folder, -F <s>:   Source folder to copy from (default: INBOX)
  --from-pass, -p <s>:   Source server password (default: prompt)
  --from-user, -u <s>:   Source server username (default: prompt)
         --to, -t <s>:   URI of the destination IMAP server.
  --to-folder, -T <s>:   Destination folder to copy to (default: INBOX)
    --to-pass, -P <s>:   Destination server password (default: prompt)
    --to-user, -U <s>:   Destination server username (default: prompt)

Sync Options:
            --all, -a:   Copy all folders recursively
 --all-subscribed, -s:   Copy all subscribed folders recursively
       --exclude <s+>:   List of mailbox names/patterns that shouldn't be
                         copied
   --exclude-file <s>:   Filename containing mailbox names/patterns that
                         shouldn't be copied

General Options:
     --config, -c <s>:   Specify a non-default config file to use (default:
                         ~/.larch/config.yaml)
       --database <s>:   Specify a non-default message database to use
                         (default: ~/.larch/larch.db)
        --dry-run, -n:   Don't actually make any changes
    --max-retries <i>:   Maximum number of times to retry after a recoverable
                         error (default: 3)
   --no-create-folder:   Don't create destination folders that don't already
                         exist
      --ssl-certs <s>:   Path to a trusted certificate bundle to use to
                         verify server SSL certificates
         --ssl-verify:   Verify server SSL certificates
  --verbosity, -V <s>:   Output verbosity: debug, info, warn, error, or fatal
                         (default: info)
        --version, -v:   Print version and exit
           --help, -h:   Show this message

Usage Examples

Larch is run from the command line. The following examples demonstrate how to run Larch using only command line arguments, but you may also place these options in a config file and run Larch without any arguments if you prefer. See the “Configuration” section below for more details.

For an overview of all available options, run:

larch -h

At a minimum, you must specify a source server and a destination server in the form of IMAP URIs:

larch --from imap://mail.example.com --to imap://imap.gmail.com

Larch will prompt you for the necessary usernames and passwords, then sync the contents of the source’s INBOX folder to the destination’s INBOX folder.

To connect using SSL, specify a URI beginning with imaps://:

larch --from imaps://mail.example.com --to imaps://imap.gmail.com

If you’d like to sync a specific folder other than INBOX, specify the source and destination folders using --from-folder and --to-folder. Folder names containing spaces must be enclosed in quotes:

larch --from imaps://mail.example.com --to imaps://imap.gmail.com \
  --from-folder 'Sent Mail' --to-folder 'Sent Mail'

To sync all folders, use the --all option (or --all-subscribed if you only want to sync subscribed folders):

larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all

By default Larch will create folders on the destination server if they don’t already exist. To prevent this, add the --no-create-folder option:

larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all \
  --no-create-folder

You can prevent Larch from syncing one or more folders by using the --exclude option, which accepts multiple arguments:

larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all \
  --exclude Spam Trash Drafts "[Gmail]/*"

If your exclusion list is long or complex, create a text file with one exclusion pattern per line and tell Larch to load it with the --exclude-file option:

larch --from imaps://mail.example.com --to imaps://imap.gmail.com --all \
  --exclude-file exclude.txt

The wildcard characters * and ? are supported in exclusion lists. You may also use a regular expression by enclosing a pattern in forward slashes, so the previous example could be achieved with the pattern /(Spam|Trash|Drafts|\[Gmail\]\/.*)/

Configuration

While it’s possible to control Larch entirely from the command line, this can be inconvenient if you need to specify a lot of options or if you run Larch frequently and can’t always remember which options to use. Using a configuration file can simplify things.

By default, Larch looks for a config file at ~/.larch/config.yaml and uses it if found. You may specify a custom config file using the --config command line option.

The Larch configuration file is a simple YAML file that may contain multiple sections, each with a different set of options, as well as a special default section. The options in the default section will be used unless they’re overridden either in another config section or on the command line.

Example

Here’s a sample Larch config file:

default:
  all-subscribed: true # Copy all subscribed folders by default

# Copy mail from Gmail to my server, excluding stuff I don't want.
gmail to my server:
  from: imaps://imap.gmail.com
  from-user: example
  from-pass: secret

  to: imaps://mail.example.com
  to-user: example
  to-pass: secret

  exclude:
    - "[Gmail]/Sent Mail"
    - "[Gmail]/Spam"
    - "[Gmail]/Trash"

# Copy mail from my INBOX to Gmail's INBOX
my inbox to gmail inbox:
  all-subscribed: false

  from: imaps://mail.example.com
  from-folder: INBOX
  from-user: example
  from-pass: secret

  to: imaps://imap.gmail.com
  to-folder: INBOX
  to-user: example
  to-pass: secret

This file contains three sections. The options from default will be used in all other sections as well unless they’re overridden.

To specify which config section you want Larch to use, just pass its name on the command line (use quotes if the name contains spaces):

larch 'gmail to my server'

If you specify additional command line options, they’ll override options in the config file:

larch 'gmail to my server' --from-user anotheruser

Running Larch with no command line arguments will cause the default section to be used. With the example above, this will result in an error since the default section doesn’t contain the required from and to options, but if you only need to use Larch with a single configuration, you could use the default section for everything and save yourself some typing on the command line.

Server Compatibility

Larch should work well with any server that properly supports IMAP4rev1, and does its best to get along with servers that have buggy, unreliable, or incomplete IMAP implementations.

Larch has been tested on and is known to work well with the following IMAP servers:

  • Dovecot

  • Gmail

  • Microsoft Exchange 2003

The following servers do not work well with Larch:

  • BlitzMail - Buggy server implementation; fails to properly quote or escape some IMAP responses, which can cause Larch to hang waiting for a terminating character that will never arrive.

Known Issues

  • Larch uses Ruby’s Net::IMAP standard library for all IMAP operations. While Net::IMAP is generally a very solid library, it contains a bug that can cause a deadlock to occur if a connection drops unexpectedly (either due to network issues or because the server closed the connection without warning) when the server has already begun sending a response and Net::IMAP is waiting to receive more data.

    If this happens, Net::IMAP will continue waiting forever without passing control back to Larch, and you will need to manually kill and restart Larch.

  • The Ruby package on Debian, Ubuntu, and some other Debian-based Linux distributions doesn’t include the OpenSSL standard library. If you see an error like uninitialized constant Larch::IMAP::OpenSSL (NameError) when running Larch, you may need to install the libopenssl-ruby package. Please feel free to complain to the maintainer of your distribution’s Ruby packages.

Support

The Larch mailing list is the best place for questions, comments, and discussion about Larch. You can join the list or view the archives at groups.google.com/group/larch

Credit

The Larch::IMAP class borrows heavily from Sup by William Morgan, the source code of which should be required reading if you’re doing anything with IMAP in Ruby.

Larch uses the excellent Trollop command-line option parser (also by William Morgan) and the HighLine command-line IO library (by James Edward Gray II).

License

Copyright © 2009 Ryan Grove <[email protected]>

Licensed under the GNU General Public License version 2.0.

This program is free software; you can redistribute it and/or modify it under the terms of version 2.0 of the GNU General Public License as published by the Free Software Foundation.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, visit www.gnu.org/licenses/old-licenses/gpl-2.0.txt or write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.