Announcing a New Continuous Integration and Email Package using Travis-CI and Vagrant

Years ago I wrote a library, Fetch, which was designed to read email using the PHP IMAP Extension. At the time I did not expect it to get much reuse so I skipped out on much of the test suite- if I'm going to be perfectly honest here though I have to admit that a huge reason for not building out a test suite then was that I was unsure how to go about it. Building a test suite for an email library has some very interesting requirements, especially if you want to test it in as real world a setting as possible.

• You need a mail server.
• That mail server needs a variety of messages, folders, flags and attachments for all the for different types of tests that are needed.
• Those messages have to remain consistent between test runs.
• Resetting the mail server to it's original state should be fast to encourage lots of testing during development.
• The test suite, and thus the mail server, need to be portable so other people can run tests on their changes.
• Continuous integration (integrated in with something like Github) is really really nice.

Unfortunately for me all of my great excuses have vanished, thanks in large part to Travis-CI and Vagrant. Both of these systems are fantastic for testing. Travis is a continuous integration environment that's setup for a number of languages and has direct integration with a number of services such as Github. Vagrant makes sharing server environments as easy as passing along a configuration file. With the two of these tools I was able to put together a testing package for Fetch that met all of those requirements, and after a bit of additional work I've broken that up into a stand alone package for people to use in their own projects.

Using this is really simple- you just need to call SetupEnvironment.sh before each run of your test suite. Once that is finished running you have a fully functional Dovecot server with a consistent set of messages. It doesn't matter if you're running this from your home computer or if it's run on Travis-CI, as the SetupEnvironment.sh script will recognize where it's being run and work accordingly.

There are also some cool optimizations to make testing faster. The test environment will stay up and running for a half hour after the last test before shutting itself down, so you don't have to wait for the virtual machine to boot between tests. Rather than start from a blank slate each time, the environment has a reset function that restores the original inboxes. With an already provisioned test environment there is almost no overhead by this package between tests, as resetting the inbox takes only a couple of seconds.

The project is up on Github at tedivm\DovecotTesting, so if you're writing an email reading library there's no more excuses. I've also put together a more detailed post describing how this was put together.