Blog


Jul

Continuous Integration Testing for Ruby on Rails with Integrity


Doing test-driven development usually means you have a lot of tests in a project. While this is almost entirely a good thing, running the thousands of Cucumber features and RSpec examples in a large project takes a couple of minutes. If you run your entire test suite every time you commit this will easily eat up a large chunk of your day. Offloading some of this to a continuous integration server will allow you to save time by running your tests asynchronously, in addition to its other benefits.

At eLabs we usually run our unit tests locally—as well as the Cucumber feature for the story we're currently working on—before checking in. Then we let our CI server run the rest of our Cucumber features and notify us if something goes wrong. Here's the setup we use:

Integrity

At eLabs we've looked at a number of different CI servers, such as CruiseControl.rb and Run Code Run, but our favorite by far is Integrity.

Screenshot of our Integrity site

Integrity suits us perfectly. It fetches our code from our private GitHub repositories, can run any testing command and notify us in a variety of ways such as email and Campfire. It also has a very nice and clean interface. Its one major shortcoming is its complete lack of error reporting. If there's something wrong with your setup it will silently fail, which makes troubleshooting a nightmare. Hopefully the instructions below will help you avoid some of the pitfalls.

Installation

We installed Integrity on a server running Mac OS X and Passenger under Apache. Here's a quick guide.

First we installed the gem:

$ sudo gem install integrity

Then set it up in your chosen directory using the --passenger option:

$ integrity install --passenger /Library/WebServer/Sites/integrity

Next, set up a virtual host in Apache, pointing its DocumentRoot to the public folder in your Integrity installation.

DocumentRoot "/Library/WebServer/Sites/integrity/public"

One absolutely crucial step that we missed at first is to make sure that the system user that runs the Integrity passenger processes has git in its PATH. The simplest way to do this is to set the PATH in the virtual host configuration:

SetEnv PATH /opt/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

After configuring Apache you have to configure Integrity by editing config.yml in the root directory of your Integrity installation. We used SQLite for the database (couldn't get it to work with MySQL). If you want to use a hash password for the admin user, here's a simple way to get the SHA1 of a password:

$ ruby -r 'digest/sha1' -e 'puts Digest::SHA1.hexdigest("password")'

The final step is to create the database:

$ integrity migrate_db

You should now be able to log in to your Integrity site and add your projects.

Setting Up a Project

The most important part of setting up a project for CI is the build command. This is the command that Integrity runs to test your app, and it can be anything that exits with a status of 0 when successful. We use a simple rake task that prepares our project by copying a database.yml file and runs RSpec and Cucumber tests.

namespace :ci do
  task :copy_yml do
    system("cp #{Rails.root}/config/database.yml.ci #{Rails.root}/config/database.yml")
  end

  desc "Prepare for CI and run entire test suite"
  task :build => ['ci:copy_yml', 'db:migrate', 'spec', 'features'] do
  end
end

With that committed to our repository (along with a database.yml.ci file) we add the project to Integrity. The important parts here are the Git repository and Build script settings.

Add a project to Integrity

You must also make sure that the Integrity user can access your repository on GitHub. There are a couple of different ways you can do this, but we created a separate free GitHub account that we add as a collaborator to our projects.

After you add the project you should be able to request a manual build from the Integrity web interface. Note that the build is done synchronously—so you'll have to wait a while—but if the build succeeds you're ready to set up the Post-Receive hook for GitHub to have Integrity run your tests whenever you push your code to GitHub.

GitHub Post-Receive URL settings

Go to your project's page on GitHub and click the Admin link in the top menu, and then Service Hooks in the sub menu. Enter the push URL for your Integrity project as Post-Receive URL. The URL has the following format:

http://username:password@hostname/project-name/push

After you've updated the settings, click the Test Hook link and Integrity should start a new build. If that works, you're all set for having automated builds on every push to GitHub.

Notifiers

While Integrity's interface is nice, you probably don't want to visit your Integrity site after every commit to check the status of your build. The point of asynchronous tests after all is to get notified when somethings goes wrong. Integrity has a bunch of different notifiers you can use. We use the ones for email and Campfire. Find more and installation instructions on the Integrity site.

In addition to Integrity's own notifiers we also use CCMenu, a Mac OS X Menu extra built for showing CruiseControl build status. It works with Integrity as well with the gem integritray.

We also use GitHub's Campfire service hook that posts a message to our Campfire room every time someone pushes new code. This makes it very easy to keep track of what other people in the company are working on.

Campfire screenshot

Not having to wait for our entire test suite to run before each commit saves us a lot of time. But we can still feel confident knowing that Integrity has our backs and will alert us if something goes wrong.