Blog


Nov

Our Presentations at RubyConf 2010


Two weeks ago, Jonas and I were in New Orleans for the 10th annual RubyConf. This was my 2nd RubyConf, but I had a very different experience this time. This time I was one of the speakers. And I was giving two presentations.

The Front End Testing Frontier

One of the presentations was an extended version of the Front End Testing Frontier presentation I gave at Mountain.rb in October. This time I was fortunate enough to have my colleague Jonas Nicklas as a co-presenter. Jonas is the author of the testing libraries Capybara and Evergreen, our primary tools for JavaScript testing at Elabs.

I've uploaded the slides from the Front End Testing Frontier presentation, but they're pretty sparse. Video should be available in a few weeks, and I'll update this post then.

Socialist Software Development

My other presentation was titled Socialist Software Development. I came up with the title after seeing a clip from The Daily Show about socialism in Sweden. I saw some similarities between the principles of socialism and agile development, and since it seems most Americans are terrified by the mere mention of socialism, I thought it would be fun to go to the US and talk about it. The essence of my talk came from this passage from the British Labour Party constitution:

The Labour Party is a democratic socialist party. It believes that, by the strength of our common endeavour we achieve more than we achieve alone, so as to create, for each of us, the means to realise our true potential, and, for all of us, a community in which power, wealth, and opportunity are in the hands of the many, not the few.

While I drew some inspiration from my flippant title, I mostly talked about how Sharing is Caring, and how important that is in software development.

Giving such a "soft" talk was quite a bit harder than giving a technical presentation, but I was very happy with how it went. I've posted the slides for the Socialist Software Development presentation as well, and I'll update when the video is online.

What's Next?

I've been to a lot of conferences this year, and it's been great presenting at Mountain.rb and RubyConf. The next conference on our schedule is the Scottish Ruby Conference in April next year. All the Elabs developers will be there, and I'm really looking forward to that. Then there's our own conference, Nordic Ruby, in June. This year was amazing, and it looks like the one next year will be even better.

I don't have any confirmed speaking events for next year, but I would love to do a couple and continue to improve as a speaker. But I think I'll limit it to one presentation per conference. Giving two at RubyConf was a bit overwhelming.

Nov

Build, RubyConf and Dave Hoover


Today we are 4 people at the office, half of the crew. Jimmy and Johannes, our designers, left on Monday for Build which is a design conference in Belfast. And yesterday CJ and Jonas flew to RubyConf in New Orleans were they will be giving the talk: The Front End Testing Frontier. CJ will also give a talk about Socialist Software Development.

On Tuesday we had a short but very appreciated visit by Dave Hoover, who flew by Gothenburg to see us and join Got.rb in the evening. Early on Wednesday he continued his journey by train to Malmö and Øredev Developer Conference.

A quiet day at the office.

Sep

Speaking About Testing and Socialism at Mountain.rb and RubyConf


After attending and organizing it is finally time for the next step, to speak at a conference, two actually, with three talks. Dive in head first. CJ will be speaking at Mountain.rb and RubyConf.

First: Mountain.rb in Boulder, Colorado, October 6-8
The Talk: The Front End Testing Frontier

While most Ruby developers are very familiar with testing their code, frontend and JavaScript-testing is still a new frontier for many. This talk will show you how to easily write and run JavaScript integration tests with Capybara and Cucumber, and unit tests with Evergreen and Jasmine. The goal is to get you excited about frontend testing, and point you in the right direction to get started yourself!

Second and Third: RubyConf in New Orleans, November 11-13
Talk #1: The Front End Testing Frontier, extended, with co-presenter Jonas Nicklas
Talk #2: Socialist Software Development

Socialism is often portrayed as pure evil by US media (hello Fox News), yet many socialist countries are ranked as some of the best countries in the world (Newsweek). So maybe it's not all bad? If you look at job listings for software developers, it seem like a lot of companies are looking for "programmer rock stars", "coding ninjas", etc. There is a romantic notion about the ultra productive independent super developer. This talk examines software development from a socialist perspective.

Might we get better results and provide more value if we set aside our egos and work together?

CJ Kihlbom is speaking at Mountain.rb

Aug

You're Cuking It Wrong


Opinions on cucumber seem to be divided in the Ruby community. Here at Elabs we've been using cucumber to fantastic success on all of our projects for more than a year. At the same time Steak and projects like it seem to be gaining traction; some people are seemingly frustrated and fed up with cucumber.

So where does this gulf of experiences come from, why is cucumber loved by some and hated by others. At the risk of over-generalisation and mischaracterisation I recently came up with a theory: the cucumber detractors are not using cuke the way it was intended.

This is in fact not their fault. The entire cucumber ecosystem, and in fact even cucumber itself, encourage its misuse.

A while ago someone created an issue on the Capybara issue tracker. The interesting thing about this issue wasn't the problem itself, but rather the cucumber feature that the author presented in order to replicate the problem. This is the feature the author submitted:

Scenario: Adding a subpage
  Given I am logged in
  Given a microsite with a Home page
  When I click the Add Subpage button
  And I fill in "Gallery" for "Title" within "#document_form_container"
  And I press "Ok" within ".ui-dialog-buttonpane"
  Then I should see /Gallery/ within "#documents"

At first glance this seems reasonable. But contrast this with the following, improved version:

Scenario: Adding a subpage
  Given I am logged in
  Given a microsite with a home page
  When I press "Add subpage"
  And I fill in "Title" with "Gallery"
  And I press "Ok"
  Then I should see a document called "Gallery"

The difference isn't huge, the steps are largely the same, and there's an argument to be made for writing in a more declarative style, but there's one crucial difference: the first feature is code, the second isn't.

The argument against cucumber that's often presented is that as a programmer, plain text is unnecessary, because we can all read code. While it's true that we all can read code, I still find it beneficial to jump out of the code writing mode for describing the behaviour of the application. When you're writing features first, you don't want to be bothered with the details of how this functionality works. In this initial stage you care nothing about the implementation, about how the result is achieved. You care nothing about things like #document_form_container or .ui-dialog-buttonpane.

I believe that it's in this switching between designer mode and developer mode where cucumber, done right, really shines.

In order to evaluate the bigger picture before hacking
As a developer
I want to write my stories before writing my code

There are some secondary benefits as well. Writing truly plain text features leads to better maintainability as well, since the features are robust against code changes. Plain text is also easier to understand for new developers coming to an existing project. Probably the nicest advantage though is that over time a library of steps is built up, which can then be simply combined to describe new features.

The above feature is nicely illustrative of this anti-pattern, but it is far from the only example. In many of our cucumber suites here at Elabs, we have steps like the above, some of them were written by me. Which leads me to what's really wrong with the last three lines of the above feature. They are written using nothing else than the standard web steps generated by cucumber-rails own generator. Cucumber itself ships with steps which in my opinion encourage an anti-pattern.

Pickle my fancy

Another tool which we've experimented a bit with is Pickle, which allows you to easily generate models from your feature files. A basic example from the README:

Given a user exists
And a post exists with author: the user

Given a person: "fred" exists
And a person: "ethel" exists
And a fatherhood exists with parent: user "fred", child: user "ethel"

It actually looks fairly nice, reads quite naturally, so a first instinct might be to call this plain text. But on closer inspection, there is a whole language in there. To comprehend what these steps are doing you'd need to understand not only the domain models involved, but also the language Pickle uses to manipulate these. I'm pretty sure a non-technical person couldn't make sense of the above. This is really no different, and in fact worse, than writing actual code:

@user = User.make
Post.make(:user => @user)

@fred = Person.make(:name => 'Fred')
@ethel = Person.make(:name => 'Ethel')
Fatherhood.make(:user => @fred, :child => @ethel)

Note how there is an almost one-to-one mapping between the feature above, and the code below. The only thing cucumber does in this case is act as some kind of phoney translator. We write code, but not actual code. So we can do some stuff, but mostly it comes out worse than if we'd just written it as code in the first place. I can't blame anyone for disliking cucumber when using it like this.

However, try this instead:

Given there is a user called "Jimmy"
And there is a post authored by "Jimmy"

Given there is a person called "Fred"
And there is a person called "Ethel"
And "Fred" is the father of "Ethel"

There's not a huge difference between the first couple of lines, even though they read somewhat nicer when written out like this. The real difference is in the last line. Here cucumber is adding value by explaining this abstract concept of a Fatherhood into something very concrete: one person is the other's dad. Cucumber added value to this feature, instead of only acting as a hindrance.

I believe that Pickle is flawed as a concept, in order to achieve readable steps, they need to be written by hand.

The worst feature ever written.

As a curiosity, I present the worst cucumber feature known to man. If you are responsible for something like this, please go slap yourself in the face as hard as you can.

Scenario: User creates some sites and circuits, check connected sites list
    Given a "site" exists with {"name"=>"Somewhere1", "identifier" => "TER1", "provider"=>"TER1 Provider"}
    And a "site" exists with {"name"=>"Somewhere2", "identifier" => "TER2", "provider"=>"Some Provider"}
    And a "site" exists with {"name"=>"Somewhere3", "identifier" => "TER3", "provider"=>"TER3 Provider"}
    And a "circuit" exists with {"provider_name"=>"Another provider", "redacted_circuit_id"=>"ABC1", "provider_circuit_id"=>"C1", "circuit_type"=>CircuitType.find_by_name("Peering"), "service_type"=>CircuitServiceType.find_by_name("Dark Fiber"), :capacity => CircuitCapacity.find_by_name("1 Gbps"), "physical_wire_type"=>PhysicalWireType.find_by_name("Multi Mode Fiber"), "status"=> CircuitStatus.find_by_name("Cancelled"), "a_end"=>Site.find_by_identifier("TER1"), "b_end"=>Site.find_by_identifier("TER2")}
    And a "circuit" exists with {"provider_name"=>"Switch and Data", "redacted_circuit_id"=>"ABC2", "provider_circuit_id"=>"C2", "circuit_type"=>CircuitType.find_by_name("Backbone"), "service_type"=>CircuitServiceType.find_by_name("Dark Fiber"), :capacity => CircuitCapacity.find_by_name("1 Gbps"), "physical_wire_type"=>PhysicalWireType.find_by_name("Multi Mode Fiber"), "status"=> CircuitStatus.find_by_name("Cancelled"), "a_end"=>Site.find_by_identifier("TER1"), "b_end"=>Site.find_by_identifier("TER3")}
    When I am on the "connected_sites" page for site "TER1"
    Then the "connected-sites-list" should look like
      |   Site ID    |  Site Name | Site Provider | Provider Circuit ID  | Provider Name    | Circuit Status |
      |     TER2     | Somewhere2 | Some Provider |         C1           | Another provider |  Cancelled     |
      |     TER3     | Somewhere3 | TER3 Provider |         C2           | Switch and Data  |  Cancelled     |
    When I am on the "connected_sites" page for site "TER2"
    Then the "connected-sites-list" should look like
      |   Site ID    |  Site Name | Site Provider | Provider Circuit ID  | Provider Name    | Circuit Status |
      |     TER1     | Somewhere1 | TER1 Provider |         C1           | Another provider |  Cancelled     |
    When I am on the "connected_sites" page for site "TER3"
    Then the "connected-sites-list" should look like
      |   Site ID    |  Site Name | Site Provider | Provider Circuit ID  | Provider Name    | Circuit Status |
      |     TER1     | Somewhere1 | TER1 Provider |         C2           | Switch and Data  |  Cancelled     |

Yes, those are Hashes inside a feature, which are then eval'd. Make sure to scroll to the right to experience the full horror of it all. I challenge anyone to find a worse cucumber feature than this. I assure you, that thing is real (from one of our rescue mission projects), and there is much more where it came from.

Writing better steps

So how do we write better steps? For me personally, I've found that sticking to the following rule seems to lead to nice, maintainable steps:

A step description should never contain regexen, CSS or XPath selectors, any kind of code or data structure. It should be easily understood just by reading the description.

Jul

Continuous Integration Testing for Ruby on Rails with Integrity


Doing test-driven development usually means you have a lot of tests in a project. While this is almost entirely a good thing, running the thousands of Cucumber features and RSpec examples in a large project takes a couple of minutes. If you run your entire test suite every time you commit this will easily eat up a large chunk of your day. Offloading some of this to a continuous integration server will allow you to save time by running your tests asynchronously, in addition to its other benefits.

At eLabs we usually run our unit tests locally—as well as the Cucumber feature for the story we're currently working on—before checking in. Then we let our CI server run the rest of our Cucumber features and notify us if something goes wrong. Here's the setup we use:

Integrity

At eLabs we've looked at a number of different CI servers, such as CruiseControl.rb and Run Code Run, but our favorite by far is Integrity.

Screenshot of our Integrity site

Integrity suits us perfectly. It fetches our code from our private GitHub repositories, can run any testing command and notify us in a variety of ways such as email and Campfire. It also has a very nice and clean interface. Its one major shortcoming is its complete lack of error reporting. If there's something wrong with your setup it will silently fail, which makes troubleshooting a nightmare. Hopefully the instructions below will help you avoid some of the pitfalls.

Installation

We installed Integrity on a server running Mac OS X and Passenger under Apache. Here's a quick guide.

First we installed the gem:

$ sudo gem install integrity

Then set it up in your chosen directory using the --passenger option:

$ integrity install --passenger /Library/WebServer/Sites/integrity

Next, set up a virtual host in Apache, pointing its DocumentRoot to the public folder in your Integrity installation.

DocumentRoot "/Library/WebServer/Sites/integrity/public"

One absolutely crucial step that we missed at first is to make sure that the system user that runs the Integrity passenger processes has git in its PATH. The simplest way to do this is to set the PATH in the virtual host configuration:

SetEnv PATH /opt/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

After configuring Apache you have to configure Integrity by editing config.yml in the root directory of your Integrity installation. We used SQLite for the database (couldn't get it to work with MySQL). If you want to use a hash password for the admin user, here's a simple way to get the SHA1 of a password:

$ ruby -r 'digest/sha1' -e 'puts Digest::SHA1.hexdigest("password")'

The final step is to create the database:

$ integrity migrate_db

You should now be able to log in to your Integrity site and add your projects.

Setting Up a Project

The most important part of setting up a project for CI is the build command. This is the command that Integrity runs to test your app, and it can be anything that exits with a status of 0 when successful. We use a simple rake task that prepares our project by copying a database.yml file and runs RSpec and Cucumber tests.

namespace :ci do
  task :copy_yml do
    system("cp #{Rails.root}/config/database.yml.ci #{Rails.root}/config/database.yml")
  end

  desc "Prepare for CI and run entire test suite"
  task :build => ['ci:copy_yml', 'db:migrate', 'spec', 'features'] do
  end
end

With that committed to our repository (along with a database.yml.ci file) we add the project to Integrity. The important parts here are the Git repository and Build script settings.

Add a project to Integrity

You must also make sure that the Integrity user can access your repository on GitHub. There are a couple of different ways you can do this, but we created a separate free GitHub account that we add as a collaborator to our projects.

After you add the project you should be able to request a manual build from the Integrity web interface. Note that the build is done synchronously—so you'll have to wait a while—but if the build succeeds you're ready to set up the Post-Receive hook for GitHub to have Integrity run your tests whenever you push your code to GitHub.

GitHub Post-Receive URL settings

Go to your project's page on GitHub and click the Admin link in the top menu, and then Service Hooks in the sub menu. Enter the push URL for your Integrity project as Post-Receive URL. The URL has the following format:

http://username:password@hostname/project-name/push

After you've updated the settings, click the Test Hook link and Integrity should start a new build. If that works, you're all set for having automated builds on every push to GitHub.

Notifiers

While Integrity's interface is nice, you probably don't want to visit your Integrity site after every commit to check the status of your build. The point of asynchronous tests after all is to get notified when somethings goes wrong. Integrity has a bunch of different notifiers you can use. We use the ones for email and Campfire. Find more and installation instructions on the Integrity site.

In addition to Integrity's own notifiers we also use CCMenu, a Mac OS X Menu extra built for showing CruiseControl build status. It works with Integrity as well with the gem integritray.

We also use GitHub's Campfire service hook that posts a message to our Campfire room every time someone pushes new code. This makes it very easy to keep track of what other people in the company are working on.

Campfire screenshot

Not having to wait for our entire test suite to run before each commit saves us a lot of time. But we can still feel confident knowing that Integrity has our backs and will alert us if something goes wrong.