Handle secret credentials in Ruby On Rails

This blog post aims to lay out a simple and concrete strategy for handling sensitive data in your Ruby On Rails applications, and to explain the importance of such a strategy.

Never, ever check them into source control

Even if your project is closed source and your trusted colleagues are the only ones with access, you never know when a freelancer or consultant might be joining the project. Even if that never occurs, how do you keep track of all the locations where that repository is checked out? Who knows on how many hard drives your company's credit card transaction secret API key might be stored. What happens when someone with a weak login password forgets their laptop on the bus or at the airport?

Also note that it's not always as simple as removing secrets after the fact, especially with version control. It's usually impossible to do this without drastically changing your entire project's history!

Do it right

For a long time, we've been using YAML files to store our application configuration. It's easy to manage and can be configured for different Rails environments. These YAML files could look like the following:


development: &defaults
  awesomeness_score: 3
  host: "localhost:3000"
  s3_bucket: "example-development-us"

  <<: *defaults
  host: ""
  s3_bucket: "example-production-us"

  <<: *defaults


  development: &defaults
  aws_access_key_id: ""
  aws_secret_access_key_id: ""

  <<: *defaults

  <<: *defaults


development: &defaults
  aws_access_key_id: "ACTUAL-ID-WOULD-GO-HERE"
  aws_secret_access_key_id: "ACTUAL-SECRET-WOULD-GO-HERE"

  <<: *defaults

  <<: *defaults

Only the first two files would be checked in to source control, and the application's README would instruct developers to cp config/app_secret.yml.example config/app_secret.yml and fill in the gaps from the company keychain.

To make sure we never check in the secrets by mistake, we ignore the app_secret.yml file:


# ...

We then use the econfig gem written by Jonas Nicklas to easily merge them together:


# ...
gem "econfig", require: "econfig/rails"


# ...
module YourApp
  extend Econfig::Shortcut
  # ...

Now we can access any configuration variable and secret credential: # => "localhost:3000"
YourApp.aws_secret_access_key_id # => "ACTUAL-SECRET-WOULD-GO-HERE"


When you deploy the application, you must manually manage the secrets on the server(s).


If you deploy with Capistrano, you'll want to place the app_secret.yml in your /shared folder. Once that's done, it can be copied to each release with symlink task:


# ...
namespace :config do
  desc "Symlink application config files."
  task :symlink do
    run "ln -s {#{shared_path},#{release_path}}/config/app_secret.yml"  

after "deploy", "config:symlink"


If you're deploying your application where you don't have file access, such as Heroku, you're better off storing this kind of information in ENV. The econfig gem has built in support for this and a few other storage backends, but that's another blog post.


With this method, we now have a clear separation of sensitive and non-sensitive data. There's no risk of checking in any sensitive data, since we have only one place to put it all and it's hidden from source control. Data access within the application hasn't changed, and we no longer have to concern ourselves with how sensitive it is.

We can now be sure that giving access to a repository does not imply giving access to other systems.


If you have any feedback on how the blog post can be improved, or if you spot any errors, please let me know by posting a comment below!


A Capybara future

UPDATE: Some of these proposed changes have made it into Capybara 2.1. Please read our introduction to Capybara 2.1.

Over the last month or so, I've spent a lot of time thinking about the current state of Capybara.

As you may know, we released version 2.0 a while ago, which brought a number of significant changes. The biggest change, and for many the most aggravating change, was that matches now needed to be unambiguous. That is that if we try to do click_link("Remove") and there are multiple remove links on the page, we would throw an exception. I believe that this was a very good change. I have over the last years spent a lot of time tracking down issues where Capybara was simply interacting with the wrong element on the page.

The mistake we made however was that we still allowed substring matches, so that fill_in("Password", :with => "capybara"), not only matched "Password" but also "Password confirmation". This leaves us with a situation where we have to bend over backwards to avoid the ambiguity error. Clearly we should have gone all the way and disallowed substring matches.

Furthermore we should have provided a sane default, strictness, and allowed that default to be changed in specific cases, through options. For example we could have allowed:

click_link("Remove") # very strict
click_link("Remove", :exact => false) # allow substrings
click_link("Remove", :ambiguous => true) # pick any Remove link at random
click_link("Remove", :ambiguous => true, :exact => false) # 1.x behaviour

But even if we had made in hindsight those smarter choices, that still leaves us with an inherent ambiguity in what the arguments to action methods, like click_link actually do. This is what the current documentation says about fill_in:

Locate a text field or text area and fill it in with the given text. The field can be found via its name, id or label text.

This is actually not the whole truth. Fields can also be found via their placeholder, which isn't documented. When found via name, id or placeholder the match needs to be exact, via label it is allowed to be a substring.

The XPath selector we use to find fields like this is quite complicated, and consequently quite slow as well.

What if fill_in worked like this instead:

fill_in "Some label", :with => "Text"
fill_in :id => "some-id", :with => "Text"
fill_in :placeholder => "Fill in some text", :with => "Text"
fill_in :name => "some[name]", :with => "Text"

We still preserve the simplicity of the normal case of selecting fields by their label, yet we make it clear when we deviate from that. This would also allow us to compile an XPath selector specific for this scenario, which would probably be quite a bit faster.

For click_link, and click_button the default would be to find them by their text, with :id and :title being valid alternatives.

I'm thinking that combining these two changes, the addition of the exact and ambiguous options, as well as defaulting to only finding fields, links and buttons by their labels or text respectively, would give us clearer, faster more easily understandable tests.

Taking it further

Requiring options such as :id works quite well for click_link and fill_in, but it works less well for select, which selects an element from a select box.

select "Programmer", :from => { :id => "profession" }
select "Programmer", :id => "profession"
select "Programmer", :from_id => "profession"

None of these options strike me as particularly elegant.

Departing from the current API even further, we could take a slice from the watir-webdriver API and instead change the API to look something like this:

text_field("Name", :exact => false).fill_in("Jonas")
button("Create", :ambiguous => false).click

This would make the API much more consistent, while perhaps sacrificing readability a little. In addition, this would be an even more radical departure from the current API, and for various reasons, we would probably not be able to provide a compatibility layer for this new style.

Please tell me what you think!

This blog post is just me thinking out loud. I haven't decided yet that these are changes that we should make, and I would love to hear what you think. Do you think these changes are necessary? Is it worth the API breakage? Should we switch over completely or work to maintain backward compatibility at least for the time being through a configuration option? Do you have any other alternatives or suggestions? Please let me know.


The Year 2012 — A Summary

It's time for the traditional "year in review" post, as I've done for the two previous years (2011 and 2010).

Nordic Ruby

As always, one of the big highlights of the year was Nordic Ruby, our annual Ruby conference here in Sweden. Moving the conference from Gothenburg to a beautiful Japanese style spa in Stockholm was a big hit. I had lots more to say in my post about Nordic Ruby 2012.

Johannes, Anders T, and Jimmy relaxing at Nordic Ruby 2012

New team members

Early this year, we welcomed two new developers to the team: Ivan Navarrete and Anders Carlsson. Ivan had been freelancing with us every now and then since the start of Elabs, and he finally agreed to join us full-time. We knew Anders C (not to be confused with Anders T) from the Ruby and Cocoa user groups, and he's been a great addition to the team.

Right before our summer vacation we also hired Robin Rundkvist, our third designer. He proved himself right away by taking charge of the design of our project for Pernod Ricard.

This brings us to 11 people total. We now have 3 designers and 3 developer pairs, a perfect balance for us.

Open source

We released several new open source projects this year. Two of the most interesting ones are Serenade.js, a JavaScript MVC framework, and Pundit, a Ruby on Rails authorization library. We also spent lots of time improving our old projects. Most notably, Jonas released Capybara 2.0.

Client projects

We've been fortunate to work on some very interesting projects for great clients this year, including Pernod Ricard, LivingSocial, Fujitsu, SEOmoz, Shotbox, and Menyou to name a few. I look forward to telling you more about some of these projects next year.


In between client projects, we also managed to find some time to finally launch our first product: ProjectPuzzle. We've been using ProjectPuzzle to keep track of our own project scheduling and staffing for a long time, but getting merchant accounts, payment gateways, etc. in order so that we could start selling subscriptions took way too long. I'm very happy that's it's finally released, and I'm excited about our plans for it next year.

What's next?

In addition to continuing working with our existing and new clients, I have three things that I'm personally very excited about working on next year.

Marketing. I've been thinking a lot lately about how we talk about who we are and what we do. We've started working on a new website that will do a much better job of explaining how we help our clients. Sign up for our newsletter, and we'll keep you posted as we work on our progress.

ProjectPuzzle. Now that we have released the first version, it's time to focus on getting more customers for it, and to start steadily improving the product. If you haven't tried ProjectPuzzle yet, you can sign up for a free trial.

Nordic Ruby. This year's Nordic Ruby was definitely the best one yet. Next year's is going to be even better. We'll release a few early tickets in January. Follow @nordicruby on Twitter so you don't miss it.

Happy New Year!

/ CJ & the Elabs team


Why wait_until was removed from Capybara

The release of Capybara 2.0.0 removed the wait_until method from the API. This seems to have frustrated a few people, so let me explain why this decision was reached and what your options are for the future.

wait_until was removed for several reasons:

  1. It simply isn't necessary for most applications and test suites.
  2. Its existence confuses people into thinking that it is necessary, when in fact it isn't.
  3. It was added at a point where Capybara's auto-waiting was much more primitive than it is now.
  4. It used to be used internally, it no longer is.

Capybara has a had a very important feature since pretty much the beginning which is that it automatically waits for elements to appear or disappear on the page. If you do something like find("#foo"), this will block until an element with id "foo" appears on the page, as will has_content?("bar"), click_link("baz") and most other things you can do with Capybara.

Let's look at a more complicated case:


Even if #foo is originally on the page and then removed and replaced with a #foo which contains baz after a short wait, Capybara will still figure this out. Let's make that really clear, Capybara is ridiculously good at waiting for content.

For the most part, this behaviour is completely transparent, and you don't even really have to think about it, because Capybara just does it for you. What trips a lot of people up is that they try to do something like this:

page.find("foo").text.should contain("login failed")

And now they have introduced potential timing issues. text, being just a regular method, which returns a regular string, isn't going to sit around and wait for anything. It will simply return the text as it appears when Capybara gets to this line and call it a day. Now after a long debugging session, our developer has found the timing issue. They now realize that there is a wait_until method in the API, and immediately think that, "hey, this sounds like what I need!"

wait_until do
  page.find("#foo").text.should contain("login failed")

Fantastic, it now works! The thing is though, Capybara could have easily figured out how to wait for this content, without you muddying up your specs with tons of explicit calls to wait_until. Our developer could simply have done this:

page.find("#foo").should have_content("login failed")

Or even:

page.should have_selector("#foo", :text => "login failed")

And the problem would have solved itself.

As long as you stick to the Capybara API, and have a basic grasp of how its waiting behaviour works, you should never have to use wait_until explicitly.


Capybara 2 introduces a new method called synchronize. While this method is part of the public API, don't run off and use it just yet. It has a very distinct use case, which is completely different from what you're probably using wait_until for. You will most likely only ever have to call this if you access the low level driver directly through #native. In this case, you might receive errors like Selenium's StaleElementReferenceError. It's these kinds of errors that synchronize prevents.

Assserting on model objects

I am firmly convinced that asserting on the state of the interface is in every way superior to asserting on the state of your model objects in a full-stack test. In some cases, especially if your interface is asynchronous, you might still want to do it though.

This is the only legitimate use case for wait_until I've heard of in Capybara.

Imagine that in the following scenario, "Liked" is shown immediately, through JS, regardless of if the change was actually persisted to the server or not:

page.should have_content("Liked")
post.reload.should be_liked

This would cause timing issues, if the AJAX request is slower than the reload. We could have written this as:

page.should have_content("Liked")
wait_until { post.reload.liked? }

Where wait_until could be implemented like this:

Though I personally would have preferred something like:

page.should have_content("Liked")
expect { post.reload.liked? }.to become_true

Which could be implemented like this:

You could have also asserted this through the UI:

page.should have_content("Liked")
visit current_path
page.should have_content("Liked")

But it is kind of verbose, and it's also a lot slower, so I can understand why the model test might be preferred.

But then why not just bundle it

If and when you need some kind of behaviour that waits for things, wait_until is a giant big sledgehammer. There are more fine grained, sophisticated tools built into Capybara, and I want you to learn about them, because those are some of the best features of the library. And when the built in tools aren't enough, there are more sophisticated tools available than that clunky hammer. So hopefully the removal of wait_until encourages you to write better tests.


Simple authorization in Ruby on Rails apps

Here at Elabs, we've been using CanCan for authorization in a number of applications. Ryan Bates managed to build an authorization system which is both simple and powerful. A step away from the bloated role based system available at the time, yet more sophisticated than simply tacking on methods on ActiveRecord models.

Over time though we've come against a few grievances with CanCan.

  • Ability files quickly become too big to manage, and there is no built in strategy for splitting up abilities across multiple files.
  • Even worse, there is no natural way to structure ability files. We usually resort to comments to divide the file into sections for different models.
  • All ability rules need to be evaluated for every request. While not a huge performance hit, it seems like a built in wastefulness.

And finally: at the time of writing, CanCan has 128 open issues, 28 open pull requests. Important functionality in the gem is broken, and attempts to fix it through pull requests are ignored. The test suite depends on ActiveRecord < 3.1 and won't even run with later versions of ActiveRecord, unless someone fixes this, we don't actually know if CanCan works at all with newer versions of AR.

In a recent project we worked on, we were running against bugs in CanCan which forced us to run a forked version, and we were fighting against an ability file which was growing out of control. We decided that we needed a new way to approach the problem.

Back to basics

We really like CanCan's simple approach. The ability file isolates all authorization logic, and it leaves you free to handle authorization however you want to. You are free to grow your authorization system from a single user role to whatever complexity you need. We were intent on keeping this flexibility.

We wanted something simpler though. Something which we can implement without really needing a library at all. We wanted to have full control over how the authorization system works.

We took inspiration from objectify and Bryan Helmkamp's excellent blog post 7 Patterns to Refactor Fat ActiveRecord Models among others and pared the whole thing down to creating a plain Ruby class for each domain model.

We call these classes policies and we put them in app/policies. They might look like this:

class PostPolicy
  attr_reader :user, :post

  def initialize(user, post)
    @user = user
    @post = post

  def create?
    user.admin? or not post.published?

Using these classes from the controller is fairly easy:

def create
  @post =[:post])
  raise NotAuthorizedError unless, @post).create?
    redirect_to @post
    render :new

This works quite nicely, but unfortunately it's a lot more code to write in the controller. Controllers are un-DRY enough as it is; we need to make this easier. It's simple enough to introduce a helper method for fetching a policy for a given record:

def policy(record)
  "#{record.class}Policy", record)

Now we can simplify our create method somewhat.

def create
  @post =[:post])
  raise NotAuthorizedError unless policy(@post).create?

We can easily wrap this pattern in another method:

def authorize(record)
  raise NotAuthorizedError unless policy(record).public_send(params[:action] + "?")

And we end up with this:

def create
  @post =[:post])

That looks a lot closer to what we have in CanCan.

This pattern works fine for 6 of the 7 restful actions, but what about #index?

CanCan can automatically construct a query based on the permissions you have specified, as long as the hash based syntax is used, anyway. Unfortunately we've found this magic to be error prone and sometimes insufficient. We really want to use scopes for this, but we don't want those to pollute our model objects. Again, taking inspiration from Bryan's blog post, we create a class for this:

class PostPolicy <, :post)
  class Scope <, :scope)
    def resolve
      if user.admin?
        scope.where(:published => true)


Usage for this from the index action is also fairly easy:

def index
  @posts =, Post.scoped).resolve

Again, we can simplify this with a helper method:

def index
  @posts = policy_scope(Post.scoped)

Both the policy and policy_scope method are especially useful in views. We can do things like this:

<% policy_scope(@category.posts).each do |post| %>
    <h2><%= post.title %></h2>
    <p><%= link_to "Edit", [:edit, post] if policy(post).edit? %></p>
<% end %>

Our views are kept quite nice and DRY, just like with CanCan.

We have bundled up these helpers in a very simple gem we're calling Pundit. It has a few more tricks up its sleeves, but basically it does exactly what this post outlines. We found that we could replace CanCan with this pattern very effectively. The resulting code is simpler, easier to understand and easier to test.


While we do think that Pundit has been useful to us, there is a bigger takeaway from this. We had a problem, and we threw a large library with a complicated DSL at the problem, and as the old saying goes, we now had two problems. Sometimes the simpler solution is better. Sometimes it makes sense to leverage Ruby, over creating your own mini language.

Spend some time reconsidering the dependencies you have in your application and whether they are actually helping you, or if you're spending more time fighting them than you're getting out.