Blog


Mar

Introducing Capybara 2.1


With the release of Capybara 2.0, we made a few changes to how Capybara acts in certain situations, which were designed to reduce unexpected and unintuitive behaviour. We got a lot of feedback that the new behaviour was too restrictive and that upgrading existing test suites from Capybara 1.x was too difficult.

Our goal with Capybara 2.1 has been to correct these problem. To provide more forgiving defaults, and more configurability for those who need it. To provide a smoother upgrade path for those on Capybara 1.x who failed to upgrade their apps due to too many breaking changes.

We focused on these key problems:

  • Matching exactly or allowing substrings
  • Behaviour when multiple elements match a query
  • Visibility
  • Asserting against the page title
  • Finding disabled elements

Aside from this, Capybara 2.1 contains many tweaks and new features.

Since we're following semver, we promise to maintain backward compatibility in all minor releases. Capybara 2.1 makes a compromise in maintaining backwards compatibilit but changing a few defaults. In order to have Capybara 2.1.0 behave identically to 2.0.x, set these options:

Capybara.configure do |config|
  config.match = :one
  config.exact_options = true
  config.ignore_hidden_elements = true
  config.visible_text_only = true
end

We have also enabled a smoother upgrade path for Capybara 1.x users. There are still changes which break compatibility between 1.x and 2.x, even with this configuration enabled, but most of them should be fairly easy to deal with. Try this:

Capybara.configure do |config|
  config.match = :prefer_exact
  config.ignore_hidden_elements = false
end

Matching exactly or allowing substrings

Capybara has always been very lenient about what it matches. Sometimes that's not what you want. To that end find, as well as all action methods like click_link and fill_in now accept an option called :exact which works together with the is expression inside the XPath gem. Without going too much into the internals, it allows you to specify whether for example click_link will allow you to specify a substring, or will need to match the entire link text exactly.

We have also added a global config option, Capybara.exact, which controls the default value of this property. Just as in Capybara 2.0.x however, this option defaults to false. That is, by default, matches are not exact, and substrings are allowed.

Behaviour when multiple elements match a query

When using metods such as click_link, and fill_in, what happens when more than one element matches? Under Capybara 1.0, we tried to find an element that matched exactly, and failing that, or if there were multiple such elements, we would simply return the first one and move on.

This has the obvious problem that sometimes, the element you end up interacting with isn't the one you expected at all.

We tried to solve this problem in Capybara 2.0 by being stricter and raising an exception in such a case instead. This was the biggest change in terms of compatibility between 1.x and 2.0 and has been very frustrating for many users.

In Capybara 2.1 we are making this behaviour configurable, and allowing users to pick which strategy they prefer, including reverting to the 1.x behaviour. We also changed the default behaviour to a slightly more lenient strategy.

We've added the match option, which takes four possible arguments: :one, :first, :prefer_exact, and :smart. Let's go through what they do:

:one is the current behaviour in Capybara 2.0.x. When two elements are found which both match the selector, a Capybara::Ambiguous error is raised.

:first is a looser behaviour which, when confronted with two elements which match the selector, simply grabs the first one and uses that one.

:prefer_exact is the behaviour present in Capybara 1.x. If multiple matches are found, some of which are exact, and some of which are not, then the first eaxctly matching element is returned.

:smart is the new default. The behaviour of :smart depends on the value of :exact. If :exact is true, the behaviour is identical to :one, that is, Capybara will perform a search for an exactly matching element, and if there is more than one, it will raise an error.

If :exact is false, things get more interesting. In that case, Capybara will first perform a search for elements which match the selector exactly, if there is exactly one, that element is returned. If there is more than one, a Capybara::Ambiguous error is raised. If no element matches, a new search is performed, allowing inexact matches. Again, if more than one element matches that search, Capybara::Ambiguous is raised.

This solves the much discussed Password confirmation problem, if there is a field with the exact label Password and another with Password Confirmation, and someone does this:

fill_in "Password", :with => "Capybara"

Then Capybara will pick the Password field and fill that in. If, however, the label of the password field would have been Password * (to indicate that it is required). Then an ambiguous error would have been raised, since both Password * and Password Confirmation are inexact matches for Password.

The idea is to strike a compromise between strictness and user friendliness. Those that prefer strictness can set:

Capybara.exact = true

Exactness of options

With Capybara 2.0 we changed the behaviour so that in the following case…

select "1", :from => "Number of people"

The option needed to match "1" exactly. So that if "10" was an option, it would be possible to differentiate between them. Options now use the same smart matching by default, as outlined above. To revert to the old behaviour of always requiring an exact match, the config options exact_options can be set to true.

Visibility

We have had an option which makes Capybara ignore all hidden elements for a long time: Capybara.ignore_hidden_elements. This option has always been false by default. This has confused people for a long time, on occasion even myself. We've made no further change to this behaviour other than the fact that this option now defaults to true. To revert to the old behaviour, simply set:

Capybara.ignore_hidden_elements = false

Visibility of text

In Capybara 1.x, the behaviour of text in the presence of invisible (display: none), DOM elements was undefined. While RackTest offers some rudimentary support for visibility, it was ignored for text, and even text in, for example, script tags was returned. Selenium ignored hidden text and other drivers did what they wanted.

In Capybara 2.0.x, we specified that text should only return text visible to the user, never hidden text, even for RackTest.

In Capybara 2.1.0, the visibility of text depends on ignore_hidden_elements. Setting ignore_hidden_elements to false means that even invisible text is returned.

We also make it possible to retrieve override this default by passing :all or :visible to the text method:

find("#thing").text           # depends on Capybara.ignore_hidden_elements
find("#thing").text(:all)     # all text
find("#thing").text(:visible) # only visible text

Since that is a departure from the Capybara 2.0.x API, we offer a special option for backward compatibility, which will make text always return only the text which is visible, even if ignore_hidden_elements is false.

Capybara.visible_text_only = true

Asserting against the page title

A lot of people migrating to Capybara 2.0 had problems with code like this:

page.should have_css("title", :text => "Whatever")

They found that title has no text, and thus this never matches

This is not however, a bug in Capybara. The above code is wrong, and should not work. To understand why, it's important to realize that the page title is quite distinct from the title element. The title element is invisible by default, and thus has no text which is visible to the user. Asking for its text very correctly returns nothing.

Don't believe me? Try pasting the following CSS into any web page:

head, head title { display: block }

You can now see the title element on the page. And making it visible this way will make the Selenium driver return the text inside that element, just as it should.

Instead of fixing Capybara to work with broken code, we are introducing a new API, which provides a nicer way of querying the page title:

page.title # => "The title"
page.has_title?("The title") # => true
page.should have_title("The title")

The has_title? and have_title matchers both have the same waiting behaviour as all other matchers in Capybara.

Finding disabled elements

Since Capybara 2.0, methods which interact with form fields and buttons, such as fill_in and click_button, do not allow interaction with disabled form elements and buttons. In addition, the matcher has_field? and the finder method find_field no longer match on disabled fields. It is still possible to locate disabled fields and buttons through other means, such as via find or has_selector?.

Capybara 2.1 makes it possible to override this behaviour by passing :disabled => true to any of these methods, which will find only disabled elements.

More

There are more new features in Capybara 2.1 which did not fit in this blog post. Please refer to the History file for a complete list.

Jan

A Capybara future


UPDATE: Some of these proposed changes have made it into Capybara 2.1. Please read our introduction to Capybara 2.1.

Over the last month or so, I've spent a lot of time thinking about the current state of Capybara.

As you may know, we released version 2.0 a while ago, which brought a number of significant changes. The biggest change, and for many the most aggravating change, was that matches now needed to be unambiguous. That is that if we try to do click_link("Remove") and there are multiple remove links on the page, we would throw an exception. I believe that this was a very good change. I have over the last years spent a lot of time tracking down issues where Capybara was simply interacting with the wrong element on the page.

The mistake we made however was that we still allowed substring matches, so that fill_in("Password", :with => "capybara"), not only matched "Password" but also "Password confirmation". This leaves us with a situation where we have to bend over backwards to avoid the ambiguity error. Clearly we should have gone all the way and disallowed substring matches.

Furthermore we should have provided a sane default, strictness, and allowed that default to be changed in specific cases, through options. For example we could have allowed:

click_link("Remove") # very strict
click_link("Remove", :exact => false) # allow substrings
click_link("Remove", :ambiguous => true) # pick any Remove link at random
click_link("Remove", :ambiguous => true, :exact => false) # 1.x behaviour

But even if we had made in hindsight those smarter choices, that still leaves us with an inherent ambiguity in what the arguments to action methods, like click_link actually do. This is what the current documentation says about fill_in:

Locate a text field or text area and fill it in with the given text. The field can be found via its name, id or label text.

This is actually not the whole truth. Fields can also be found via their placeholder, which isn't documented. When found via name, id or placeholder the match needs to be exact, via label it is allowed to be a substring.

The XPath selector we use to find fields like this is quite complicated, and consequently quite slow as well.

What if fill_in worked like this instead:

fill_in "Some label", :with => "Text"
fill_in :id => "some-id", :with => "Text"
fill_in :placeholder => "Fill in some text", :with => "Text"
fill_in :name => "some[name]", :with => "Text"

We still preserve the simplicity of the normal case of selecting fields by their label, yet we make it clear when we deviate from that. This would also allow us to compile an XPath selector specific for this scenario, which would probably be quite a bit faster.

For click_link, and click_button the default would be to find them by their text, with :id and :title being valid alternatives.

I'm thinking that combining these two changes, the addition of the exact and ambiguous options, as well as defaulting to only finding fields, links and buttons by their labels or text respectively, would give us clearer, faster more easily understandable tests.

Taking it further

Requiring options such as :id works quite well for click_link and fill_in, but it works less well for select, which selects an element from a select box.

select "Programmer", :from => { :id => "profession" }
select "Programmer", :id => "profession"
select "Programmer", :from_id => "profession"

None of these options strike me as particularly elegant.

Departing from the current API even further, we could take a slice from the watir-webdriver API and instead change the API to look something like this:

text_field("Name", :exact => false).fill_in("Jonas")
select_box("Profession").select("Programmer")
button("Create", :ambiguous => false).click

This would make the API much more consistent, while perhaps sacrificing readability a little. In addition, this would be an even more radical departure from the current API, and for various reasons, we would probably not be able to provide a compatibility layer for this new style.

Please tell me what you think!

This blog post is just me thinking out loud. I haven't decided yet that these are changes that we should make, and I would love to hear what you think. Do you think these changes are necessary? Is it worth the API breakage? Should we switch over completely or work to maintain backward compatibility at least for the time being through a configuration option? Do you have any other alternatives or suggestions? Please let me know.

Nov

Why wait_until was removed from Capybara


The release of Capybara 2.0.0 removed the wait_until method from the API. This seems to have frustrated a few people, so let me explain why this decision was reached and what your options are for the future.

wait_until was removed for several reasons:

  1. It simply isn't necessary for most applications and test suites.
  2. Its existence confuses people into thinking that it is necessary, when in fact it isn't.
  3. It was added at a point where Capybara's auto-waiting was much more primitive than it is now.
  4. It used to be used internally, it no longer is.

Capybara has a had a very important feature since pretty much the beginning which is that it automatically waits for elements to appear or disappear on the page. If you do something like find("#foo"), this will block until an element with id "foo" appears on the page, as will has_content?("bar"), click_link("baz") and most other things you can do with Capybara.

Let's look at a more complicated case:

find("#foo").click_link("baz")

Even if #foo is originally on the page and then removed and replaced with a #foo which contains baz after a short wait, Capybara will still figure this out. Let's make that really clear, Capybara is ridiculously good at waiting for content.

For the most part, this behaviour is completely transparent, and you don't even really have to think about it, because Capybara just does it for you. What trips a lot of people up is that they try to do something like this:

page.find("foo").text.should contain("login failed")

And now they have introduced potential timing issues. text, being just a regular method, which returns a regular string, isn't going to sit around and wait for anything. It will simply return the text as it appears when Capybara gets to this line and call it a day. Now after a long debugging session, our developer has found the timing issue. They now realize that there is a wait_until method in the API, and immediately think that, "hey, this sounds like what I need!"

wait_until do
  page.find("#foo").text.should contain("login failed")
end

Fantastic, it now works! The thing is though, Capybara could have easily figured out how to wait for this content, without you muddying up your specs with tons of explicit calls to wait_until. Our developer could simply have done this:

page.find("#foo").should have_content("login failed")

Or even:

page.should have_selector("#foo", :text => "login failed")

And the problem would have solved itself.

As long as you stick to the Capybara API, and have a basic grasp of how its waiting behaviour works, you should never have to use wait_until explicitly.

Synchronize

Capybara 2 introduces a new method called synchronize. While this method is part of the public API, don't run off and use it just yet. It has a very distinct use case, which is completely different from what you're probably using wait_until for. You will most likely only ever have to call this if you access the low level driver directly through #native. In this case, you might receive errors like Selenium's StaleElementReferenceError. It's these kinds of errors that synchronize prevents.

Assserting on model objects

I am firmly convinced that asserting on the state of the interface is in every way superior to asserting on the state of your model objects in a full-stack test. In some cases, especially if your interface is asynchronous, you might still want to do it though.

This is the only legitimate use case for wait_until I've heard of in Capybara.

Imagine that in the following scenario, "Liked" is shown immediately, through JS, regardless of if the change was actually persisted to the server or not:

click_link("Like")
page.should have_content("Liked")
post.reload.should be_liked

This would cause timing issues, if the AJAX request is slower than the reload. We could have written this as:

click_link("Like")
page.should have_content("Liked")
wait_until { post.reload.liked? }

Where wait_until could be implemented like this: https://gist.github.com/d8da686061f0a59ffdf7

Though I personally would have preferred something like:

click_link("Like")
page.should have_content("Liked")
expect { post.reload.liked? }.to become_true

Which could be implemented like this: https://gist.github.com/4129937

You could have also asserted this through the UI:

click_link("Like")
page.should have_content("Liked")
visit current_path
page.should have_content("Liked")

But it is kind of verbose, and it's also a lot slower, so I can understand why the model test might be preferred.

But then why not just bundle it

If and when you need some kind of behaviour that waits for things, wait_until is a giant big sledgehammer. There are more fine grained, sophisticated tools built into Capybara, and I want you to learn about them, because those are some of the best features of the library. And when the built in tools aren't enough, there are more sophisticated tools available than that clunky hammer. So hopefully the removal of wait_until encourages you to write better tests.

Nov

Simple tricks to clean up your Capybara tests


I'd like to share with you a few simple tricks we use all the time at Elabs to clean up our Capybara tests.

Often we want to find a specific area of the page, and perform some action in that area. With the imminent release of Capybara 2.0, you will probably find yourselves doing this even more, since you will sometimes have to be more specific about which element you want to interact with, since Capybara will raise an error when more than one element can be found.

You are probably familiar with the within method in Capybara, so say you want to click a link in the main menu, you might do it like this:

within "header nav" do
  click_link "Archive"
end

But in Capybara, all methods like click_link and fill_in and so on can be chained, so you could have written the above more succinctly as:

find("header nav").click_link("Archive")

That's better. However, the central idea behind Capybara is that you should test your application the way a regular user would use it; your user knows what the main menu is, but they don't really know anything about that CSS selector.

I once wrote a blog post called "You're cuking it wrong", about best practices when using Cucumber, it ended with the following general rule:

A step description should never contain regexen, CSS or XPath selectors, any kind of code or data structure. It should be easily understood just by reading the description.

While it doesn't make sense to be as strict about this ideal in plain Capybara tests, there is still a point to be made about avoiding low level detail even in these tests.

With that in mind, let's revisit that example. What if we could simply write:

main_menu.click_link("Archive")

That's much more obvious. We clearly communicate the intent, and we don't include any irrelevant details in the test. The implementation is super simple:

module MenuSteps
  def main_menu
    find("header nav")
  end
end

Now you might ask, what if I want to perform a lot of actions in a particular area of the page, this doesn't look so nice:

login_form.fill_in "Email", :with => "jonas@elabs.se"
login_form.fill_in "Password", :with => "capybara"
login_form.click_button "Login"

Thankfully, the within method can actually take as an argument an actual element, not just a selector. So we can write this instead:

within login_form do
  fill_in "Email", :with => "jonas@elabs.se"
  fill_in "Password", :with => "capybara"
  click_button "Login"
end

Concise and clear.

Arguments

Of course, since main_menu and login_form are just methods, we could create similar methods which take arguments. Imagine we wanted to delete a particular comment on our blog. We could write this:

find(".comment", :text => "Worst article ever!").click_on("Remove")

Hopefully you can see where this is going:

comment("Worst article ever").click_on("Remove")

And the implementation:

module CommentSteps
  def comment(text)
    find(".comment", :text => text)
  end
end

Much better!

Assertions

This is a bit specific to RSpec, but if you're using a different test framework, you can adapt this to your needs.

In another test for our blog, we are creating comments, and we want to assert that the comment appears as intended.

page.should have_selector(".comment", :text => "Best article ever!")

Remember our guideline/rule. That selector has to go! It would be fantastic if we could simply write:

page.should have_comment("Best article ever!")

But now we have a problem. We could create an RSpec matcher, but that would actually be a bit cumbersome. It could look like this:

module CommentSteps
  extend RSpec::Matchers::DSL

  matcher :have_comment do |text|
    match_for_should { |node| node.has_selector?(".comment", :text => text) }
    match_for_should_not { |node| node.has_no_selector?(".comment", :text => text) }
  end
end

It's important that you add both match_for_should and match_for_should_not, otherwise Capybara's waiting behaviour doesn't work properly.

This also doesn't give us particularly great error messages. We will get some variation on "expected true to be false", which isn't particular helpful.

There is a much easier way, which works well for simple cases like these:

module CommentSteps
  def have_comment(text)
    have_selector(".comment", :text => text)
  end
end

Same functionality, but easier to understand, shorter, and it even gives us better errors. We are making use of the fact that have_selector just returns an RSpec matcher.

Ensure on

One thing we face often in our tests is that we need to be on a particular page to do something. This is especially important if we have abstracted something into a helper. Consider this:

module SessionSteps
  def login
    visit login_path
    fill_in "Email", :with => "jonas@elabs.se"
    fill_in "Password", :with => "capybara"
    click_button "Login"
  end
end

But what if we already are on the login page when we call login, we'll visit the page again, which wastes valuable execution time. Let's make sure we don't do that:

module SessionSteps
  def login
    visit login_path unless current_path == login_path
    fill_in "Email", :with => "jonas@elabs.se"
    fill_in "Password", :with => "capybara"
    click_button "Login"
  end
end

We immediately spot that this pattern would be sensible to abstract. We have this in almost all of our projects, and we call it ensure_on:

module CommonSteps
  def ensure_on(path)
    visit(path) unless current_path == path
  end
end

Usage should be fairly obvious:

module SessionSteps
  def login
    ensure_on login_path
    fill_in "Email", :with => "jonas@elabs.se"
    fill_in "Password", :with => "capybara"
    click_button "Login"
  end
end

And for our final trick…

Suppose that on our blog, comments are moderated, and when someone posts a comment, we want to check that it appears in the moderation queue:

fill_in("Comment", :with => "Hi there")
click_on("Post")
visit moderation_queue_path
find(".moderation-queue").should have_comment("Hi there")

Of course we should create a moderation_queue method, like we did in the first example. But then we still need to remember that we need to actually visit that page before we fetch the element. Let's abstract that away:

module ModerationQueueSteps
  def moderation_queue
    ensure_on moderation_queue_path
    find(".moderation-queue")
  end
end

Now we can write:

fill_in("Comment", :with => "Hi there")
click_on("Post")
moderation_queue.should have_comment("Hi there")

That reveals our intent much clearer. We have practically created a small little DSL for our tests, just by creating a couple of really simple helper methods.

So that's it, some really simple tricks which allow you to write cleaner, more understandable tests with Capybara.

Feb

Capybara and testing APIs


Before you read this blog post, go read the Capybara README. Back? Good. Did you see any mention of methods like get, post or response? No? That's because those don't exist in Capybara. Let's be very clear about this, Capybara does not have those methods. So when you're writing a test like this:

visit '/'
response.body.should have_content("Hello")

Or possibly:

get '/'
page.should have_content("Hello")

Now you know why this couldn't possibly ever work. Because while visit and have_content are indeed part of the Capybara API, get and response are not. If these methods are defined inside your test/spec context, some other framework is defining them, most likely RackTest, and that framework and Capybara do not interact. Capybara knows nothing about what happens when you call get and likewise, RackTest knows nothing about what happens when you call visit.

For most people who have used Capybara for a while, this is old news. They tried variations like the above and they didn't work, they moved on and learned from those mistakes. The interesting question is why these methods do not exist in Capybara. Omitting these methods was a very conscious decision, a decision based on the following premise: Capybara is not a library suited to testing APIs.

There you have it. Do not test APIs with Capybara. It wasn't designed for it.

Capybara simulates user behaviour. Its API was designed to run against as large a variety of backends as possible. To achieve this, we have placed severe restrictions on what you can and cannot do with Capybara. We have limited the API to the most common subset of actions that a user would want to perform in a web application. This immediately precludes exposing the details of HTTP. A user knows exactly how to submit a form, but they have no idea that that form is sent as an HTTP POST request with certain HTTP headers.

I've seen so many people hack into the internals of the RackTest driver in Capybara. I want to make it clear that doing something like the following is using a private API, which is liable to change at any point (yes, even point releases):

page.driver.browser.post('/foo', data: "here")

Don't ever do this! Why would you? You could just use RackTest directly. In fact, as we saw before, it's probably already included in your spec context. So why make it harder than it needs to be:

post '/foo', data: "here"
last_response.body.should contain("Hello World")

Why would you write page.driver.browser.post when you could just write post? See that doesn't make any sense.

By extension, Capybara assumes the response to be HTML, if you're parsing page.body or page.source as JSON or XML, why not just use RackTest instead? You'll have a much nicer, cleaner API which won't break in the future.

tl;dr:

Use RackTest for testing APIs, use Capybara for testing user behaviour.